-
Notifications
You must be signed in to change notification settings - Fork 68
feat: AgentRC Readiness Scanner Web App #90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
aca6c44
feat: add webapp with report rendering, Docker support, and CI/CD
webmaxru 8df638e
fix: update Trivy action version and improve Dockerfile for backend d…
webmaxru 4fa11cb
chore: initialize frontend package with vitest for testing
webmaxru c2a6dd9
fix: update Dockerfile to ignore scripts during npm install and simpl…
webmaxru 7a23ed4
fix: harden frontend tests with dedicated vitest config and static im…
webmaxru faed0e9
fix: make Bicep secrets conditional for empty GH token
webmaxru 34eae96
fix: update achievedLevel validation to accept 0 and adjust related t…
webmaxru a2e015a
fix: enhance report validation and rendering by adding safe class han…
webmaxru 1864e37
fix: improve report validation and enhance frontend theme handling
webmaxru ce10d0a
fix: enhance share button functionality and add tooltip for better us…
webmaxru 341f209
feat: add Azure Container Registry resource and update container imag…
webmaxru 519fbf1
fix: update storage account naming convention to ensure uniqueness an…
webmaxru 017aba3
fix: update container image handling and improve resource naming conv…
webmaxru cee3191
fix: refine rate limiter to only skip OPTIONS requests and update all…
webmaxru efab3c5
fix: rename GitHub token parameter for consistency in scanning config…
webmaxru a15b5b5
fix: remove existing env storage before Bicep deploy (Container Apps …
webmaxru 9608d4c
fix: use ARM REST API for storage removal with propagation delay
webmaxru 2702d7d
fix: delete container app before storage to allow Bicep recreation
webmaxru 7406a17
fix: import GHCR image into ACR, add GHCR auth for security scan, ret…
webmaxru 4e745fd
fix: enhance report validation for areaReports and policies, add comp…
webmaxru 473e142
fix: improve error handling in SPA route and enhance report validatio…
webmaxru e3b25ef
Merge remote-tracking branch 'upstream/main' into webapp
webmaxru b157b13
feat(apm): add APM configuration checks and integrate into readiness …
webmaxru c1f84f8
feat(report-validator): enhance validation logic for pillars, levels,…
webmaxru 5aca6fc
fix: remove duplicate APM criteria already merged in main via PR #92
webmaxru aa33033
Merge upstream/main into webapp
webmaxru cebf008
fix: improve URL parsing and handle empty segments in owner/repo format
webmaxru 149b7f2
refactor: update build process and add esbuild configuration
webmaxru 2d45b23
fix(scanner): improve error handling for clone timeout and sanitize e…
webmaxru fa4574b
fix(Dockerfile): correct paths for backend files and improve director…
webmaxru 1d5176c
fix(report-validator): enhance validation and sanitization of criteri…
webmaxru 2c58d2f
feat(storage): add report cleanup functionality and integrate with se…
webmaxru f40c8bb
fix(Dockerfile): simplify directory creation and ownership setup
webmaxru 116934c
fix(bicep): enforce constraints on name prefix parameters for resourc…
webmaxru File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Dependencies (re-installed inside Docker) | ||
| node_modules | ||
|
|
||
| # Build outputs | ||
| dist | ||
| *.tgz | ||
|
|
||
| # Git / CI | ||
| .git | ||
| .github | ||
| .husky | ||
|
|
||
| # CLI source (not needed for webapp) | ||
| src | ||
| plugin | ||
| infra | ||
|
|
||
| # VS Code extension | ||
| vscode-extension | ||
|
|
||
| # Docs / examples | ||
| docs | ||
| examples | ||
|
|
||
| # Non-essential root files | ||
| eval-results.* | ||
| readiness-report.html | ||
| CHANGELOG.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| SECURITY.md | ||
| SUPPORT.md | ||
| CODEOWNERS | ||
| LICENSE | ||
| AGENTS.md | ||
| agentrc.eval.json | ||
| eslint.config.js | ||
| vitest.config.ts | ||
| tsconfig.json | ||
| tsup.config.ts | ||
| .vscode |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| --- | ||
| description: Compare CLI visual readiness report with local webapp report for a given repo, identify differences in checks/rendering/scoring, and fix them | ||
| --- | ||
|
|
||
| You are debugging consistency between two readiness report outputs for the **AgentRC** project: | ||
|
|
||
| 1. **CLI visual report** — generated by `npm run dev -- readiness --visual` from the repo root (produces an HTML file) | ||
| 2. **Webapp report** — rendered by the local dev server at `http://localhost:3000/{owner}/{repo}` | ||
|
|
||
| Both should produce identical results for the same repository because they share the same core engine (`packages/core/src/services/readiness.ts`). In practice they can diverge due to rendering differences or scoring logic bugs. | ||
|
|
||
| ## Architecture Reference | ||
|
|
||
| ### Shared Core (source of truth for checks) | ||
|
|
||
| - `packages/core/src/services/readiness.ts` — All criteria definitions, `countStatus()`, `buildCriteria()`, `buildExtras()`, pillar/level aggregation | ||
| - Criteria scopes: `repo` (always), `app` (per-application), `area` (only with `--per-area`) | ||
| - `countStatus()` **excludes** skipped checks from the denominator when computing pillar pass/total | ||
| - Extras (bonus checks) are **not scored** — they don't affect levels or totals | ||
|
|
||
| ### CLI Rendering | ||
|
|
||
| - `packages/core/src/services/visualReport.ts` — Generates the standalone HTML report | ||
| - `calculateAiToolingData()` — Aggregates AI criteria across repos (counts all including skipped in the hero display) | ||
| - Total checks: `report.pillars.reduce((s, p) => s + p.total, 0)` | ||
| - Does **not** render bonus/extras section in HTML output | ||
|
|
||
| ### Webapp Backend | ||
|
|
||
| - `webapp/backend/src/services/scanner.js` — Clones repo, calls `runReadinessReport()` from `@agentrc/core` | ||
| - `webapp/backend/src/routes/scan.js` — `POST /api/scan` endpoint | ||
| - Returns the raw `ReadinessReport` JSON (same shape as CLI) | ||
| - Uses `@agentrc/core` as a `file:../../packages/core` dependency — always uses local source code | ||
|
|
||
| ### Webapp Frontend | ||
|
|
||
| - `webapp/frontend/src/report.js` — Independent rendering implementation (NOT shared with CLI) | ||
| - `buildHero()` — Total from `report.pillars.reduce((s, p) => s + p.total, 0)` | ||
| - `buildAiToolingHero()` — Renders all AI criteria (including skipped) with pass/total count | ||
| - `buildPillarDetails()` — Shows per-pillar expandable cards | ||
| - **Does** render bonus checks section (unlike CLI) | ||
| - Has Service Information section (policy chain, engine signals) — CLI doesn't | ||
|
|
||
| ### Known Inconsistency Patterns | ||
|
|
||
| - **AI Hero vs Pillar scoring**: Both implementations count skipped checks as non-passing in the AI Hero but exclude them from pillar denominator via `countStatus()` | ||
| - **Rendering gaps**: Webapp shows bonus checks + service info; CLI doesn't | ||
| - **Icon mapping**: CLI uses HTML entities via `getAiCriterionIcon()`; webapp uses emoji via `AI_ICONS` map — new criteria IDs may get fallback icon (`🔧`) in webapp | ||
|
|
||
| ## Step-by-Step Procedure | ||
|
|
||
| ### Phase 0: Start Local Webapp | ||
|
|
||
| 1. Start the webapp backend dev server (from the repo root): | ||
|
|
||
| ``` | ||
| cd webapp/backend && npm run dev | ||
| ``` | ||
|
|
||
| This starts the Express server at `http://localhost:3000` with the local `@agentrc/core` source. | ||
|
|
||
| 2. Optionally serve the frontend for full visual testing: | ||
| ``` | ||
| cd webapp/frontend && npx vite --port 5173 | ||
| ``` | ||
|
|
||
| ### Phase 1: Generate Both Reports | ||
|
|
||
| 3. Run the CLI against the target repo to produce the visual HTML report: | ||
|
|
||
| ``` | ||
| npm run dev -- readiness --visual --repo {owner}/{repo} | ||
| ``` | ||
|
|
||
| Save the output HTML (typically `readiness-report.html`). | ||
|
|
||
| 4. Hit the local webapp API to get the raw JSON: | ||
|
|
||
| ``` | ||
| POST http://localhost:3000/api/scan | ||
| Body: {"repo_url":"https://github.com/{owner}/{repo}"} | ||
| ``` | ||
|
|
||
| Example with PowerShell: | ||
|
|
||
| ```powershell | ||
| $response = Invoke-RestMethod -Uri "http://localhost:3000/api/scan" -Method POST -ContentType "application/json" -Body '{"repo_url":"https://github.com/{owner}/{repo}"}' -TimeoutSec 120 | ||
| ``` | ||
|
|
||
| 5. Also open the local webapp page for visual comparison: `http://localhost:5173/{owner}/{repo}` (if frontend dev server is running) or `http://localhost:3000/{owner}/{repo}` (if backend serves static files). | ||
|
|
||
| ### Phase 2: Compare Data Layer | ||
|
|
||
| 6. Extract from CLI HTML: total checks, per-pillar passed/total, AI hero passed/total/percentage, criteria list with statuses, achieved level, fix-first items. | ||
|
|
||
| 7. Extract from webapp JSON: same fields. Use: | ||
|
|
||
| ```powershell | ||
| $pillars = $response.pillars | ||
| $totalPassed = ($pillars | Measure-Object -Property passed -Sum).Sum | ||
| $totalChecks = ($pillars | Measure-Object -Property total -Sum).Sum | ||
| Write-Host "Total: $totalPassed of $totalChecks checks" | ||
| Write-Host "Criteria count: $($response.criteria.Count)" | ||
| ``` | ||
|
|
||
| 8. Diff the two — check for: | ||
| - **Missing criteria** in either side (criteria list length mismatch) | ||
| - **Status mismatches** for the same criterion ID | ||
| - **Total check count** differences (pillar aggregation) | ||
| - **AI Tooling hero** percentage/label differences | ||
| - **Achieved level** and next-level calculation differences | ||
| - **Fix-first** list ordering differences | ||
|
|
||
| ### Phase 3: Compare Rendering Layer | ||
|
|
||
| 9. Compare how both renderers handle: | ||
| - Skipped checks display (icon, text, inclusion in totals) | ||
| - Bonus/extras section presence | ||
| - Pillar grouping (repo-health vs ai-setup) | ||
| - AI criterion icons for new/unknown IDs | ||
| - Score thresholds for labels (Excellent/Good/Fair/Getting Started/Not Started) | ||
|
|
||
| ### Phase 4: Root Cause & Fix | ||
|
|
||
| 10. For each difference found, classify as: | ||
| - **Rendering divergence** → Fix in either `visualReport.ts` (CLI) or `report.js` (webapp) to align | ||
| - **Scoring logic bug** → Fix in `readiness.ts` (core) which fixes both | ||
| - **Icon/label mapping gap** → Update the icon map in the affected renderer | ||
|
|
||
| 11. Implement the fixes directly in the source files. | ||
|
|
||
| 12. After fixing, restart the webapp dev server and re-run Phase 1-2 to verify alignment. | ||
|
|
||
| ### Phase 5: Validate | ||
|
|
||
| 13. Confirm both reports show identical: | ||
| - Total check count (e.g., "11 of 20") | ||
| - Per-pillar passed/total | ||
| - AI Tooling hero percentage and label | ||
| - Achieved maturity level | ||
| - Fix-first items (same set, same order) | ||
|
|
||
| 14. Note any **acceptable differences** that are by-design (e.g., webapp shows bonus checks, CLI doesn't). | ||
|
|
||
| 15. Run existing tests to ensure no regressions: | ||
| ``` | ||
| npm test | ||
| cd webapp/backend && npm test | ||
| ``` | ||
|
|
||
| ## Output Format | ||
|
|
||
| Produce a comparison table: | ||
|
|
||
| | Aspect | CLI Value | Webapp Value | Match? | Root Cause | Fix | | ||
| | ------------ | --------- | ------------ | ------ | ---------- | --- | | ||
| | Total checks | X of Y | X of Z | ... | ... | ... | | ||
| | AI Hero % | ...% | ...% | ... | ... | ... | | ||
| | ... | ... | ... | ... | ... | ... | | ||
|
|
||
| Then implement the fixes and verify. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,178 @@ | ||
| name: Webapp CD | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "webapp/**" | ||
| - "packages/core/**" | ||
| - "Dockerfile.webapp" | ||
| - "infra/webapp/**" | ||
|
|
||
| concurrency: | ||
| group: agentrc-webapp-production | ||
| cancel-in-progress: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| id-token: write | ||
|
|
||
| env: | ||
| REGISTRY: ghcr.io | ||
| IMAGE_NAME: ${{ github.repository_owner }}/agentrc-webapp | ||
| RESOURCE_GROUP: agentrc-webapp-rg | ||
| BICEP_FILE: infra/webapp/main.bicep | ||
|
|
||
| jobs: | ||
| build-push: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| image-tag: ${{ steps.meta.outputs.version }} | ||
| image-full: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Log in to GHCR | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Docker metadata | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} | ||
| tags: | | ||
| type=sha,prefix=sha- | ||
| type=raw,value=latest | ||
|
|
||
| - name: Build and push | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| file: Dockerfile.webapp | ||
| push: true | ||
| tags: ${{ steps.meta.outputs.tags }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
|
|
||
| security-scan: | ||
| runs-on: ubuntu-latest | ||
| needs: build-push | ||
| permissions: | ||
| security-events: write | ||
| packages: read | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - name: Log in to GHCR | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ${{ env.REGISTRY }} | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
| - name: Run Trivy vulnerability scanner | ||
| uses: aquasecurity/trivy-action@v0.35.0 | ||
| with: | ||
| image-ref: ${{ needs.build-push.outputs.image-full }} | ||
| format: sarif | ||
| output: trivy-results.sarif | ||
| severity: CRITICAL,HIGH | ||
| - name: Upload Trivy scan results | ||
webmaxru marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| uses: github/codeql-action/upload-sarif@v3 | ||
| if: always() | ||
| with: | ||
| sarif_file: trivy-results.sarif | ||
|
|
||
| deploy: | ||
| runs-on: ubuntu-latest | ||
| needs: [build-push, security-scan] | ||
| environment: production | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Azure Login (OIDC) | ||
| uses: azure/login@v2 | ||
| with: | ||
| client-id: ${{ secrets.AZURE_CLIENT_ID }} | ||
| tenant-id: ${{ secrets.AZURE_TENANT_ID }} | ||
| subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }} | ||
|
|
||
| - name: Ensure resource group | ||
| run: | | ||
| az group create \ | ||
| --name ${{ env.RESOURCE_GROUP }} \ | ||
| --location eastus \ | ||
| --tags application=agentrc-webapp managedBy=bicep | ||
|
|
||
| - name: Push image to ACR | ||
| run: | | ||
| ACR_NAME=$(az acr list --resource-group ${{ env.RESOURCE_GROUP }} --query "[0].name" -o tsv) | ||
| if [ -z "$ACR_NAME" ]; then | ||
| echo "ACR not yet provisioned — will be created by Bicep and image imported after" | ||
| else | ||
| az acr import \ | ||
| --name "$ACR_NAME" \ | ||
| --source ${{ needs.build-push.outputs.image-full }} \ | ||
| --image agentrc-webapp:${{ needs.build-push.outputs.image-tag }} \ | ||
| --image agentrc-webapp:latest \ | ||
| --force | ||
| fi | ||
|
|
||
| - name: Pre-deploy cleanup (Container Apps storage update limitation) | ||
| run: | | ||
| az containerapp delete \ | ||
| --name agentrc-webapp \ | ||
| --resource-group ${{ env.RESOURCE_GROUP }} \ | ||
| --yes 2>&1 || echo "Container app does not exist yet" | ||
| SUB_ID=$(az account show --query id -o tsv) | ||
| az rest --method DELETE \ | ||
| --url "https://management.azure.com/subscriptions/${SUB_ID}/resourceGroups/${{ env.RESOURCE_GROUP }}/providers/Microsoft.App/managedEnvironments/agentrc-env/storages/reportsshare?api-version=2024-03-01" \ | ||
| 2>&1 || echo "Storage mount does not exist yet" | ||
| sleep 10 | ||
|
|
||
| - name: Deploy infrastructure | ||
| uses: azure/arm-deploy@v2 | ||
| with: | ||
| resourceGroupName: ${{ env.RESOURCE_GROUP }} | ||
| template: ${{ env.BICEP_FILE }} | ||
| parameters: > | ||
| containerImageTag=${{ needs.build-push.outputs.image-tag }} | ||
| ghTokenForScan=${{ secrets.GH_TOKEN_FOR_SCAN }} | ||
|
|
||
| - name: Ensure image in ACR | ||
| run: | | ||
| ACR_NAME=$(az acr list --resource-group ${{ env.RESOURCE_GROUP }} --query "[0].name" -o tsv) | ||
| az acr import \ | ||
| --name "$ACR_NAME" \ | ||
| --source ${{ needs.build-push.outputs.image-full }} \ | ||
| --image agentrc-webapp:${{ needs.build-push.outputs.image-tag }} \ | ||
| --image agentrc-webapp:latest \ | ||
| --force | ||
|
|
||
| - name: Restart container app to pick up new image | ||
| run: | | ||
| az containerapp revision restart \ | ||
| --name agentrc-webapp \ | ||
| --resource-group ${{ env.RESOURCE_GROUP }} \ | ||
| --revision "$(az containerapp revision list --name agentrc-webapp --resource-group ${{ env.RESOURCE_GROUP }} --query '[0].name' -o tsv)" | ||
|
|
||
webmaxru marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - name: Smoke test | ||
| run: | | ||
| APP_URL=$(az containerapp show \ | ||
| --name agentrc-webapp \ | ||
| --resource-group ${{ env.RESOURCE_GROUP }} \ | ||
| --query "properties.configuration.ingress.fqdn" -o tsv) | ||
| echo "Testing https://${APP_URL}" | ||
| for i in 1 2 3 4 5; do | ||
| if curl -sf "https://${APP_URL}/api/health" | grep -q '"ok"'; then | ||
| break | ||
| fi | ||
| echo "Attempt $i failed, retrying in 15s..." | ||
| sleep 15 | ||
| done | ||
| curl -sf "https://${APP_URL}/api/health" | grep -q '"ok"' | ||
| curl -sf "https://${APP_URL}/" | grep -q "AgentRC" | ||
| echo "Smoke tests passed!" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.