Skip to content

Enforce comprehensive migration HTML reports with post-write validation#78

Open
herosjourney wants to merge 12 commits into
awslabs:mainfrom
herosjourney:feat/comprehensive-migration-report
Open

Enforce comprehensive migration HTML reports with post-write validation#78
herosjourney wants to merge 12 commits into
awslabs:mainfrom
herosjourney:feat/comprehensive-migration-report

Conversation

@herosjourney

@herosjourney herosjourney commented Jun 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Expands generate-artifacts-report.md so migration-report.html includes full appendices (combined infra+AI TCO, security baseline components, gap analysis, assumptions, architecture diagram, effort hours) — not JSON stubs
  • Adds scripts/validate-migration-report.py post-write gate: section IDs, TOC integrity, appendix depth, readability rules, and artifact-driven checks
  • Adds reference fixtures (migration-report-reference.html, estimation-*-reference.json) plus inverse stub (migration-report-stub.html) for CI regression
  • Readability pass on reference report: no numbered headings, security teaser in exec flow, <details> mapping rationale, verdict banner, glossary, accessible tables
  • Validator gates (when estimation JSON is passed): exec-security-teaser required when security_baseline exists; verdict banner when recommendation exists; --migration-dir fixture-bleed detection on real runs
  • Adds an exec-flow reader-vocabulary gate (frontend-design skill): executive sections (decision-summary / exec-*) must not expose artifact filenames (*.json) or Terraform resource IDs (aws_*.*) — those names stay in the technical appendices. Clarifies the numbered-heading ban targets decorative labels only; genuine sequences (cluster order, phases, weeks, rollback) keep their numbering

Problem

Generate-phase agents often produced executive-summary-only HTML with appendices that only linked to JSON. Partners manually mined estimation-infra.json for GuardDuty costs and line-item breakdowns that were never rendered. The reference fixture also had exec-flow readability issues (patched section numbering, full security table before timeline, leaked Rubric: lines).

Solution

Change Purpose
Report spec + readability conventions Mandate TCO, security teaser, verdict, anti-stub appendices
Post-write validator Automated structural + readability gate after HTML write
Reference + stub fixtures Golden pass target and deliberate fail target for CI
--migration-dir on real runs Catch verbatim copies of SF Beach fixture into other migrations

Test plan

  • python3 migrate/plugins/migration-to-aws/scripts/validate-migration-report.py fixtures/migration-report-reference.html --estimation-infra fixtures/estimation-infra-reference.json --estimation-ai fixtures/estimation-ai-reference.json → REPORT_OK
  • Same command on fixtures/migration-report-stub.html → REPORT_FAIL (11+ actionable errors)
  • pytest migrate/plugins/migration-to-aws/tests/test_validate_migration_report.py -q → 28 passed
  • Re-run Generate on a test migration; confirm $MIGRATION_DIR/migration-report.html passes with --migration-dir

@cursoragent
Enforce comprehensive migration reports with post-write validation.
f0f6df8 
Expand generate-artifacts-report.md with combined TCO, security baseline
component costs, gap analysis, and assumptions sections; add an HTML validator
script, reference fixture, and tests so stub appendices fail validation instead
of shipping as complete reports.

Co-authored-by: Cursor <cursoragent@cursor.com>
@herosjourney herosjourney requested a review from a team as a code owner June 17, 2026 16:35
Logan Kleier and others added 11 commits June 17, 2026 09:41
@cursoragent
Address Kiro review: TOC integrity, strict section checks, safer defa…
e0341f0 
…ults.

Fix broken TOC anchors in the reference fixture; validate nav links match
section ids; require exactly one <section id> per required block; scope
GuardDuty checks to cost/security appendices; gate exec-tco on both estimate
JSON files; default to rename-on-fail; expand test coverage to 11 cases.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Close Kiro gaps: reference+JSON contract, dollar security check, REPO…
b927862 
…RT_OK scope.

Validate reference HTML with estimation-infra/ai fixture JSON; replace dead
GuardDuty regex with component dollar matching; document that REPORT_OK means
structure complete not numerically audited; expand tests to 14 cases.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Document migration-report.html and validator in plugin README.
8e734e8 
Add Generate-phase report overview, REPORT_OK scope, pytest/validator
commands, and fix stale skill/docs paths in Development section.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Remove machine-specific stub test; tighten dollar matching.
35c196d 
Replace the hardcoded SF Beach path with STUB_FAIL pytest coverage, add a
committed stub fixture for manual CLI validation, and use word-boundary
regex in _dollar_amount_present so $13 does not match $130.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Remove SF Beach provenance labels from fixture docs.
696148c 
Drop project-specific structural-reference wording from the PR-facing
README and reference HTML footer.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Enforce migration report readability in validator and reference fixture.
db23aef 
Add readability gates (no Rubric:/Section N headings), rewrite the reference
report with security teaser + appendix split, and document conventions in the
spec so generated reports stay exec-friendly.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Add security teaser, verdict, and fixture-bleed validator gates.
b73462f 
Require exec-security-teaser when security_baseline exists, a verdict banner
when recommendation exists, and --migration-dir canary checks on real runs;
refresh the stub regression fixture and wire --migration-dir into Generate Step 4.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Clarify BigQuery requires specialist path, not migration block.
51d361a 
Reframe Stay if, risk severity, deferred callout, and estimate/report spec
so phased infra+AI migration proceeds while analytics target is evaluated
in parallel with AWS account team or data partner.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursoragent
Fix CloudWatch free-tier copy and apply allowance-aware cost formula.
5a5726b 
Correct observability messaging (5 GB logs / 10 metrics / 10 alarms always-free vs GCP's larger tier), subtract allowances in Step 4, and recascade SF Beach reference totals ($118→$112 infra, ~$503 combined savings).

Co-authored-by: Cursor <cursoragent@cursor.com>
docs(migrate): label cost figures as estimated monthly costs
3b545a7 
Ensure user-facing cost output consistently uses 'estimated monthly
costs' phrasing across the estimate phase summaries, report template,
README, and report fixture. Adds an explicit cost-labeling rule to the
estimate orchestrator and each sub-estimate Present Summary.
feat(migrate): enforce reader-facing vocabulary in migration report e…
43287dc 
…xec flow

Apply Anthropic frontend-design skill principles to the migration report:

- "Name things by what people control, not how the system is built":
  add a post-write gate so executive-flow sections (decision-summary,
  exec-*) cannot expose artifact filenames (*.json) or Terraform resource
  IDs (aws_<resource>.<name>). Those identifiers stay in the technical
  appendices, which remain exempt. Gated under --no-readability.
- "Structure is information": clarify the numbered-heading ban targets
  decorative labels only — genuine sequences (cluster order, phased weeks,
  migration phases, rollback steps) keep their numbering.
- "One name per concept": require a single consistent label for the
  recommended model and cost tier across verdict, tables, and appendices.

Updates the reference fixture to reader-facing copy in exec-tco and
exec-security-teaser, documents check awslabs#14 + the sequence nuance in
validate-migration-report.md and generate-artifacts-report.md, and adds
4 tests (28 pass).
@herosjourney herosjourney mentioned this pull request Jun 29, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@herosjourney