QA hardening: 16 bug fixes + measured coverage/mutation/fuzz gates#13
Merged
Conversation
…s (A–P) Root-cause fixes from a 10-cycle adversarial QA pass; each has a regression test that fails before the fix and passes after. See QA_LOG.md for per-bug repro/evidence/root-cause. Core engine & spec: - validator: linear termination on recursive $ref + cyclic values; conjunctive composition keywords (allOf/anyOf/oneOf no longer skip sibling constraints); array/union types honored (BUG-1/3/4/5/C) - mock: collapse adjacent path params (O(n^k) ReDoS -> linear); clamp response status to a valid final range 200–599 (writeHead crash + 1xx client hang); guard the request handler so a bad request can't crash the server (BUG-B/H/K) - scaffold: dedup colliding filename slugs; non-empty name fallback (BUG-F/J) - runner: insert query params before any #fragment; redirect: manual so a 3xx is observable/contract-validatable (BUG-A/L) - importers/schema: empty-name fallback on import; .strict() on nested schema objects so typo'd keys surface (BUG-E/G) - discovery: per-file error isolation so one malformed file doesn't abort the whole listing (BUG-I) Web server & UI: - CSRF/clickjacking: require Origin to match Host + X-Frame-Options/CSP; blocks cross-site AND cross-port loopback requests (BUG-M/N) - a11y: document-level editor keyboard shortcuts; aria-labels + sr-only h1 + landmark (BUG-O/P) - XSS: confirmed React-escaped end to end (invariant test) CLI: - run: exit non-zero when zero requests are found, so a misconfigured CI path fails loudly instead of going green (BUG-D)
Drive the suite to objective thresholds and persist the state under qa/ so re-runs are cumulative instead of redundant (INVENTORY, COVERAGE, TRIED, SEEDS, corpus). No new product bugs — this exercises surfaces the suite left untested. Coverage 86.93%->95.18% lines, 79.06%->86.52% branch (+69 tests): - VS Code extension 0->90%+ via a `vscode` module mock (activate, all commands, CodeLens) — previously an entirely untested public surface - cli/serve.ts and web/src/api.ts 0->covered - CLI command error/--json/--live/--min branches; output formatters; Postman importer variants; mock generateExample/pickResponse - exclude pure-type format/types.ts (no runtime code) Mutation (Stryker 9.6.1) on the contract validator -> 85.4% (300 killed): added exact-message/path/edge tests to kill survivors (no-coverage 37->1). Fuzz: committed seeded, deterministic property-fuzz (5 invariant targets) + qa/corpus crash regressions, replacing the campaigns' scratch harnesses. CI gates: - coverage thresholds enforced in vitest.config (lines>=90, branches>=85) - pnpm gen:schema no-diff check (catches runtime/contract drift) - weekly Stryker mutation job (break threshold 80)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A QA hardening pass that (1) fixes 16 root-caused bugs found by adversarial testing, and (2) makes the test suite measurably strong and resumable. Two commits, separated by intent.
1. Bug fixes (A–P) —
4dd1b06Each fix has a regression test that fails before and passes after. Full per-bug repro/evidence in
QA_LOG.md.validate-response.ts): linear termination on recursive$ref/cyclic values; conjunctive composition keywords; array/union types (BUG-1/3/4/5/C)#fragment;redirect: manualso 3xx is observable (BUG-A/L).strict()nested objects (BUG-E/F/G/J)X-Frame-Options/CSP — blocks cross-site and cross-port loopback (BUG-M/N)runexits non-zero on zero requests found (BUG-D)2. Measured QA + CI gates —
7d1c791vscodemodule mock.qa/corpus/crash regressions.qa/so re-runs are cumulative (INVENTORY,COVERAGE,TRIED,SEEDS,corpus).gen:schemano-diff check, weekly Stryker job (break ≥80).Behavior changes (flagged for review)
truspec runnow exits non-zero when zero requests are found (was 0). Matches jest/pytest "no tests found" convention.redirect: manual), so a 3xx is observable and contract-validatable.Verification
312 tests pass, typecheck 7/7, build 5/5, coverage gate green, JSON Schema in sync. The README's documented blog flow runs exactly as advertised in the built binary.