Person C: blue-team gap fixes, dashboard, demo, docs (C1–C11)#1
Open
ezzy1630 wants to merge 5 commits into
Open
Person C: blue-team gap fixes, dashboard, demo, docs (C1–C11)#1ezzy1630 wants to merge 5 commits into
ezzy1630 wants to merge 5 commits into
Conversation
Audit of Person C deliverables C1-C8 against the spec acceptance criteria surfaced six gaps; all fixed test-first: - C1 repro_writer: add a dedicated "Confidence and Caveats" section (replay reliability, root-cause certainty, evidence basis). - C4 triage: add the missing repro_rate term to the priority formula (severity_weight * blast_radius * repro_rate / fix_complexity). - C5 patch_generator: add expected_tests + confidence to PatchCandidate and the generation prompt/parser. - C6 test_generator: add the third test type, a policy regression test that catches silent bypasses (behavior blocked, telemetry missing). - C7 patch_verifier: expand 3 gates to 6 — diff-applies, control-plane weakening detection (7 checks), and telemetry-evidence gate. - C8 regression_runner: surface flaky (oscillating) tests. Interface changes are additive optional fields only (PatchCandidate, RegressionTestInput, RegressionRunResult) — backward-compatible with Person A's schema. 152 tests pass (137 prior + 15 new). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The dashboard previously rendered four panels. The spec requires eight views covering the full red-to-blue lifecycle: - Overview, coverage heatmap, finding timeline, repro packages, blue team (patches + regression tests), evidence timeline, search intelligence, and cost/model stats. Each view is a read-only query function with a matching /api/* endpoint. The page now builds its DOM with textContent rather than innerHTML, so LLM-authored KB text cannot inject markup. Adds test/test_dashboard.py — 10 tests over a Database-bootstrapped fixture plus the FastAPI app (empty and seeded DB). 162 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The demo must be robust even without live model APIs or a live NemoClaw target (spec C10): - demo/seed_demo_db.py — builds a deterministic, fully-populated KB (cycles, findings, repro packages, patches, regression tests, alerts). Findings are drawn from the General Analysis "Securing Coding Agents" adversarial corpus + detection catalog, mapped onto MonkeyClaw's 18 zones. - demo/run_hackathon_demo.sh — one-command runner with fallback mode (seeded DB, no credentials) and live mode (one real cycle); live degrades to fallback when NVIDIA_API_KEY is absent. - demo/README.md, docs/demo_script.md, docs/judge_quickstart.md. Two new tests cover the seed (dashboard-ready + idempotent). 164 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- README: add "why continuous red/blue testing", a text architecture diagram, a demo section, and a project-status section (what works / what is mocked / what production adds). Correct the blue-team description to six verifier gates + three regression tests, and the test count (164). - docs/pitch_script.md: problem -> insight -> demo flow -> architecture -> why this wins -> future path. - docs/zone_failure_class_mapping.md: the 18 attack zones mapped to the failure classes, controls, and detections from the General Analysis "Securing Coding Agents" guide. Verified: 164 tests pass, ruff clean, dashboard serves all 8 endpoints over HTTP against the seeded demo DB. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements Person C's deliverables (C1–C11) from
docs/spec_person_c_repro_blue_demo.md. Branched off the shared baseline2ca46dc(pushed here asperson-c-base) so this PR shows only Person C's work — it does not include Person A's in-progressinfra/WIP.Summary
An audit of the existing blue-team pipeline (C1–C8) against the spec's acceptance criteria found six gaps; all were closed test-first. The dashboard (C9), demo assets (C10), and final docs (C11) were then built.
164 tests pass · ruff clean · dashboard verified over HTTP.
C1–C8 — blue-team gap fixes (
713b08c)## Confidence and Caveatssection (replay reliability, root-cause certainty, evidence basis).repro_rateterm to the priority formula (severity_weight × blast_radius × repro_rate / fix_complexity).expected_tests+confidencetoPatchCandidateand the generation prompt/parser.Interface changes are additive optional fields only (
PatchCandidate,RegressionTestInput,RegressionRunResult) — backward-compatible with Person A's schema, no DB migration.C9 — dashboard (
fb0ed61)Expanded
infra/dashboard.pyfrom 4 to the 8 spec views (overview, coverage heatmap, finding timeline, repro packages, blue team, evidence timeline, search intelligence, cost/model). Each view is a read-only query function with a matching/api/*endpoint. The page now builds its DOM withtextContentinstead ofinnerHTML, so LLM-authored KB text cannot inject markup. Addstest/test_dashboard.py(12 tests).C10 — demo assets (
5e08ad1)demo/seed_demo_db.py— deterministic, fully-populated KB seeded from the General Analysis "Securing Coding Agents" adversarial corpus + detection catalog.demo/run_hackathon_demo.sh— one-command runner; fallback mode needs no credentials, live mode runs one real cycle.demo/README.md,docs/demo_script.md,docs/judge_quickstart.md.C11 — docs (
a0900b0)docs/pitch_script.mdanddocs/zone_failure_class_mapping.md(18 zones ↔ whitepaper failure classes).Verification
uv run pytest— 164 passeduv run ruff check— cleandemo/seed_demo_db.pyis idempotentNotes for reviewers
person-c-baseis the shared committed baseline (2ca46dc) — review only the 5 commits on top.Argyphwas evaluated for integration and intentionally not wired in: it is redundant with MonkeyClaw's existinginfra/codebase_indexer.py+search_codebase, and would cross into Person A'sinfra/ownership.🤖 Generated with Claude Code