Skip to content

[codex] add Judge QA delivery gates - #66

Merged
greenthree merged 1 commit into
mainfrom
codex/judge-qa-cli
Aug 6, 2026
Merged

[codex] add Judge QA delivery gates#66
greenthree merged 1 commit into
mainfrom
codex/judge-qa-cli

Conversation

@greenthree

Copy link
Copy Markdown
Owner

Summary

  • add the judge-qa CLI with content-addressed compile caching while rerunning every fixture and probe
  • require current passed Judge QA evidence in seal and formal build checkpoints
  • expose bounded evidence state in lint, status, and report
  • document the Schema v1 workflow and Agent delivery requirements
  • strip Validator, Checker, and Interactor diagnostic bodies from evidence and retain only bounded byte summaries

Why

P0 defined the fixture schema and P1 added isolated execution, but authors still had no supported CLI or formal delivery gate. This change makes active Checker/Interactor testing part of the Schema v1 sealing workflow without changing WebUI or Legacy behavior.

User impact

New or modified custom/interactive problems can run probhub judge-qa <ID> --no-cache. Existing workspaces without judge.qa remain compatible. Failed, cancelled, stale, or invalid QA runs do not replace the last successful evidence or formal PDF/ZIP/Manifest artifacts.

Validation

  • npm run check: 663 passed, 30 skipped
  • Judge QA/build focused regression: 80 passed
  • npm run pack:check: 218 main-package files, 5 compatibility-package files
  • Skill validation and git diff --check
  • local-only phtest nine-problem lint
  • L02 no-cache Judge: 33 inputs, 3 solutions, 99 case executions, all_expectations_met

Stacked on Draft PR #65 (codex/judge-qa-execution).

@greenthree
greenthree changed the base branch from codex/judge-qa-execution to main August 6, 2026 04:49
@greenthree
greenthree marked this pull request as ready for review August 6, 2026 04:49
@greenthree
greenthree merged commit d2f9a9d into main Aug 6, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant