feat: transactional fingerprints and deterministic mock validation (#1926, #1939)

[Serhan-Asad]

Summary

This consolidates the full work from #1937 and the stacked #1939 follow-up into one main-targeting PR.

• add one shared FingerprintTransaction for fingerprint calculation, validation, serialization, and atomic persistence across every mutating PDD path
• fail closed when required finalization cannot resolve paths, hash artifacts, clean stale reports, buffer state, or persist safely
• preserve complete example/test path sets and write nested-project metadata beneath the correct .pddrc root
• add deterministic, diff-aware Python AST validation for generated mock collection/query fields and payload shapes
• ground mock validation in exact schema evidence and independent production readers/writers
• reject contradictory generated mocks before persistence or agentic commit/push, while treating missing contracts as inconclusive
• retain the broader Step 9 mock API audit as defense in depth
• include synchronized prompts, architecture metadata, examples, fingerprints, and run reports for both changes

Issue #1926: transactional fingerprint finalization

• centralizes artifact-write ⇒ fingerprint-write enforcement in pdd/fingerprint_transaction.py
• routes sync, generate, example, update, auto-deps, fix, CI heal, metadata sync, operation logging, and the legacy pin-example path through the transaction
• uses same-filesystem atomic replacement and typed FingerprintFinalizeError propagation
• adds the CI-collected post-command freshness harness
• includes regressions for CI auto-heal can loop forever because example writes null fingerprints and heal_update keeps regenerating examples #1305 null hashes and pdd fix from a parent dir writes orphan files to run cwd instead of the subproject #1211/Follow-up to #1211: save_fingerprint doesn't thread project_root, e2e tests don't catch the bug #1290 wrong-root writes

Issue #1939: deterministic schema/mock contract validation

• detects supported Python query/mock structures syntactically
• validates them only against exact resource contracts or independent production evidence
• excludes changed paths, baseline aliases, tests, vendors, and generated trees from self-certification
• emits evidence-linked MOCK_CONTRACT_DIVERGENCE failures for real contradictions
• restores loop-mode inputs exactly when an in-place candidate is rejected
• gates all terminal agentic success routes before the initial commit/push and re-checks Step 11 cleanup

Verification

• exact GitHub unit-test matrix: 12,940 passed, 31 skipped, 1 xfailed
• Transactional fingerprint finalization: enforce 'artifact write ⇒ fingerprint write' in one code path #1926 focused fingerprint/sync/update/auto-deps/metadata/CI-heal regressions: 777 passed
• pdd fix can generate tests that mock away the bug it introduced (green tests certify broken code); validate generated mocks against the real interface/schema contract #1939 deterministic and workflow regressions: 34 passed
• validator source set: 22 passed, 92% coverage
• fix_main source set: 51 passed, 2 skipped, 72% coverage
• agentic orchestrator source set: 422 passed, 85% coverage
• public CLI and story regressions: passed
• architecture/include validation, compilation, JSON parsing, pylint, and git diff --check: passed
• branch already contains current origin/main at 49504fc9a

Cross-repository companion

The related app-page routing fix lives in a different repository and therefore remains a separate PR: promptdriven/pdd_cloud#3251.

Together these coordinated PRs address:

• Transactional fingerprint finalization: enforce 'artifact write ⇒ fingerprint write' in one code path #1926
• pdd fix can generate tests that mock away the bug it introduced (green tests certify broken code); validate generated mocks against the real interface/schema contract #1939
• promptdriven/pdd_cloud#3024

Closes #1926
Closes #1939

Related to promptdriven/pdd_cloud#3024

[Serhan-Asad]

Verification for commit 0a19b126:

• Orchestrator, pdd fix can generate tests that mock away the bug it introduced (green tests certify broken code); validate generated mocks against the real interface/schema contract #1939 regression, and legacy Step 9 suites: 397 passed.
• Prompt, cross-step, and changed-module contract suites: 43 passed.
• Focused Step 9/mock-contract batch: 11 passed.
• New regression pylint: 10.00/10.
• Python compile and git diff --check: pass.
• The live #4235 fixture proves the failure mode: production queries nonexistent userId, a generated mock fabricates it, and MOCK_CONTRACT_MISMATCH now overrides an otherwise green test claim.
• Primary, retry, and cached-resume Step 9 paths all enforce the same exact evidence markers.

The host aggregate exposed the known nested-pytest -11 macOS baseline; a representative failure reproduces unchanged on origin/main. Remote Linux CI is the authoritative broad run.

[Serhan-Asad]

Final readiness verification (2026-07-10):

• All GitHub checks are green, including Run Unit Tests, auto-heal/heal, Public CLI Regression, Story Regression, Package Preprocess Smoke, Repo Bloat Docker E2E, CodeQL, and all analysis lanes.
• Focused orchestrator suites: 422 passed.
• Measured current-module coverage: 78.43%.
• Exact PR auto-heal drift detector after manual code↔prompt reconciliation: TOTAL=0.
• Fresh fingerprint and run-report hashes were committed for the reviewed implementation.
• PR is mergeable and ready for campaign-stack integration.

No merge to main was performed.

[Serhan-Asad]

Final verification (latest stacked head)

Verified head: d80299b89c691baebede4afedba36b2a633b6455

• This PR is intentionally stacked on fully green parent PR feat: transactional fingerprint finalization via FingerprintTransaction (#1926) #1937 (change/issue-1926).
• Fresh auto-heal and heal checks passed on this exact head.
• Exact GitHub unit-test command: 12,940 passed, 31 skipped, 1 xfailed.
• Deterministic/workflow pdd fix can generate tests that mock away the bug it introduced (green tests certify broken code); validate generated mocks against the real interface/schema contract #1939 regressions: 34 passed.
• Combined final-tree pdd fix can generate tests that mock away the bug it introduced (green tests certify broken code); validate generated mocks against the real interface/schema contract #1939 focused suite in the manager stack: 424 passed, 0 failed.
• Validator source set: 22 passed, 92% coverage.
• fix_main source set: 51 passed, 2 skipped, 72% coverage.
• Agentic orchestrator source set: 422 passed, 85% coverage.
• Public CLI, Story regression, architecture/include validation, compile, pylint, and git diff --check: passed.

This exact head is integrated into manager bundle PR #1998 at combined head cc45f4f1. No merge to main was performed here.

[Serhan-Asad]

Codex review pass 1

Reviewed full head SHA: 2fd9d71a6bfa5668dcf4d146dafea0d412636c06
Base reviewed: main at 49504fc9aa11a9ee70539c93a6f1afe64badad94

Intent

This PR consolidates the transactional-fingerprint work from #1926 and the deterministic mock-contract work from #1939: every mutating PDD command is meant to finalize a complete, non-null fingerprint atomically and fail the command if finalization is unsafe, while pdd fix and the agentic E2E-fix workflow are meant to reject newly introduced Python query/mock pairs that contradict exact repository schema or independent production evidence before those changes can be persisted or pushed.

Failure modes

1. [P1] Re-run the mock-contract gate after CI remediation, before its commit/push.

   • Trigger: The initial fix passes the gate at pdd/agentic_e2e_fix_orchestrator.py:3564, but a later Step 10 CI remediation edits a production query and its test mock to use the same nonexistent field.
   • Failure: run_ci_validation_loop commits and pushes every non-artifact remediation file through _commit_ci_fix (pdd/ci_validation.py:1692) before control returns, and the orchestrator never invokes _validate_changed_mock_contracts again after the call at pdd/agentic_e2e_fix_orchestrator.py:3704. A green mock can therefore be pushed to the PR and the workflow can return success even though the latest head violates the contract this PR is intended to enforce.
   • Evidence: pdd/agentic_e2e_fix_orchestrator.py:3564, pdd/agentic_e2e_fix_orchestrator.py:3704, pdd/ci_validation.py:1692.

2. [P1] Diff mock occurrences by location/association, not only by global field-name count.

   • Trigger: A changed test replaces one pre-existing, unrelated {"userId": ...} mock payload with one new user_waitlist mock payload using the same userId key; the total occurrence count for userId remains one. The production user_waitlist.userId query already exists and is unchanged.
   • Failure: new_mock_names is empty because current and baseline counters are keyed only by field name. validate_changed_files consequently never loads the unchanged reader, returns not_applicable, and allows the newly fabricated query/mock pairing to pass. I reproduced exactly this case against an exact user_waitlist schema that omits userId; the report was not_applicable with zero findings.
   • Evidence: pdd/mock_contract_validation.py:625-647 and pdd/mock_contract_validation.py:812-827.

3. [P1] Recognize pytest's *_test.py naming pattern as a test path.

   • Trigger: A generated test is written as backend/reader_test.py, outside a directory literally named test or tests.
   • Failure: Pytest collects this file by default, but _is_test_path classifies it as production because it only recognizes names beginning with test_ (plus JS-style .test./.spec.). The validator therefore extracts no mock fields and returns not_applicable, allowing a contradictory mock to bypass the terminal gate. Focused reproduction: pytest collected backend/reader_test.py::test_collected, while validate_changed_files reported not_applicable, zero queries, zero mock fields, and zero findings.
   • Evidence: pdd/mock_contract_validation.py:513-518 and pdd/mock_contract_validation.py:797-805.

Performance

I scanned the new validator and its agentic/fix call sites for unbounded loops, subprocess/network/disk I/O in hot paths, added complexity, allocations, and event-loop blocking. The only notable scaling cost is the deliberate repository-wide rglob("*.py") plus file reads/AST parsing in validate_changed_files (pdd/mock_contract_validation.py:521-532, 833-855), with a possible second repository scan for sibling evidence (543-565, 682-683). This is O(repository Python source) and synchronous, but it runs at workflow validation boundaries rather than in a request handler/event loop, and the sibling scan is skipped when schema evidence already allows every candidate. I did not find an additional release-blocking performance regression.

Focused verification completed: the 66 changed mock-contract/fingerprint tests passed; direct reproductions confirmed failure modes 2 and 3; changed production modules compile; git diff --check passes; the worktree remains clean.

Verdict: changes requested

[Serhan-Asad]

Codex review pass 1 remediation

Addressed all three findings from the pass 1 review in commit b0c664e2e1121ec0b43886f6e0e475342a747d26:

1. Added a fail-closed pre_commit_check for Step 10 CI remediation and local pre-checkup remediation, plus a final recheck before drift-sync commit/push. A schema-divergent remediation can no longer be committed or pushed after the terminal gate.
2. Made mock diffing resource-aware by comparing (resource, field) associations rather than global field-name totals. Replacing an unrelated same-named mock now exposes the new association and loads unchanged readers for validation.
3. Recognized pytest-valid *_test.py files in validator classification and agentic test discovery, including committed-file and fallback scans.

Also synchronized prompts, architecture metadata, and fingerprints.

Verification:

• 424 agentic orchestrator tests passed
• 139 mock-contract / CI / focused orchestrator tests passed
• 110 architecture / prompt / fingerprint tests passed
• compilation, JSON parsing, fingerprint hash validation, and git diff --check passed

[Serhan-Asad]

Codex review pass 2

Reviewed full head SHA: b0c664e2e1121ec0b43886f6e0e475342a747d26
Base reviewed: main at 49504fc9aa11a9ee70539c93a6f1afe64badad94

Intent

This PR consolidates #1926 and #1939: all mutating PDD paths should finalize complete, non-null fingerprints through one atomic, fail-closed transaction, while pdd fix and the agentic E2E-fix workflow should deterministically reject newly introduced Python query/mock contracts that contradict exact schema or independent production evidence. The pass-2 remediation additionally intends to recognize pytest *_test.py files, compare new mocks by resource/field association rather than global field totals, and rerun the mock-contract gate before every CI/pre-checkup remediation commit or push.

Failure modes

1. [P1] File-level resource co-occurrence still misses a reassigned mock when both resource names remain in the file.

   • Trigger: A test file already contains both "user_waitlist" and "other_collection" and one {"userId": ...} payload for mock_other. A fix changes that payload to mock_waitlist while retaining both resource strings and the same one-key count; the existing production query is user_waitlist.userId.
   • Failure: _mock_association_counts takes the Cartesian product of every resource substring in the whole file and every extracted mock field, without using the mock target or function scope. Baseline and current therefore both contain (user_waitlist, userId), so new_mock_pairs is empty and the gate returns not_applicable. I reproduced this exact reassignment against an exact user_waitlist schema that omits userId; it produced zero findings.
   • Evidence: pdd/mock_contract_validation.py:594-607 and pdd/mock_contract_validation.py:647-663.

2. [P1] Nested mock payloads cannot establish dotted query-field associations.

   • Trigger: Production already queries the dotted field user_waitlist.profile.userId, and a changed test newly returns {"profile": {"userId": "fabricated"}} for that resource.
   • Failure: _dict_fields flattens the payload into the two bare names profile and userId; it never emits profile.userId. The unchanged-reader loader and new_mock_pairs therefore cannot match the production query field, and validation returns not_applicable even when the exact schema omits profile.userId. The focused reproduction returned mock fields [profile, userId], zero findings, and not_applicable.
   • Evidence: pdd/mock_contract_validation.py:240-249, pdd/mock_contract_validation.py:604-606, and pdd/mock_contract_validation.py:650-663.

3. [P1] Generated/vendor Python can still self-certify a contradictory field as sibling evidence.

   • Trigger: The repository contains generated/client.py with the same invalid user_waitlist.userId query, then a changed production query/test mock introduces that pair while the exact schema omits userId.
   • Failure: _SKIP_DIRS does not exclude generated, vendor, or vendors, so _source_files treats that code as independent production evidence. sibling_allows then overrides the exact schema omission and returns clean. I reproduced this with generated/client.py; the report was clean, and its contracts explicitly contained sibling evidence from that generated file. This contradicts the PR description’s stated generated/vendor-tree exclusion and lets generated copies certify the bug they duplicate.
   • Evidence: pdd/mock_contract_validation.py:35-47, pdd/mock_contract_validation.py:527-538, and pdd/mock_contract_validation.py:709-715.

Performance

I scanned the complete validator/fingerprint diff and the remediated CI, pre-checkup, cleanup, and terminal-success call paths for unbounded loops, subprocess/network/disk I/O in hot paths, complexity changes, tight-loop allocations, lock contention, and event-loop blocking. The notable cost remains the synchronous repository-wide rglob("*.py"), file reads, and AST parsing in validate_changed_files and _repository_evidence (pdd/mock_contract_validation.py:527-570, 844-866). Pass 2 can invoke that O(repository Python source) work before each CI/pre-checkup remediation commit and again before the final drift-sync push. These are workflow boundaries rather than request/event-loop hot paths, and the sibling scan is skipped when schema evidence already allows candidates, so I did not identify a separate release-blocking performance regression beyond the correctness findings above.

Focused verification completed: 31 remediation/mock-contract tests passed (484 deselected); direct reproductions confirmed all three failure modes; changed production modules compile; git diff --check passes; the dedicated worktree remains clean. The pass-1 pre-commit remediation callback and *_test.py fixes themselves are correctly wired and covered by the focused tests.

Verdict: changes requested

[Serhan-Asad]

Codex review pass 2 remediation

Addressed all three findings from the pass 2 review in commit 82538ca65878344b951333c08519950af6a40e4c:

1. Mock association identity now includes resource, field, source path, mock target, and qualified lexical scope. Reassigning a payload between mocks in a multi-resource test module is detected even when both resource strings and global field counts remain unchanged.
2. Nested mock dictionaries now preserve qualified paths such as profile.userId, allowing them to match dotted production query fields.
3. generated, vendor, and vendors trees are excluded from repository sibling evidence so copied/generated readers cannot certify a contradiction against the exact schema.

Prompts, architecture metadata, and fingerprints were synchronized again.

Verification:

• 41 direct mock-contract regression/wiring tests passed
• 144 mock-contract / CI / focused orchestrator tests passed
• 110 architecture / prompt / fingerprint tests passed
• compilation, JSON parsing, fingerprint hash validation, and git diff --check passed