Skip to content

Bundle: checkup resilience — auto-degrade, target-repo prompt paths, test adoption (#1941, #1957, #1903)#1962

Open
Serhan-Asad wants to merge 13 commits into
mainfrom
bundle/checkup-resilience
Open

Bundle: checkup resilience — auto-degrade, target-repo prompt paths, test adoption (#1941, #1957, #1903)#1962
Serhan-Asad wants to merge 13 commits into
mainfrom
bundle/checkup-resilience

Conversation

@Serhan-Asad

Copy link
Copy Markdown
Collaborator

Bundle: checkup resilience

Off current main, rolls up three independent pdd checkup resilience fixes plus the CodeQL containment follow-up into one deployable branch.

Constituent changes

Closes #1941
Closes #1957
Closes #1903

Note

#1906 is superseded by #1914's approach and will be closed at merge time (left open for now).

Validation

Local suites pass on the bundle branch: test_checkup_review_loop, test_issue_1903_adopt_collocated_test, test_content_selector, test_sync_determine_operation — 641 passed, 1 skipped.

🤖 Generated with Claude Code

Serhan-Asad and others added 9 commits July 7, 2026 22:48
…cal test path (#1903)

PDD derived a module's test-output path purely from `.pddrc`/defaults
(`tests/test_{name}{ext}`), blind to the project's real test runner. On a
jest/Next.js project it maintained a root `tests/` shadow while the co-located
`__test__/*.test.tsx` the runner collects went stale — `generate`/`change`/`sync`
reported "tests pass" (PDD ran its own shadow) while CI failed. A `.tsx` module
also derived a `.ts` test extension.

PDD now adopts a single, unambiguous, existing co-located test as its canonical
test path so generate/change/sync target the real test CI runs:

- content_selector: pure `find_collocated_test` (Python siblings + JS/TS
  jest/vitest/Next.js conventions; single match else None; suffix-gated; never
  raises), the `resolve_test_output_path` adopter, and presence-based
  explicit-vs-default provenance (`_pins_test_output_location` /
  `configured_test_output_pinned`).
- sync_determine_operation: `get_pdd_file_paths` wraps the raw derivation and
  adopts across every return branch, threading a single authoritative
  explicit-vs-default signal read from the raw `.pddrc` context (never the
  `resolved_config` construct_paths pollutes).
- cmd_test_main: adopts for direct `pdd test`, keeps the native/cloud generation
  destination in sync with the adopted write target, and pins sync's
  merge-into-existing write to the real test (no numbered `page.test_1.tsx`
  shadow).
- code_generator_main: `_is_test_output_path` recognizes adopted `.test/.spec`
  `.mjs/.cjs` and the singular `__test__` dir, so adopted human tests stay behind
  the TestChurnError guard.

Explicit user pins (CLI `--output`, `PDD_TEST_OUTPUT_PATH`, explicit `.pddrc
test_output_path`/`outputs.test.path`) are always honored; the standard Python
`src/foo.py -> tests/test_foo.py` flow is unchanged. Regression tests drive the
real `construct_paths`/`architecture.json`.

Source-of-truth: the four owning `prompts/*_python.prompt` files
(content_selector, sync_determine_operation, cmd_test_main, code_generator_main)
are updated to describe this behavior so PDD's regeneration / drift-heal
reproduces it (in this repo the `.py` is generated from the prompt).

Adoption is applied consistently across the write paths AND the
`pdd test --evidence` manifest resolver, so evidence/replay audits point at the
file PDD actually writes, not the shadow.

Part of #1903 (scoped: adopts an existing co-located test; greenfield
runner-config detection and never-block-on-churn are tracked as follow-ups
#1916 / #1917).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ily is down (#1941)

The review loop dead-locked instead of degrading when only one AI provider
family was alive. With `--reviewer codex --fixer claude
--fallback-reviewer-on-failure`, a codex hard-failure promoted claude (the
fixer's own role) to fallback-review the PR; when claude reported real
findings the loop hit `break` with "secondary reviewer ... reported findings
(fallback)" and never ran a fix round — even though a fresh same-family claude
session could execute the concrete findings and be re-reviewed.

Fix: in the fallback-reviewer "findings" branch of `run_checkup_review_loop`,
instead of terminating, hand the actionable findings to the fixer on the next
iteration (mirrors the existing fallback-clean gate-findings handoff just
above), reassign the active reviewer to the fallback role, and let the normal
fix + fresh-verify path run. Disclose the weaker guarantee honestly:

- new `ReviewLoopState.role_independence` (`"independent"` |
  `"degraded (<role> unavailable)"`), rendered as a `role-independence:` report
  line and a `role_independence` key in final-state.json;
- `same_role_review_fix` is stamped true for the degraded run;
- the fix-failure stop reason names the vacancy explicitly instead of a bare
  "could not address".

The superseded primary already renders `(optional, superseded by <fallback>)`,
so the pdd_cloud verdict adapter drops it from the required-reviewer set and
ships on the degraded-but-clean result — no cloud change needed. Behavior is
unchanged when both families are alive.

Adds regression tests: the deadlock→ship path (asserts the review-loop Machine
Verdict passes + final-state disclosure), both-families-alive stays
independent, and degraded-fixer-also-fails names the vacancy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review of #1942: the code change added runtime same-role degradation
and a new `role_independence` state/report/final-state field, but the
prompt-driven source-of-truth surfaces still described the pre-#1941 contract.
Left unreconciled, `pdd sync` / the auto-heal bot would regenerate the code
from the stale prompt and drop the fix.

Reconcile all four surfaces with the implemented behavior:
- pdd/prompts/checkup_review_loop_python.prompt: specify the auto-degrade rule
  under `fallback_reviewer_on_failure` (fallback-reviewer findings → fresh
  same-family fix + required verify instead of dead-end), the `role-independence:`
  report header line, the `role_independence` final-state.json key + revised
  `same_role_review_fix` semantics (config-time OR runtime), the appended-field
  ordering, and the degraded fix-failure stop-reason suffix.
- architecture.json: add the #1941 auto-degrade sideEffect + `role_independence`
  to the final-state.json field list for run_checkup_review_loop.
- README.md: document the auto-degrade behavior + disclosure under
  `--fallback-reviewer-on-failure`.
- context/checkup_review_loop_example.py: add `role_independence` to the
  ReviewLoopState example, the final-state payload, the report-header comment,
  and the rendered-report example.

No code/behavior change; docs-only reconciliation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…gnostics (#1941)

Review hardening for #1942: the cloud verdict adapter reads a
`role-independence:` header line to route a degraded run to `ship_degraded`,
but untrusted reviewer stderr rendered in `### Reviewer Diagnostics` could echo
`role-independence: independent` and — since the adapter previously took the
last match in the whole report — override the authoritative header, silently
downgrading a degraded disclosure back to a plain ship (or the reverse).

Add `_defang_role_independence_inline` to `_defang_adapter_trip_wires` so any
`role-independence:`/`role_independence=` marker leaking from reviewer stderr is
neutralized at the render boundary, exactly like the existing
`reviewer-status:` / `fresh-final-review:` / `max-*-reached:` markers. The
on-disk `final-state.json` and `reviewer_status_details` keep the original text.

Prompt spec (27b) updated to list the new marker. Companion adapter-side
defense (header-scoped parse) is in promptdriven/pdd_cloud#3126.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#1957)

The checkup source-of-truth guards reconstructed each module's owning
prompt path by string-joining a hardcoded `pdd/prompts/` prefix onto the
`filename` stored in `architecture.json`. That prefix only ever resolved
on pdd's own self-hosted checkout (prompts live under `pdd/prompts`, with
a `prompts -> pdd/prompts` symlink). On every OTHER repo the guard probed
a nonexistent path, classified the owning prompt as `missing_prompt`, and
refused to repair — blocking the bug→fix→checkup chain on all external
repos (observed on staging E2E against promptdriven/test_repo #4234/#4235).

`_load_prompt_source_map` and `_extract_arch_pairs` now resolve the base
prompts root via `_resolve_target_prompts_root(worktree)`, which reuses
the same `.pddrc` discovery/parse helpers generate/sync use
(`construct_paths._find_pddrc_file` / `_load_pddrc_config`) and falls back
to `prompts` then `pdd/prompts`, picking the first existing directory and
resolving symlinks to the real git-tracked path. A prompt is only reported
deleted when it is missing from the target repo's resolved root.

Owning prompt `pdd/prompts/checkup_review_loop_python.prompt` updated to
match (10a/10b mapping rules + new resolver spec).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-resilience

Auto-degrade to same-family fixer when one provider family is down (#1941).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…esilience

Resolve owning-prompt paths against the repo-under-repair layout, not pdd's self-hosted pdd/prompts (#1957).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…p-resilience

Adopt the runner-collected co-located test as PDD's canonical test path (#1903).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… CodeQL)

CodeQL flagged 4 py/path-injection alerts (#339-#342) on the new adoption
code: candidates derived from a caller-supplied code-file path — issue-
influenced in the hosted/agentic flow — flowed into resolve()/write-target
selection without containment, so a traversal-shaped input could adopt a
test path outside the repo.

find_collocated_test now rejects candidates that resolve outside cwd, and
both adoption sinks (resolve_test_output_path, _adopt_collocated_test)
re-assert containment before retargeting. Traversal inputs degrade to
no-adoption (derived path unchanged). Adds 4 containment regression tests;
5 existing tests updated to run with cwd=the project like the rest of the
suite (matches real CLI invocation).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread pdd/content_selector.py Fixed
Comment thread pdd/content_selector.py Fixed
Comment thread pdd/sync_determine_operation.py Fixed
Comment thread pdd/sync_determine_operation.py Fixed
@Serhan-Asad

Copy link
Copy Markdown
Collaborator Author

Staging E2E evidence (2026-07-09, staging1, stack/e2e-staging-v2 = 288d053)

  • pdd#1957 fix validated live: checkup layer1 (project discovery, dependency audit, build check, cross-module interface check) ran green on an external repo (test_repo#4248/#4250), resolving owning prompts from the target repo's layout — the exact operation that previously concluded "owning prompt deleted" and refused repair on any non-self-hosted repo.
  • checkup_roles waterfall reorder confirmed in executor logs: Reordered credential waterfall (source=checkup_roles, model=claude) put Claude lanes first, codex last.
  • Full checkup E2E (layer2 review loop, fix(checkup): auto-degrade to same-family fixer when one provider family is down (#1941) #1942 auto-degrade path) could not be exercised yet: every layer1 run was killed by a newly-discovered platform bug — interactive Claude launches hang on Claude Code's workspace-trust dialog (pdd_cloud#3194, fix in flight tonight) — unrelated to this bundle's content. Checkup re-runs follow once #3194 is deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants