fix(checkup): auto-degrade to same-family fixer when one provider family is down (#1941)#1942
fix(checkup): auto-degrade to same-family fixer when one provider family is down (#1941)#1942Serhan-Asad wants to merge 3 commits into
Conversation
…ily is down (#1941) The review loop dead-locked instead of degrading when only one AI provider family was alive. With `--reviewer codex --fixer claude --fallback-reviewer-on-failure`, a codex hard-failure promoted claude (the fixer's own role) to fallback-review the PR; when claude reported real findings the loop hit `break` with "secondary reviewer ... reported findings (fallback)" and never ran a fix round — even though a fresh same-family claude session could execute the concrete findings and be re-reviewed. Fix: in the fallback-reviewer "findings" branch of `run_checkup_review_loop`, instead of terminating, hand the actionable findings to the fixer on the next iteration (mirrors the existing fallback-clean gate-findings handoff just above), reassign the active reviewer to the fallback role, and let the normal fix + fresh-verify path run. Disclose the weaker guarantee honestly: - new `ReviewLoopState.role_independence` (`"independent"` | `"degraded (<role> unavailable)"`), rendered as a `role-independence:` report line and a `role_independence` key in final-state.json; - `same_role_review_fix` is stamped true for the degraded run; - the fix-failure stop reason names the vacancy explicitly instead of a bare "could not address". The superseded primary already renders `(optional, superseded by <fallback>)`, so the pdd_cloud verdict adapter drops it from the required-reviewer set and ships on the degraded-but-clean result — no cloud change needed. Behavior is unchanged when both families are alive. Adds regression tests: the deadlock→ship path (asserts the review-loop Machine Verdict passes + final-state disclosure), both-families-alive stays independent, and degraded-fixer-also-fails names the vacancy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review of #1942: the code change added runtime same-role degradation and a new `role_independence` state/report/final-state field, but the prompt-driven source-of-truth surfaces still described the pre-#1941 contract. Left unreconciled, `pdd sync` / the auto-heal bot would regenerate the code from the stale prompt and drop the fix. Reconcile all four surfaces with the implemented behavior: - pdd/prompts/checkup_review_loop_python.prompt: specify the auto-degrade rule under `fallback_reviewer_on_failure` (fallback-reviewer findings → fresh same-family fix + required verify instead of dead-end), the `role-independence:` report header line, the `role_independence` final-state.json key + revised `same_role_review_fix` semantics (config-time OR runtime), the appended-field ordering, and the degraded fix-failure stop-reason suffix. - architecture.json: add the #1941 auto-degrade sideEffect + `role_independence` to the final-state.json field list for run_checkup_review_loop. - README.md: document the auto-degrade behavior + disclosure under `--fallback-reviewer-on-failure`. - context/checkup_review_loop_example.py: add `role_independence` to the ReviewLoopState example, the final-state payload, the report-header comment, and the rendered-report example. No code/behavior change; docs-only reconciliation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
36665ac to
9bee527
Compare
Review responses (changes requested)Thanks — all three failure modes addressed. Failure Mode 1 — source-of-truth prompt/doc drift (BLOCKING): fixed. Reconciled every generated/authored surface with the implemented behavior so a
(Note: the auto-heal bot pushed a heal commit based on the code-only commit before the prompt was reconciled, so it regenerated the docs to the pre-#1941 contract. I force-pushed the #1941-aware prompt so a re-heal derives from the correct source.) Failure Mode 2 — cloud adapter disclosure: addressed via companion PR promptdriven/pdd_cloud#3126. The Failure Mode 3 — degrade with a live independent |
…gnostics (#1941) Review hardening for #1942: the cloud verdict adapter reads a `role-independence:` header line to route a degraded run to `ship_degraded`, but untrusted reviewer stderr rendered in `### Reviewer Diagnostics` could echo `role-independence: independent` and — since the adapter previously took the last match in the whole report — override the authoritative header, silently downgrading a degraded disclosure back to a plain ship (or the reverse). Add `_defang_role_independence_inline` to `_defang_adapter_trip_wires` so any `role-independence:`/`role_independence=` marker leaking from reviewer stderr is neutralized at the render boundary, exactly like the existing `reviewer-status:` / `fresh-final-review:` / `max-*-reached:` markers. The on-disk `final-state.json` and `reviewer_status_details` keep the original text. Prompt spec (27b) updated to list the new marker. Companion adapter-side defense (header-scoped parse) is in promptdriven/pdd_cloud#3126. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes #1941.
Root cause
The checkup review loop deadlocked instead of degrading when only one AI provider family was alive. With the cloud's config
--reviewer codex --fixer claude --fallback-reviewer-on-failure:_maybe_run_fallback_reviewerpromotes claude — the fixer's own role — to review the PR (this is the only fallback reviewer available).breakat the_maybe_run_fallback_reviewer"findings" branch with stop reason "secondary reviewer claude reported findings (fallback)" — no fix round ever ran, because the only fixer (claude) was now the active reviewer and same-role fixing was blocked.The findings were legitimate (the gate worked); the self-heal was structurally impossible. Result on both staging PRs:
same-role-review-fix: false,max-rounds-reached: false, terminalreview_findings_remain.Note the asymmetry that made this a bug: the
reviewer_fallbackpath (a distinct 3rd reviewer) falls through to a fix round, but the_maybe_run_fallback_reviewerpath (fixer-as-reviewer) terminated.Fix
In the fallback-reviewer "findings" branch of
run_checkup_review_loop, instead of terminating, auto-degrade: hand the actionable findings to the fixer on the next iteration (this mirrors the pre-existing fallback-clean gate-findings handoff a few lines above —pending_findings = ...; continue), reassign the active reviewer to the fallback role, and let the normal fix + fresh-verify path run a fresh same-family session.Disclosed honestly:
ReviewLoopState.role_independence:"independent"|"degraded (<role> unavailable)"— rendered as arole-independence:report line and arole_independencekey infinal-state.json;same_role_review_fixstampedtruefor the degraded run;Fixer claude could not address claude's findings [role-independence degraded (codex unavailable)].) instead of a bare "could not address".The superseded primary already renders
(optional, superseded by <fallback>)(becausestate.active_reviewerpoints at the fallback), so the pdd_cloud verdict adapter drops it from the required-reviewer set and ships on the degraded-but-clean result — verified against the real adapter; no cloud change needed (this is exactly what the issue asked the CLI to make automatic + disclosed).Behavior is unchanged when both families are alive (this path only triggers on primary-reviewer failure).
Acceptance criteria
same-role-review-fix: true+role-independence: degraded (codex unavailable).Tests
tests/test_checkup_review_loop.py:test_issue_1941_auto_degrades_to_same_family_fixer_on_fallback_findings— the deadlock→ship path; asserts the review-loop Machine Verdict passes (ships) plusfinal-state.jsondisclosure fields.test_issue_1941_both_families_alive_stays_independenttest_issue_1941_degraded_same_family_fixer_failure_names_vacancyFull
test_checkup_review_loop.pysuite: 306 passed. Broadertest_agentic_checkup.py/test_checkup_report.py/commands/test_checkup.py: 91 passed, 1 skipped. pyflakes + pylint(E) clean on the changed file.Known limitations (out of scope, documented)
pending_findings(layer1_step5_evidence, epic-only) + provider down at verify time — still terminates; this change is neutral there (it neither helps nor worsens), and fixing it means relaxing the deliberately-disabled verify-path independence guard. Separate concern.fixer_fallbackis configured alongsidefallback_reviewer_on_failure, the degrade runs the configured primary fixer (same-family) first and only consults the independent fallback if it fails; disclosure then errs conservative (claims a weaker guarantee than reality). Reordering to prefer a live independent fallback would be a policy change beyond Checkup review loop deadlocks when one provider family is down: auto-degrade to same-family fixer with report disclosure instead of terminating with findings #1941.max_rounds >= 2(prod default is 5).Reviewer notes
Independently adversarially reviewed (2 rounds) — approved, no blockers, no new defects. (codex-work was dead pool-wide during this work — the exact failure this PR fixes — so a Claude Opus reviewer stood in.)
🤖 Generated with Claude Code