Skip to content

fix: resolve unrecoverable incarnation state (#584)#585

Open
peterbraden wants to merge 3 commits into
mainfrom
fix/584-unrecoverable-incarnation-state
Open

fix: resolve unrecoverable incarnation state (#584)#585
peterbraden wants to merge 3 commits into
mainfrom
fix/584-unrecoverable-incarnation-state

Conversation

@peterbraden

@peterbraden peterbraden commented May 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Resolves the deadlock described in #584 where incarnations become permanently stuck after force-pushes leave the latest change with commit_pushed=False and a nonexistent commit SHA.

  • Self-healing mutation path: get_latest_change_for_incarnation_if_completed now catches IncompleteChange and runs the existing update_incomplete_change repair logic. Only the latest change can be in this state (creating a new change requires the previous one to be complete), so a single retry suffices. All mutation paths (PUT, PATCH, direct changes) automatically recover.

No schema changes, no new endpoints, no API contract changes.

The stale branch collision (root cause 2 from #584) is addressed separately in #586.

Test plan

  • Regression tests reproducing the force-push deadlock scenario (red/green verified)
  • Regression test verifying GET reports broken state without mutating, then mutation recovers
  • All existing tests pass unchanged

🤖 Generated with Claude Code

peterbraden and others added 2 commits May 21, 2026 11:48
Three tests reproducing the unrecoverable incarnation state described in
issue #584. Marked xfail until the fix is implemented:

- Committed change with missing SHA (force-push scenario): update_incomplete_change
  short-circuits on commit_pushed=True, leaving a broken change as the latest.
- Multiple broken changes: repairing each change individually still leaves
  the incarnation pointing at a nonexistent commit.
- Stale branch from closed MR: deterministic branch name collides with an
  orphaned remote branch, blocking new change creation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four tests reproducing the unrecoverable incarnation state described in
issue #584. Marked xfail until the fix is implemented:

- Single broken change: after force-push, creating a new direct change
  should self-heal by invalidating the broken change and succeeding.
- Multiple broken changes: same recovery across a chain of invalid commits.
- GET reports broken state without mutating: verifies GET returns UNKNOWN
  for broken incarnations, and that recovery only occurs via mutation.
- Stale branch: a branch from a closed MR should not block new changes.

Tests verify both the mutation (create_change_direct) and the read path
(get_incarnation_with_details) — no /fix endpoint needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@peterbraden peterbraden requested a review from defreng as a code owner May 21, 2026 10:13
@peterbraden peterbraden marked this pull request as draft May 21, 2026 10:13
@peterbraden peterbraden changed the title Draft: Fix #584 - unrecoverable incarnation state fix: resolve unrecoverable incarnation state (#584) May 21, 2026
@peterbraden peterbraden force-pushed the fix/584-unrecoverable-incarnation-state branch from 206ae00 to f948c9e Compare May 21, 2026 10:33
@peterbraden peterbraden marked this pull request as ready for review May 21, 2026 10:52
@peterbraden

Copy link
Copy Markdown
Contributor Author

From review discussion:

  • Split into 2 PR's
  • We likely don't need while: True, as the theory is, only the last revision can be broken.

@peterbraden peterbraden force-pushed the fix/584-unrecoverable-incarnation-state branch from f948c9e to d8b85bd Compare June 2, 2026 11:45
get_latest_change_for_incarnation_if_completed now catches IncompleteChange
and runs the existing update_incomplete_change repair logic (which deletes
changes whose commits no longer exist in git). Only the latest change can
be in this state, since creating a new change requires the previous one to
be complete — so a single retry is sufficient.

This means PUT, PATCH, and direct changes automatically recover from broken
state without manual intervention.

No schema changes, no new endpoints, no API contract changes.

Closes #584

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@peterbraden peterbraden force-pushed the fix/584-unrecoverable-incarnation-state branch from d8b85bd to 922af21 Compare June 2, 2026 11:48
@peterbraden

Copy link
Copy Markdown
Contributor Author

Branch name split out to #586

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants