Skip to content

fix(agent-edit-v2): detect live ref drift against persisted base not live fallback#62

Open
devYRPauli wants to merge 2 commits into
EveryInc:mainfrom
devYRPauli:fix/agent-edit-v2-live-ref-drift-baseline
Open

fix(agent-edit-v2): detect live ref drift against persisted base not live fallback#62
devYRPauli wants to merge 2 commits into
EveryInc:mainfrom
devYRPauli:fix/agent-edit-v2-live-ref-drift-baseline

Conversation

@devYRPauli

Copy link
Copy Markdown

Problem

POST /api/agent/:slug/edit/v2 could silently clobber a concurrent human edit and apply an agent's replace_block to the wrong block, returning 200 success instead of rejecting.

Scenario (covered by the existing agent-edit-v2-live-structural-drift-regression test):

  1. An agent snapshots a document and captures a block ref (e.g. b3 = "Second paragraph.").
  2. A live collaborator inserts a block before that block, so the live fragment shifts ("Second paragraph." moves to b4), without bumping the persisted revision.
  3. The agent sends replace_block on its captured b3 with baseRevision = the snapshot revision.

Expected: 409 FRAGMENT_DIVERGENCE (the block topology drifted under the agent's ref).
Actual: 200 - the ref resolved positionally to the now-shifted b3 (the human's new block) and replaced it, discarding the concurrent insert.

Root cause

The ref-drift guard findLiveRefDrift(persistedDescriptors, liveDescriptors, ...) is meant to compare the persisted base (what the agent's refs were captured against) with the live authoritative fragment. But both sides were sourced from the same value:

  • liveDescriptors come from authoritativeBase.base.markdown (the live fragment).
  • persistedDescriptors come from doc.markdown, where doc = await getCanonicalReadableDocument(slug, 'state') ?? getDocumentBySlug(slug).

Under live presence, getCanonicalReadableDocument(..., 'state') returns the live (yjs) fallback, so doc.markdown is the live fragment too. The guard therefore compared the live fragment against itself and never detected drift - exactly when it is needed.

Instrumented confirmation: with a live client connected, persistedDescriptors and liveDescriptors were identical (both the 5-block live fragment, including the human insert), while the real persisted row (getDocumentBySlug) still held the 4-block base the agent's b3 was captured against.

Fix

Source the drift baseline from the real persisted document row for baseRevision-only edits (the frame the agent's refs were actually captured against). When a baseToken is supplied, the agent explicitly opted into the live authoritative base (already validated above), so that base remains the comparison frame and behavior is unchanged.

const driftBaselineMarkdown = baseToken
  ? stripEphemeralCollabSpans(doc.markdown ?? '')
  : stripEphemeralCollabSpans(getDocumentBySlug(slug)?.markdown ?? doc.markdown ?? '');
const persistedBase = parseMarkdownWithHtmlFallback(parser, driftBaselineMarkdown);

Verification

  • src/tests/agent-edit-v2-live-structural-drift-regression.test.ts: fails on main (200), passes with this change (409, the human insert is preserved and the persisted revision is unchanged).
  • No new failures across the related agent-edit / edit-v2 regression suites; baseToken-based edits are unaffected (no spurious FRAGMENT_DIVERGENCE).
  • No new type errors (tsc -p tsconfig.server.json unchanged).

…live fallback

findLiveRefDrift compared the live authoritative fragment against a "persisted" base that was itself sourced from the live-fallback canonical read (doc.markdown from getCanonicalReadableDocument). Under live presence those are the same fragment, so the guard compared the live fragment against itself and never detected drift, exactly when it is needed.

When an agent edits against a baseRevision (no baseToken) and a concurrent client inserts a block before the agent's referenced block, the positional b-ref resolved to the now-shifted wrong block: replace_block silently clobbered the concurrent human insert and landed on the wrong content, returning 200 success.

Source the drift baseline from the real persisted document row (getDocumentBySlug) for baseRevision-only edits, the frame the agent's refs were captured against. When a baseToken is supplied the agent opted into the live authoritative base directly, so that base stays the comparison frame (behavior unchanged). Now returns 409 FRAGMENT_DIVERGENCE and preserves the concurrent edit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 48a7abde2d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread server/agent-edit-v2.ts Outdated
// (validated above), so that base is already the correct comparison frame.
const driftBaselineMarkdown = baseToken
? stripEphemeralCollabSpans(doc.markdown ?? '')
: stripEphemeralCollabSpans(getDocumentBySlug(slug)?.markdown ?? doc.markdown ?? '');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not reject freshly refreshed live refs

This makes every baseRevision-only edit compare refs against the persisted row whenever the authoritative base is live. However /snapshot//state can build the advertised bN refs from the live Yjs fallback while still returning the row revision, so a client that refreshes after a live insert and posts that fresh snapshot.revision (e.g. persisted A/B/C, live A/X/B/C, then replace fresh b3) will now compare live b3 to persisted b3 and get FRAGMENT_DIVERGENCE forever unless it switches to baseToken. Since the endpoint still accepts baseRevision and the snapshot exposes it, this rejects valid fresh live-state edits rather than only stale refs.

Useful? React with 👍 / 👎.

The drift baseline re-read the document row with a second getDocumentBySlug
call, several awaits after doc was fetched and revision-validated against
baseRevision. That second read was not itself revision-checked, so a
concurrent write landing in between returned a newer row than the frame the
agent's refs were validated against, and the newer content could be compared
as if it were the base - a spurious FRAGMENT_DIVERGENCE on an otherwise valid
edit.

Only use the re-read row when its revision still matches baseRevision, and
fall back to the already-validated doc otherwise. The baseToken path is
unchanged.

Signed-off-by: Yash Raj Pandey <yashpn62@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@devYRPauli

Copy link
Copy Markdown
Author

The bot's P2 on the drift baseline was a real find - fixed in d7dadc8.

The baseline re-read the row with a second getDocumentBySlug(slug) call several awaits after doc was fetched and revision-validated against baseRevision. That second read wasn't itself revision-checked, so a concurrent write landing in the window returned a newer row than the frame the agent's refs were validated against, and that newer content could then be compared as if it were the base - a spurious FRAGMENT_DIVERGENCE on an otherwise valid edit.

Now the re-read row is only used when its revision still matches baseRevision, falling back to the already-validated doc otherwise. The baseToken path is unchanged.

One note on the bot's framing: "permanently reject" overstates it - a retry would pick up the new revision and succeed. It's a spurious rejection, not a permanent one.

I didn't add a regression test: reproducing this deterministically needs a timing seam to inject a concurrent write between the two reads, and the current harness has no such hook - a test without it would assert on shape rather than behavior. The existing agent-edit-v2-live-structural-drift-regression test still passes, and 7 of the 8 agent-edit-v2* suites pass locally (agent-edit-v2-live-viewer-regression fails identically on a clean tree without this change, so it's pre-existing and unrelated). Happy to add a test if you'd like the seam introduced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant