Skip to content

fix(daemon): say why a no-effect claim was withheld - #1655

Merged
thymikee merged 2 commits into
mainfrom
claude/1620-veto-instrument
Aug 6, 2026
Merged

fix(daemon): say why a no-effect claim was withheld#1655
thymikee merged 2 commits into
mainfrom
claude/1620-veto-instrument

Conversation

@thymikee

@thymikee thymikee commented Aug 6, 2026

Copy link
Copy Markdown
Member

Closes #1620. Also supplies the live-evidence artifact carried over from #1615.

The finding

#1620's premise does not hold: the warning fires. Measured on the seeded Bluesky fixture and on Settings, built from this branch. Neither hypothesised veto reproduces — the backend flip never occurs on a fully-hostile screen (the tree strategy fails before the first capture, so every capture including the baseline is already private-ax, and there is nothing to flip), and truncation drift did not appear in a single sample: every veto observed had onlyInBaseline: 0, onlyInCurrent: 0. Membership is stable under the remembered depth cap; only rects ever diverged.

So this PR does not change the corroboration rule. It closes the reason the question stayed open for weeks: a withheld claim was indistinguishable from a gesture that simply worked — both render as a response with no warning. Every veto on an accept-stale verdict now says why.

The artifact (#1615 carry-over)

The exact repro from #1620's description, on this head:

$ agent-device swipe 200 250 200 650      # drag down while already at top
Flung

$ agent-device snapshot
swipe 200 250 200 650 produced no visible change: the tree still matches its pre-gesture state.
Either the container is already at its edge, or it ignores synthesized scrolls —
a raw drag moves such lists: swipe x1 y1 x2 y2 (start inside the list).

$ agent-device scroll down                # session remains usable
Scrolled down

$ agent-device snapshot
Snapshot: 167 visible nodes (181 total) (truncated)

Coordinates are retained, which is #1615's contribution — before it, the numeric-token heuristic reduced this to a contentless swipe produced no visible change. That is the full condition #1615 was held on: a no-effect swipe warning retaining its coordinates, followed by a successful command proving the session remains usable.

What the instrument showed

post_gesture_snapshot_stale_accept  action=swipe   attempts=12  durationMs=3755  matchedPreGestureBaseline=true
post_gesture_no_effect_vetoed       action=swipe   backend=tree  reason=surface_divergence
                                    onlyInBaseline=0  onlyInCurrent=0  rectMismatched=1  shared=103

That is a correct veto, and it is only legible because of this PR: on Settings the drag rubber-banded and left exactly one rect of 103 shifted past tolerance. A control run — two idle snapshots, no gesture — showed 0 moved rects, confirming the screen is stable and the movement was gesture-caused rather than ambient. The other veto seen (Bluesky, rectMismatched: 4 / shared: 158) came from feed content updating across a deliberately unrealistic 105-second gesture→capture gap.

Both are the rule working as specified. Neither was diagnosable before.

Changes

  • post_gesture_no_effect_vetoed on every withheld claim, with reason: baseline_rebased | surface_divergence, and for divergence the counts above via a new pure summarizeDiscriminatingSurfaceDivergence — one-sided keys are membership drift, rectMismatched is movement. This is what produced every number in this PR.
  • Marking no longer stores an empty baseline signature. [] is truthy, so a session with no pre-gesture snapshot had an invented baseline that the loop would then rebase onto a post-gesture capture. "No usable baseline" now has one representation — absent — matching markPendingInteractionOutcome's existing rule.
  • Claim-surface tests split into post-gesture-no-effect-claim.test.ts (fix(daemon): distrust post-gesture stability that matches the pre-gesture baseline #1563 convention), keeping the loop file at 525 lines. #1615's updated wording assertions move with them.

No behaviour change to the verdict or the corroboration bar.

Verification

typecheck / lint / format:check / check:layering green. Daemon suite 95 files / 711 tests. The five directly-related suites total 60 tests.

check:affected --run is green except the documented provider-integration contention flake — three consecutive runs produced three different timeout sets ({android-lifecycle}, {android-lifecycle, android-recording}, {doctor}), each passing in isolation. All are Android; this change cannot reach Android at all, since requiresPostGestureBaselineDistrust is Apple-only and every new path is behind an accept-stale verdict that needs a baseline.

Correction to my earlier comment on #1620

I attributed some earlier missing diagnostics to daemon takeover, using daemon_startup in the client's --debug stream as the tell. That was wrong: daemon_startup is emitted on essentially every client request, including ones where the warning fired normally. The real discriminator in the corrupted samples was ios_runner_session_invalidated — the XCTest runner dying mid-capture (** TEST EXECUTE FAILED ** in daemon.log), which loses the pending record. Cadence still matters when measuring, but for that reason rather than the one I gave.

A vetoed gestureNoEffect claim is invisible from outside: the response
looks exactly like a gesture that worked. #1620 spent weeks unable to
tell a cross-backend pair from capture drift from real movement for
precisely this reason, and its own suggested next step was to log the
corroboration inputs and re-run.

Every veto on an accept-stale verdict now emits
post_gesture_no_effect_vetoed with a reason: baseline_rebased (the pair
is cross-backend, #1569) or surface_divergence carrying
onlyInBaseline/onlyInCurrent/rectMismatched/shared from a new
summarizeDiscriminatingSurfaceDivergence — one-sided keys are membership
drift, rectMismatched is movement.

Measured live against the seeded Bluesky fixture (167 nodes, depth-56,
private-ax): inert scrolls corroborate and the warning fires 2/2,
successful gestures never claim 2/2, zero rebases, and the one veto
observed was rectMismatched:4 shared:158 onlyIn*:0 from ambient feed
motion over an artificial 105s gap. #1620's truncation-drift half does
not reproduce; membership is stable under the remembered depth cap.

Also: marking no longer stores an empty baseline signature. '[]' is
truthy, so an invented baseline was being rebased onto a post-gesture
capture; 'no usable baseline' now has one representation, matching
markPendingInteractionOutcome.
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.99 MB 1.99 MB +295 B
JS gzip 635.8 kB 635.9 kB +130 B
npm tarball 769.4 kB 769.6 kB +133 B
npm unpacked 2.69 MB 2.69 MB +295 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.2 ms 27.4 ms +0.3 ms
CLI --help 64.9 ms 68.2 ms +3.3 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/internal/daemon.js +295 B +130 B

@thymikee

thymikee commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

Reviewed exact head e7e0efd8. The production route and live evidence support the diagnosis, but one P2 test gap remains.

P2 — assert the diagnostic payload, not only that an event occurred. The new operator-facing behavior is the reason and divergence evidence carried by post_gesture_no_effect_vetoed, yet the claim tests only count events by phase. A regression that emits the wrong/missing reason, or omits all four divergence counts, stays green. Extend the production-path diagnostic tests to assert:

  • backend rebase → reason: baseline_rebased;
  • surface divergence → reason: surface_divergence with exact onlyInBaseline, onlyInCurrent, rectMismatched, and shared values.

Use the existing diagnostic scope/log helpers so the assertion pins the emitted NDJSON payload, not merely the pure summarizer. The empty-baseline fix and claim-preserving veto logic otherwise look sound. The current Select affected failure is an action-download 500/503 infrastructure flake; substantive CI is still pending.

The claim tests counted `post_gesture_no_effect_vetoed` events by phase. The
operator-facing behavior this PR adds is the *reason* and the divergence
counts, and both survive an event tally: emitting the wrong reason, or dropping
the counts entirely, keeps every count at 1.

The four veto tests now assert the whole emitted payload, read back out of a
diagnostics trace file so the assertion pins the serialized NDJSON line rather
than an in-memory event. Backend rebase pins `reason: baseline_rebased` with no
counts riding along (the pair is cross-backend; there is nothing comparable to
count over). Both divergence cases pin `reason: surface_divergence` with exact
onlyInBaseline/onlyInCurrent/rectMismatched/shared — the numbers that separate
scope drift from movement, and the two membership-drift directions from each
other.

Verified red both ways the reviewer named: dropping the counts fails 2 tests,
forcing the reason to surface_divergence fails 2. The old count assertions
stayed green under both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Rva4YGtSCAKJqH5PbpcCU

thymikee commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

Fixed at 5c54ce341. The finding was correct and the gap was exactly the one that makes instrumentation PRs worthless: the counts prove an event fired, and the event firing was never the contribution — the reason and the numbers are.

The four veto tests now assert the whole emitted payload, read back out of a diagnostics trace file rather than an in-memory array, so what is pinned is the serialized NDJSON line an operator actually reads (redaction included), via the existing withDiagnosticsScope({ traceLogPath }) seam.

  • backend rebase → { action, backend, reason: 'baseline_rebased' }, with no divergence counts riding along: once the pair is cross-backend there is nothing comparable to count over, and reporting counts there would describe two incomparable captures as movement.
  • scope drift → reason: 'surface_divergence' with onlyInBaseline: 2, onlyInCurrent: 0, rectMismatched: 0, shared: 1.
  • same-backend membership drift → reason: 'surface_divergence' with onlyInBaseline: 0, onlyInCurrent: 1, rectMismatched: 0, shared: 3.

The last two are the point of asserting numbers rather than a reason string: they are the same veto with opposite drift direction, and an event tally cannot tell them apart.

Red both ways you named:

=== divergence counts dropped ===
   × scope drift accepts stale but is vetoed from claiming no-effect, observably (#1601 P1 gate)
   × same-backend membership drift vetoes the claim and records the divergence
      Tests  2 failed | 5 passed (7)

=== reason forced to surface_divergence ===
      Tests  2 failed | 5 passed (7)

The old count assertions stayed green under both.

One thing the payload assertions surfaced that I'd have missed: the scope-drift capture carries no snapshotQuality, and the serialized line therefore omits backend entirely rather than writing null. Pinned as-is, since an aggregator reading these would otherwise have to special-case a null that never appears.

pnpm check:affected --run fully green at this head (1,139 tests through the related selector).


Generated by Claude Code

@thymikee

thymikee commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 5c54ce3. The prior P2 is fixed: the production-path tests now read the serialized diagnostics trace and pin baseline_rebased separately from surface_divergence, including exact one-sided membership and rect-mismatch counts. The focused claim suite passes 7/7 and the full TypeScript build typecheck passes locally. The live Bluesky artifact remains applicable because the new commit changes tests only. No new code findings; the PR is merge-ready from review. CodeQL is queued, with no confirmed CI failure or conflict.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 6, 2026
@thymikee
thymikee merged commit 0033002 into main Aug 6, 2026
4 checks passed
@thymikee
thymikee deleted the claude/1620-veto-instrument branch August 6, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gestureNoEffect warning is unreachable on hostile-capture screens (the ones it targets)

2 participants