Skip to content

gestureNoEffect warning is unreachable on hostile-capture screens (the ones it targets) #1620

Description

@thymikee

Summary

The gestureNoEffect warning added by #1600 appears unreachable on hostile-capture screens — which is the screen class it was built for. #1600's motivating case was element-18 burning ~40 tool calls re-issuing scrolls on a Bluesky feed, and a Bluesky feed is hostile enough to capture that #1601's corroboration gate can never be satisfied.

Found while gathering live evidence for #1615. Not caused by that PR — it changes nothing about when the warning fires.

Reproduction

CLI + daemon built from source, isolated --state-dir, explicit --platform ios --udid throughout, seeded Bluesky fixture simulator.

$ agent-device scroll top
Already at top; no hidden content above detected

$ agent-device swipe 200 250 200 650      # drag down, nothing above → cannot move
Flung

$ agent-device snapshot
Warning: ios snapshots are slow in this run: p95 13442ms over 2 captures
Snapshot: 167 visible nodes (171 total) (truncated)
Detected an overly complex or slow accessibility tree. Fell back to the private-ax snapshot backend.
Some deeper accessibility nodes were omitted; this tree is capped at depth 56.

No gestureNoEffect warning. Five no-effect gestures across two screens (Settings ×3, Bluesky ×2) all silent.

Mechanism

Two independent vetoes, each correct in isolation:

  1. Backend flip. Post-gesture stale-accept saturates routinely on the checkout screen; wall-clock retry budget starves under contention into stale-geometry interactions #1569 established that a quiet capture from a different snapshot backend can only be re-baselined against, never concluded from — backends disagree about which nodes exist. A hostile screen falls back tree→private-ax mid-sequence, so the loop never reaches a comparable pair.
  2. Truncation drift. fix(daemon): surface proven no-effect gestures to the agent #1601 requires haveIdenticalDiscriminatingSurfaces — discriminating entry sets matching exactly in both directions. A depth-56-capped, truncated 167-node tree will not match its baseline set even when nothing moved.

Both are working as specified. The composite is that the safe failure mode (silence) is the only outcome exactly where the feature is needed.

Why it matters

This is the same shape as #1609: a guard that cannot fire in the conditions it targets. The agent-facing cost is what #1600 measured — an agent re-issuing a gesture the daemon has already proven inert, because the proof cannot be corroborated.

Not yet isolated

I did not determine which veto dominates, or whether either alone is sufficient. That needs per-capture backend identity and the two signature sets, not the rendered warning. Suggested next step: log the corroboration inputs (baseline backend, capture backend, discriminating set sizes, symmetric difference) at debug level on the accept-stale path, then re-run the repro above and read which condition fails.

Possible directions, none validated:

  • allow corroboration across a backend flip when both captures are themselves internally quiet;
  • compare only the intersection of node identities present in both captures when one is known-truncated, rather than requiring set equality;
  • accept the bare accept-stale verdict when the capture is flagged truncated, on the grounds that a truncated pair cannot produce the false-positive fix(daemon): surface proven no-effect gestures to the agent #1601 was guarding against.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions