feat: add selector-targeted drag gestures

[thiagobrez]

Summary

Adds a generic selector/ref-targeted drag gesture across the CLI, Node client, MCP, recording, and replay surfaces.

• Resolves source and destination independently before dispatch, then lowers their centers to one uninterrupted pointer plan: source hold → timed movement → optional destination hold → release.
• Accepts selectors or pinned snapshot refs at either endpoint and returns selector chains plus resolution disclosures for both.
• Records portable selector chains and one targets-v1 annotation containing source and destination identity evidence; replay verifies both endpoints before pointer-down and attributes a later guard mismatch to the correct endpoint.
• Applies ref-admission and frame-expiry policy to both endpoints.
• Supports target-authored drag only where the backend can preserve every authored phase: Android touch devices and iOS/iPadOS. Android TV, tvOS, macOS, visionOS, watchOS, Linux, Vega, and web reject it before injection.
• Contains no react-native-reorderable-specific selectors, labels, fixtures, prototypes, or assets.

Public API

CLI:

agent-device gesture drag 'id="drag-source"' 'id="drop-target"'
agent-device gesture drag @e4~s12 'label="Archive"' 700 600 200

Node:

await client.interactions.drag({
  source: 'id="drag-source"',
  destination: 'id="drop-target"',
  sourceHoldMs: 700,
  moveMs: 600,
  destinationHoldMs: 200,
});

MCP:

{
  "kind": "drag",
  "source": "id=\"drag-source\"",
  "destination": "id=\"drop-target\"",
  "sourceHoldMs": 700,
  "moveMs": 600,
  "destinationHoldMs": 200
}

Defaults are 800 ms source hold, 500 ms movement, and 0 ms destination hold. The combined plan is capped at 10 seconds.

Recording and replay

A recorded drag stores one annotation of this shape:

# agent-device:targets-v1 {"source":{...},"destination":{...}}

Snapshot refs are materialized to portable selector chains. Strict publication refuses either endpoint when it remains a session-local ref or when dual identity evidence is missing. Replay resolves and verifies source and destination sequentially before dispatch, then threads both verified guards through the daemon boundary.

Validation

• pnpm check:affected --run
  • 398 test files passed
  • 3,764 tests passed
  • 94.74% changed-line coverage (126/133)
  • 87.65% changed-branch coverage (142/162, non-gating)
  • format, lint, typecheck, layering, fallow, MCP metadata, build, package, provider integration, Node integration, and replay compatibility passed
• Focused replay/publication regression set: 5 files, 62 tests passed.
• Interaction-guarantee contract: 8 tests passed after merging the selector-package extraction from current main.
• Android touch and iOS simulator fixture scenarios cover the supported device paths; capability tests explicitly cover every rejected platform listed above.
• Exact-head public drag replays pass on a fresh Pixel 10 Pro emulator (8/8 steps) and iPhone 17 Pro simulator (7/7 steps), each guarded by drag completed yes.
• Exact-head full-platform evidence: Replay Nightly run 30954741944 proves Android destination arrival in the full-emulator job and the named iOS destination step passes in the full iOS job.

Video evidence

native-ios.mp4

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-05 10:37 UTC

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.96 MB	1.97 MB	+11.6 kB
JS gzip	628.5 kB	631.4 kB	+3.0 kB
npm tarball	756.5 kB	759.5 kB	+3.0 kB
npm unpacked	2.65 MB	2.67 MB	+11.9 kB

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	25.3 ms	25.0 ms	-0.3 ms
CLI --help	61.1 ms	59.8 ms	-1.3 ms

Top changed chunks:

Chunk	Raw diff	Gzip diff
dist/src/viewport-dimension.js	+2.6 kB	+682 B
dist/src/screenshot-geometry.js	+2.3 kB	+569 B
dist/src/session.js	+1.8 kB	+467 B
dist/src/interaction.js	+1.3 kB	+370 B
dist/src/screenshot-result.js	+1.4 kB	+270 B

[thymikee]

Review: direction is right, two defects to fix, and one encapsulation theme

Direction: yes. A target-authored drag is the correct primitive to add — it belongs at the interaction layer next to press/click, not as a coordinate recipe callers assemble themselves. Three choices in particular are right:

• Resolving both endpoints to a durable selector chain and recording those instead of session-local refs. Bare-ref recordings are the thing that makes replay brittle, and assertNoUnresolvedDragEndpoint refusing publication is the correct hard stop.
• Applying ADR 0014 ref admission to both endpoints, then expiring the frame at the normal mutation seam.
• Threading the surface through RAW_COMMAND_DESCRIPTORS so CLI, MCP, and docs derive from the descriptor rather than three hand-maintained copies.

Gates I ran locally on 3b636f0: typecheck, lint, and check:layering all pass (layering reports 993 files clean, R11 intact), and the focused suite is green — 5 files / 39 tests. The load-bearing red proof in the description is the right kind of evidence.

Below: two defects I reproduced, then the encapsulation review you asked for.

---

1. Recording round-trip corrupts partially-specified drag timings

gesturePayloadToPositionals encodes drag through compact() (gesture-normalization.ts:313), and compact drops undefined anywhere in the list, not just trailing:

function compact(values: Array<string | number | undefined>): string[] {
  return values.filter((value): value is string | number => value !== undefined).map(String);
}

Drag is the first gesture with three independent optional positionals, so a hole in the middle shifts everything after it into the wrong slot. Reproduced against this branch:

encoded -> ["drag","id=\"a\"","id=\"b\"","700"]
decoded -> { kind:'drag', source:'id="a"', destination:'id="b"', sourceHoldMs:700 }

{ moveMs: 700 } round-trips as { sourceHoldMs: 700 }. Reachable from MCP ({"kind":"drag","source":…,"destination":…,"moveMs":600}) and from the Node client (client.interactions.drag(src, dst, { moveMs: 600 })) — both legal per the contract, since all three timings are optional. CLI positional syntax can't express the hole, which is why the existing round-trip test misses it: it only covers the all-fields-present payload.

The consequence is the bad kind: a recorded script silently replays a different gesture. For the long-press-drag reorder this feature exists to drive, moving 600ms out of moveMs and into sourceHoldMs changes whether the drag activates at all.

Fix is either positional placeholders for drag, or encode the timings as a trailing triple that is all-or-nothing. Worth a round-trip test over each of the 8 present/absent combinations.

2. gesture drag reports itself as gesture pan when unsupported

capabilities.ts:145 fabricates a zero-delta pan to reuse the existing checks:

const capabilityInput: GestureSemanticInput =
  input.intent === 'drag'
    ? { intent: 'pan', origin: { x: 0, y: 0 }, delta: { x: 0, y: 0 } }
    : input;

That synthetic payload is then what builds the message and the structured detail. Actual output on this branch:

{"platform":"web"}                          -> "gesture pan is not supported on web"          details.gesture="pan"
{"platform":"ios","appleOs":"visionos"}     -> "gesture pan is not supported on visionos"     details.gesture="pan"
{"platform":"ios","appleOs":"watchos"}      -> "gesture pan is not supported on watchos"      details.gesture="pan"

A user who typed gesture drag is told gesture pan is unsupported, and details.gesture — which agents key on — carries the wrong command. Given ADR 0010 and the error-system work, this one should not ship as-is. Drag wants its own branch (or an intent→capability-key map) so the message names the command the caller actually ran.

---

Module encapsulation

The surface-level encapsulation is good: contracts owns validation and the codec, the descriptor registry owns the public surface, assertRefMutationAdmitted was extracted as a proper shared throwing form rather than copy-pasted. Layering passes.

The problem is one level down. Drag is threaded through as an exception at every layer instead of being admitted into the shared gesture model, and it shows up as the same shape repeated:

Four modules independently hard-code drag's positional layout. There is no shared accessor, so the encoding in packages/contracts is now load-bearing for three consumers that reach into it by index:

Module	Coupling
packages/contracts/src/gesture-normalization.ts:264,313	owns the encoding
src/daemon/handlers/interaction-gesture.ts:237-238	writes positionals[1], positionals[2]
src/daemon/session-script-writer.ts:345	reads positionals.slice(1, 3)
src/daemon/handlers/session-replay-target-token.ts:8	reads positionals[1]

Reorder drag's positionals and three of these break silently — and defect 1 above is exactly what a hole in that layout already does. The daemon rewrite in particular would read better as a re-encode than an index patch:

gesturePayloadToPositionals({
  ...input,
  source: recording.sourceSelector ?? input.source,
  destination: recording.destinationSelector ?? input.destination,
})

…which keeps the layout knowledge in the one module that owns it, and stops being wrong the moment placeholders land for defect 1.

The type model doesn't absorb drag, so each layer widens or casts around it:

• resolveExecutionProfile returns 'hold-drag' (interaction-gesture.ts:198), but GestureExecutionProfile is 'endpoint-hold' | 'timed-pan' and buildDragGesturePlan stamps the plan 'timed-pan'. So the response advertises a profile that is not in the union and is not what executed. The declared return type is string | undefined, so nothing catches it. Either admit 'hold-drag' into the union and carry it onto the plan, or rename the response field so it isn't read as the plan's profile.
• kind: GestureIntent | 'drag' — an inline widening rather than the exported GestureCommandInput model.
• capabilities.ts types its parameter GestureSemanticInput | { intent: 'drag' } even though GestureCommandInput is exported from contracts for exactly this. Two spellings of one concept.
• Two casts where a discriminated narrow would do: options.gesture as GestureSemanticInput (gesture-command.ts:85) and resolved as ResolvedInteractionTarget & { point: Point } (line 169). The first is a direct consequence of narrowing via the separate resolvedDrag variable instead of branching on options.gesture.intent — which is also why the same block needs dragGesture?.sourceHoldMs on a value that cannot be undefined there.

None of these is individually serious. Together they mean the next gesture that carries targets repeats all of it. Branching once on the discriminant, and letting GestureCommandInput be the single spelling, removes most of the casts and the fake-pan shim at the same time.

---

Smaller notes

• Duration sum is validated too late. Each timing is capped at 10s in readGesturePayload, but the total is only checked inside buildDragGesturePlan — after captureGestureViewport and both target resolutions. gesture drag src dst 10000 10000 10000 pays a snapshot and two resolutions before failing INVALID_ARGS. The sum check belongs in contracts next to the per-field ones.
• readNonEmptyString validates value.trim().length > 0 but returns the untrimmed string, so ' id="x" ' reaches the resolver with padding.
• sourceHoldMs has min: 1 while destinationHoldMs has min: 0. Presumably deliberate (you must hold to activate; you needn't hold to release) but it's undocumented and will read as a typo.
• prepareDragTarget returns { target } and both call sites immediately unwrap .target.
• No corpus coverage. grep finds gesture drag nowhere under examples/ or test/ — only in website/docs. Unit coverage is solid, but the repo's test-app:replay:* corpora are how gestures usually earn their keep, and removing the prototype assets left the new command with no replay fixture. A small drag.ad against a test-app reorder target would also give the timing semantics somewhere to regress.

One thing I could not verify

Drag lowers to executionProfile: 'timed-pan', so it shares the runner's .sampled path with gesture pan. While testing something unrelated on iOS earlier today I saw gesture pan wall-time not scale with its requested durationMs (400 / 1200 / 3000ms all completing in roughly the same time). I could not confirm that: every one of those trials used a pan geometry that was inert, so the fast return may just be an early-out, and a re-test at a moving geometry was blocked by another daemon owning the simulator. Flagging it only because this feature's whole value rests on an 800ms activation hold actually lasting 800ms on device. If it hasn't been checked directly, it's worth asserting the observed contact duration once on each platform rather than inferring it from the plan.

[thymikee]

Reviewed exact head 3b636f0; not merge-ready.

1. P1 — replay verifies only the source endpoint. Drag records/resolves two element targets, but the replay token/guard is built only from positional 1 and resolveDragTarget explicitly disables expectedResolvedTarget for destination. A destination selector can rebind to a different drop target and still execute without REPLAY_DIVERGENCE. ADR 0012 requires identity evidence/verification for every element resolution. Introduce a versioned multi-target evidence/guard shape and verify both endpoints before pointer-down; add a shifted-destination counterfactual.

2. P1 — recording corrupts sparse timing options. gesturePayloadToPositionals passes the three independently optional drag timings through compact, dropping interior undefineds. For example Node/MCP { moveMs: 600 } records gesture drag <source> <destination> 600, which replays as sourceHoldMs=600 and default movement instead of default source hold plus a 600 ms movement. Preserve positional slots or serialize fully materialized canonical timings, and add hole-case round trips.

3. P1 — the new element-targeting dispatch path is absent from ADR 0011’s guarantee matrix. The matrix still classifies only existing runtime selector/ref touch commands and coordinate paths; drag’s dual endpoint disambiguation, occlusion/offscreen/non-hittable behavior, identity, disclosures, response construction, and errors are not declared or contract-scenario-gated. Add an honest dual-endpoint drag path/coverage. Simply adding gesture to existing lists would be dishonest while drag builds a bespoke response rather than using their declared response builder.

CLI/Node/MCP/daemon routing, ref admission/expiry, portable ref rewriting, duration planning, CI, and the reported device runs otherwise look sound. The device evidence is described but not attached/reproducible, so that remains a readiness residual.

[thymikee]

Deep code-quality audit (structural)

Follow-up to my earlier comment, which covered two reproduced defects and the positional-layout coupling. This pass is purely about structure: is this the simplest shape this feature can take? I don't think it is, and I think there are two code-judo moves that delete most of the new complexity rather than rearrange it.

Not approving on structure yet. Behavior looks right and the gates are green — that isn't the bar here.

File-size rule: clean. Nothing crosses 1k because of this PR. registry.ts 1515→1516 and client.test.ts 1381→1412 were already over and grew trivially. No objection there.

---

1. Drag logic is in the wrong module, and that's what forces the casts

src/commands/interaction/runtime/gesture-command.ts was a 101-line coordinate-only dispatcher. It's now 254 lines — 2.5× — because it absorbed target resolution, disclosure shaping, recording-evidence extraction, and selector-chain joining.

That work already has a canonical home in the same directory. runtime/gestures.ts is where target-resolving interaction commands live — focusCommand, longPressCommand, scrollCommand — and every one of them follows exactly the shape drag needs:

const resolved = await resolveInteractionTarget(runtime, options, {
  action, requireInteractive, promoteToHittableAncestor,
  expectedResolvedTarget: options.expectedResolvedTarget,
});
const point = requireResolvedPoint(resolved);
…
return { ...resolved, … };   // disclosure / selectorChain / evidence propagate by spread

The new code reimplements that spine instead of using it:

canonical, in gestures.ts	reimplemented in gesture-command.ts
requireResolvedPoint (:264) — returns a narrowed Point	inline if (!resolved.point) throw + as ResolvedInteractionTarget & { point: Point } (:168-169)
return { ...resolved, … } propagates disclosure/evidence	bespoke dragTargetDisclosure, dragTargetDisclosures, recordedDragTarget, selectorExpression

Note what that first row means: the cast at :169 exists only because the canonical helper wasn't used. requireResolvedPoint already does the narrowing. That's not a nit about a cast — it's a cast that is a symptom of the placement being wrong.

The judo move: put dragCommand in gestures.ts next to longPressCommand, and let gesture-command.ts go back to being coordinate-only. That single move deletes:

• the as GestureSemanticInput cast at gesture-command.ts:85 — the coordinate dispatcher stops receiving a union it has to narrow away
• the as ResolvedInteractionTarget & { point: Point } cast at :169 — reuse requireResolvedPoint
• dragGesture?.sourceHoldMs optional-chaining on a value that cannot be undefined in that branch
• expectedResolvedTarget on GestureCommandOptions, which only drag ever reads
• most of the four bespoke disclosure/recording helpers, in favour of the ...resolved convention

I'll grant the one real asymmetry: drag has two endpoints and the spread convention carries one. So the destination needs something bespoke. But the source is the recorded, replay-verified identity — it can use the canonical path as-is, and only the destination needs a small addition. That is a much smaller delta than the current 153 new lines in the wrong file.

2. buildDragGesturePlan duplicates buildSinglePointerPlan; it should be a decorator

gesture-plan.ts grew 475→560, and ~60 of those lines re-derive what buildSinglePointerPlan (:278) already does — finitePoint ×2, sampleOffsets, interpolatePoint, assertSamplesInViewport, and a byte-identical plan literal. The only genuine difference is a hold sample before the move and optionally one after.

A drag plan is a pan plan with contact holds bracketing it. Expressing that directly:

export function buildDragGesturePlan(input, viewport, platform): SinglePointerGesturePlan {
  const frame = normalizeViewport(viewport);
  const move = buildSinglePointerPlan(
    'pan', input.from, input.to, moveMs, frame, 'timed-pan', gesturePlatformProfile(platform),
  );
  return withContactHolds(move, { leadMs: sourceHoldMs, trailMs: destinationHoldMs });
}

function withContactHolds(plan, { leadMs, trailMs }) {
  const [pointer] = plan.pointers;
  const first = pointer.samples[0], last = pointer.samples.at(-1);
  return {
    ...plan,
    durationMs: leadMs + plan.durationMs + trailMs,
    pointers: [{ ...pointer, samples: [
      { offsetMs: 0, point: first.point },
      ...pointer.samples.map((s) => ({ ...s, offsetMs: s.offsetMs + leadMs })),
      ...(trailMs > 0 ? [{ offsetMs: leadMs + plan.durationMs + trailMs, point: last.point }] : []),
    ]}],
  };
}

~10 lines replacing ~60, no duplicated invariants, and the viewport/finite/sample rules stay owned by one builder. It also drops the .slice(1) special case in the current version: the pan plan's own offset-0 sample naturally becomes the hold-end sample once shifted by leadMs.

Worth noting this combinator is reusable the moment anything else needs "press and hold, then move" — which is the same primitive longPress + drag would want to share.

3. intent === 'drag' is now a branch in eight places across six modules

Counting the special-cases this PR adds: normalizeGestureCommandInput, publicGestureFromPayload (throw), requireGestureSupported, assertSupportedInteractionSurface remap, the plan selection in gestureCommand, the message selection, prepareGestureCommandInput, resolveExecutionProfile — plus the four modules hard-coding the positional layout that I listed in the earlier comment.

Every one of those is "is this the odd one out?" That's the signature of a model that hasn't absorbed the concept. Moves 1 and 2 remove roughly half of them on their own; the rest mostly collapse once GestureCommandInput is the single spelling instead of GestureSemanticInput | { intent: 'drag' } being re-spelled inline in capabilities.ts.

The fake-pan shim is the worst instance and I covered it above — it's also a live bug, since it makes gesture drag report itself as gesture pan.

4. 'hold-drag' isn't real

resolveExecutionProfile returns 'hold-drag', GestureExecutionProfile is 'endpoint-hold' | 'timed-pan', and the plan is stamped 'timed-pan'. Three different answers to one question, and the string | undefined return type means nothing catches it.

Under move 2 this resolves itself: a drag genuinely is timed-pan plus holds, so either report that honestly, or add 'hold-drag' to the union and carry it onto the plan so the runner can see it. What shouldn't survive is a response field advertising a profile that neither the type nor the plan agrees with.

---

Summary

The feature is the right thing to build and the surface design (portable selector chains, dual-endpoint ref admission, descriptor-driven registration) is good. But the implementation is currently threaded around the existing architecture rather than through it: it lands in the coordinate dispatcher instead of the target-resolving module next door, re-derives two helpers that already exist, and pays for that with casts, a fake capability payload, and an execution-profile string that isn't in its own union.

Suggested order: move dragCommand into gestures.ts (1) → convert the plan builder into the hold decorator (2) → then the remaining intent === 'drag' branches and 'hold-drag' mostly fall out. I'd expect that to land the feature with materially less new code than the current 1,156 lines, and no casts.

Happy to be argued out of any of this if there's a constraint I'm not seeing — particularly on the two-endpoint recording limit, which is the one place I think a bespoke shape is genuinely earned.

[thymikee]

Could this reuse gesture transform instead of adding a new kind?

Worth answering explicitly since it's the obvious "do we need new API at all?" question. I checked, and the answer is no — but it sharpens what should change.

transform is two-fingered. buildTransformPlan returns topology: 'two' with pointers: readonly [PointerTrajectory, PointerTrajectory] (gesture-plan-types.ts:69). A long-press-drag reorder needs one uninterrupted single-pointer contact. Confirmed live against the test-app gesture lab: gesture transform drives the lab's two-pointer target (minPointers={2}, GestureLab.tsx:244) and sets pan changed yes, pinch changed yes, rotate changed yes, while a single-pointer pan moves nothing on that same target. Different gestures, not different spellings of one.

And it has neither a hold nor targets — transform x y dx dy scale degrees [durationMs] is coordinate-authored with no activation hold, which is the part a reorder UI actually keys on.

Nor is a targeted drag expressible by composition today. longpress 'id="src"' followed by gesture pan … releases contact between the two commands. A drag is one contact across resolve → hold → move → release, which is exactly why this can't be assembled from what exists. That justification holds up.

What about folding it into pan? I'd argue against it, as the strongest alternative worth considering. pan is origin + relative delta; drag is absolute source→destination between two resolved targets. Overloading the positionals so gesture pan 200 430 0 -90 and gesture pan 'id="a"' 'id="b"' share one grammar is the same arg-mode ambiguity that already produced the compact() round-trip bug in my first comment — it would make the codec worse, not better.

Where this does land

The new public surface is small and earned: drag is a subcommand of the existing gesture command, not a new top-level command.

What's oversized is the internal pathway added alongside it — the GestureCommandInput union, a parallel buildDragGesturePlan, and intent === 'drag' branches across six modules — when a drag plan is a pan plan with contact holds bracketing it.

So the recommendation stays what it was, just stated more sharply: keep the new gesture kind, delete the new internal pathway. Moves 1 and 2 from the structural comment — dragCommand into gestures.ts reusing the canonical resolve/record spine, and buildDragGesturePlan collapsed into a withContactHolds decorator over buildSinglePointerPlan. Net new concepts: one public gesture kind, zero internal ones.

[thiagobrez]

Addressed all four review comments in d51af1b2d.

• Sparse drag timings now serialize as a fully materialized canonical triple; all 8 present/absent combinations round-trip in tests. Input validation also trims targets and rejects an over-budget total before capture/resolution.
• Unsupported-surface errors preserve the caller's drag intent in both the message and structured details.
• Drag dispatch now lives in the canonical target-resolving gestures.ts path, uses requireResolvedPoint, and reports the real timed-pan profile. buildDragGesturePlan is a contact-hold decorator over buildSinglePointerPlan. Endpoint positional knowledge is isolated in the contracts codec rather than repeated in daemon consumers.
• Replay now records a versioned targets-v1 annotation and verifies/guards both source and destination before pointer-down. The new shifted-destination counterfactual proves that a rebound destination refuses without dispatching.
• ADR 0011 now has an honest target-drag guarantee row with executable contract scenarios; ADR 0012 documents dual-target evidence. Generic iOS/Android test-app replay corpus entries cover the gesture without project-specific labels.
• Kept gesture drag as the public API: transform is two-pointer and cannot express one uninterrupted target-to-target contact. Internally it reuses the single-pointer pan plan as suggested.

Red proofs were run for sparse timing corruption, fake-pan capability reporting, skipped destination verification, missing ADR matrix coverage, and late total-duration validation. Final local gate: pnpm check:affected --run (all runnable checks passed; 3,521 related tests, 5,144 unit/smoke tests, 5,386 coverage tests, 149 provider-integration tests, plus replay compatibility and Node integration).

[Copilot]

Pull request overview

Adds a new selector/@ref-targeted gesture drag interaction to agent-device, spanning CLI/client/MCP projection, daemon dispatch + recording, and replay verification. This introduces a dual-endpoint resolution + disclosure model and dual-endpoint replay guards/evidence so drag replays refuse before pointer-down if either endpoint no longer matches.

Changes:

• Introduce target-authored gesture drag end-to-end (contracts planning + runtime resolution + daemon handler + client/MCP/CLI projection).
• Add targets-v1 dual-endpoint recording evidence (targetEvidences) and replay-time verification/guarding for both source and destination.
• Extend docs, ADRs, fixtures, and integration/e2e replay scripts to cover the new drag surface.

Reviewed changes

Copilot reviewed 75 out of 75 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
website/docs/docs/commands.md	Document gesture drag CLI usage and behavior.
website/docs/docs/client-api.md	Add client.interactions.drag() to the documented API surface.
test/integration/ios-simulator-e2e/live-replay-scenarios.ts	Include iOS drag replay script in the live replay fixture suite.
test/integration/interaction-contract/target-drag.coverage.ts	Add contract coverage manifest for the target-drag dispatch path.
test/integration/interaction-contract/target-drag.contract.test.ts	Add interaction-contract tests for drag resolution/guards/errors and response shape.
test/integration/interaction-contract/runtime-harness.ts	Extend contract runtime harness to allow gesture viewport + gesture dispatch overrides.
test/integration/interaction-contract/index.ts	Register drag coverage manifest in the global contract coverage aggregation.
test/integration/interaction-contract/fixtures.ts	Add a drag endpoints snapshot fixture for contract tests.
test/integration/interaction-contract/daemon-harness.ts	Add provider transcript helpers for gesture viewport + gesture dispatch.
test/integration/android-emulator-e2e/live-replay-scenarios.ts	Include Android drag replay script in the live replay fixture suite.
src/replay/plan-digest.ts	Include targetEvidences in replay plan digest canonicalization.
src/replay/tests/plan-digest.test.ts	Assert digest changes when multi-target evidence changes.
src/mcp/tests/command-tools.test.ts	Assert MCP gesture tool schema exposes drag endpoints + bounded timing phases.
src/daemon/types.ts	Add replayTargetGuards internal dual-endpoint guard channel for drag.
src/daemon/session-script-writer.ts	Refuse writing recorded drag actions that still contain unresolved @ref endpoints.
src/daemon/session-script-active-publication.ts	Enforce portability + evidence requirements for recorded drag publication.
src/daemon/session-action-recorder.ts	Persist targetEvidences (dual-endpoint targets-v1) into recorded actions.
src/daemon/handlers/session-replay-target-verification.ts	Refactor verification to support endpoint-specific token verification and role tagging.
src/daemon/handlers/session-replay-target-token.ts	Add extraction helpers for drag’s source/destination replay tokens.
src/daemon/handlers/session-replay-runtime.ts	Use multi-target verification and thread dual guards into dispatch internals.
src/daemon/handlers/session-replay-report-action.ts	Include targetEvidences in replay report action payload.
src/daemon/handlers/session-replay-multi-target-verification.ts	New helper to verify and guard both drag endpoints before dispatch.
src/daemon/handlers/interaction-ref-policy.ts	Add throwing ref-admission helper for composing multi-ref interactions.
src/daemon/handlers/interaction-gesture.ts	Add drag normalization (ref admission + suffix stripping), dual disclosure, and dual recording evidence capture.
src/daemon/handlers/interaction-gesture-response.ts	Centralize gesture response shaping, including drag targets disclosure + timing.
src/daemon/handlers/interaction-common.ts	Record targets-v1 evidence for multi-target interactions in finalize/recording path.
src/daemon/handlers/tests/session-replay-target-verification-runtime.test.ts	Add replay tests for drag dual-endpoint guard threading and mismatch reporting.
src/daemon/handlers/tests/session-replay-target-token.test.ts	Add unit test asserting drag evidence binds to the source token for single-target extraction.
src/daemon/handlers/tests/interaction-gesture-response.test.ts	Add unit coverage for the new gesture response builder and drag disclosure preservation.
src/daemon/handlers/tests/interaction-gesture-drag.test.ts	Add daemon interaction tests for drag ref admission, recording portability, and frame-expiry behavior.
src/daemon/tests/session-script-writer.test.ts	Add script-writer tests for refusing unresolved drag refs and emitting targets-v1.
src/daemon/tests/session-script-active-publication.test.ts	Add active-publication tests enforcing portable drag endpoints + targets-v1 evidence.
src/core/command-descriptor/registry.ts	Mark gesture as targetIdentityVerification: pre-dispatch.
src/core/command-descriptor/tests/parity.test.ts	Pin gesture in the evidence-carrying command set parity test.
src/core/capabilities.ts	Update gesture capability checks to accept the expanded command input shape (including drag).
src/core/tests/gesture-capabilities.test.ts	Add capability-policy tests for drag (treated like one-pointer pan).
src/commands/interaction/runtime/resolution.ts	Add targetRole to guard-mismatch details and plumb it through resolution errors.
src/commands/interaction/runtime/gestures.ts	Implement runtime dragCommand (dual resolution → build plan → dispatch → disclosure + recording details).
src/commands/interaction/runtime/gestures.test.ts	Add runtime tests for drag resolution ordering, dispatch gating, and endpoint role in guard mismatch.
src/commands/interaction/runtime/gesture-command.ts	Route drag inputs to dragCommand; share viewport resolution with coordinate gesture command.
src/commands/interaction/runtime/tests/test-utils/index.ts	Add drag snapshot fixture for runtime tests.
src/commands/interaction/metadata.ts	Extend gesture metadata schema/types to include drag endpoint + timing fields.
src/commands/interaction/index.ts	Add CLI projection from gesture drag ... to client interactions.drag().
src/commands/interaction/gesture.test.ts	Add CLI/daemon writer projection test for drag positional parsing and structured input.
src/client/client-types.ts	Add interactions.drag() to the typed client interface.
src/cli/parser/cli-help.ts	Update top-level CLI help text to include drag and describe its behavior.
src/cli/parser/tests/cli-help-topics.test.ts	Update help-topic test to expect drag in gesture usage.
src/agent-device-client.ts	Add client implementation for interactions.drag() projecting to gesture structured input.
src/tests/test-utils/property-arbitraries.ts	Rename gesture kind set used for property tests to exclude drag from coordinate-only arbitraries.
src/tests/client.test.ts	Add client projection test for drag structured input.
packages/contracts/src/target-annotation.ts	Add MultiTargetAnnotationV1 contract type for dual-endpoint evidence.
packages/contracts/src/session-action.ts	Add targetEvidences field to recorded session action contract (targets-v1).
packages/contracts/src/interaction.ts	Update resolution disclosure docstring to include drag endpoints.
packages/contracts/src/interaction-guarantees.ts	Add target-drag dispatch path classification + guarantee mapping.
packages/contracts/src/gesture-plan.ts	Add buildDragGesturePlan() and drag hold/move/hold planning helpers.
packages/contracts/src/gesture-plan.test.ts	Add unit + property tests for drag planning and duration bounds.
packages/contracts/src/gesture-plan-types.ts	Add drag command input type and default drag timing constants.
packages/contracts/src/gesture-normalization.ts	Add drag positional codec + normalizeGestureCommandInput() for runtime drag inputs.
packages/contracts/src/gesture-normalization.test.ts	Add tests for drag positional grammar, recording materialization, and normalization.
packages/contracts/src/gesture-input.ts	Add drag payload parsing/validation and total-duration enforcement at the trust boundary.
packages/contracts/src/gesture-input.test.ts	Add validation tests for drag endpoint strings and timing phase bounds.
packages/contracts/src/client-gesture.ts	Add DragOptions client contract type.
packages/ad-script/src/internal/target-annotation-serde.ts	Add targets-v1 serialization/parsing and enforce bounded wrapper payload size.
packages/ad-script/src/internal/script.ts	Bind targets-v1 annotations to the immediately-following action line during parsing.
packages/ad-script/src/internal/script-formatting.ts	Emit targets-v1 annotation lines when targetEvidences is present.
packages/ad-script/src/internal/tests/target-annotation-serde.test.ts	Add tests for targets-v1 round-trip, required endpoints, and size cap enforcement.
packages/ad-script/src/internal/tests/script.test.ts	Add parsing test asserting targets-v1 binds to one action as targetEvidences.
packages/ad-script/src/index.ts	Re-export multi-target annotation serde helpers and payload cap constant.
examples/test-app/src/screens/GestureLab.tsx	Add a drag fixture UI and state bit (drag completed yes/no) for replay verification.
examples/test-app/src/screens/gesture-lab-styles.ts	Add styles for the drag fixture endpoints.
examples/test-app/replays/drag.ad	New iOS simulator replay script validating gesture drag end-to-end.
examples/test-app/replays/drag-android.ad	New Android emulator replay script validating gesture drag end-to-end.
docs/adr/0013-unified-gesture-plans.md	Update ADR 0013 to describe target-authored drag and its planning/recording model.
docs/adr/0012-interactive-replay.md	Amend ADR 0012 with targets-v1 dual-endpoint evidence semantics and replay verification rules.
docs/adr/0011-interaction-guarantee-contract.md	Extend the dispatch-path matrix with the new target-drag path and scope notes.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[thymikee]

Re-review of exact head d51af1b2df2b912288f992cae93dccce2f632d2a: the earlier sparse-timing, capability-error, guarantee-matrix, and dual-endpoint replay findings are meaningfully fixed, with load-bearing tests. The PR is still blocked on the following:

1. P1 — target drag silently loses its authored phases on admitted platforms. requireGestureSupported admits target drag on macOS, tvOS, and Linux, but performGestureApple collapses macOS to an endpoint drag and tvOS to a directional remote swipe. The Linux interactor similarly reduces the canonical plan to endpoints plus total duration, and its input path moves immediately and then sleeps. These paths discard the authored sourceHoldMs / moveMs / destinationHoldMs phases promised by the drag plan and ADR 0013. Please either restrict drag to backends that execute the exact plan or implement phase-preserving adapters, with adapter tests and applicable live evidence.

2. Branch blocker — conflicts and incomplete exact-head CI. GitHub reports the branch as DIRTY, with conflicts including guarantee/runtime-resolution areas. Resolve against current main and rerun the full suite. This exact head currently shows only CodeQL, not Static Checks, Coverage, Integration, or device lanes.

3. Docs/API evidence is stale. ADR 0013 and the PR prose still describe source-only target-v1 evidence although the implementation now records dual targets-v1. The Node example calls drag with three arguments, but the API accepts one DragOptions object. Validation counts and practical evidence also need to be refreshed and made available for the rebased exact head.

[Copilot]

Pull request overview

Copilot reviewed 78 out of 78 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/commands/interaction/index.ts:124

• The gesture help text says pinned-ref for drag endpoints, but the ref grammar in this repo allows both bare refs (@e12) and optionally pinned refs (@e12~s3). Using pinned-ref here can read like the suffix is required.

Consider documenting this as ref to match the actual accepted syntax.
src/cli/parser/tests/cli-help-topics.test.ts:44

• This test asserts the gesture help uses pinned-ref, but refs are valid both with and without a ~s<generation> suffix. If the help text is updated to say ref, update the regex and test description accordingly so the test matches the actual CLI contract.

[Copilot]

Pull request overview

Copilot reviewed 78 out of 78 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/commands/interaction/runtime/gestures.ts:226

• The error message here is generic to "gesture" even though this code path is specifically for selector-targeted drag. If a backend supports coordinate gestures but not target-authored drag, this message will be misleading to users and makes debugging harder.

[thymikee]

P1: Android target drag is stationary despite green CI. buildDragGesturePlan emits source@0, source@sourceHold, destination@sourceHold+move, then optional destination hold. lowerAndroidTouchPlan destructures only [start, end]; for drag those are the first two source samples, so it densifies source→source across the full duration and drops movement/destination phases. Lower the canonical trajectory piecewise and add a regression asserting source hold, movement, and destination hold survive.

The branch/docs/guarantee blockers are otherwise resolved, but both drag replay scripts are in full:fixture-replays while the green Android/iOS jobs run the smoke tier. After the fix, require exact-head live Android and iOS full-tier evidence proving drag reaches its destination.

[Copilot]

Pull request overview

Copilot reviewed 80 out of 80 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 81 out of 81 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/daemon/session-script-active-publication.ts:123

• In assertPortableDragBindings, non-selector endpoint tokens currently fall through (continue) and can be published without error or targets-v1 evidence. For gesture drag, endpoints are required to be portable selector expressions at publication time; otherwise a published script can be invalid or silently skip identity enforcement.

[thymikee]

Clean code-review on 14d3ef85. The Android lowerer now preserves the canonical source-hold → movement → destination-hold trajectory, and the new regression is load-bearing against the prior two-sample phase loss. Production dispatch, dual-endpoint replay guards, and the target-drag guarantee path remain sound.

Remaining merge gates: exact-head CI and the in-progress nightly Android/iOS direct-drag plus full-tier replay evidence must complete green and prove drag completed yes on both platforms.

[Copilot]

Pull request overview

Copilot reviewed 84 out of 84 changed files in this pull request and generated no new comments.

Suppressed comments (1)

test/integration/interaction-contract/fixtures.ts:260

• In this snapshot fixture, node 3 declares parentIndex: 0 (direct child of the Application) but depth: 2, which is inconsistent with the rest of the snapshot structure. Keeping depth aligned with parentIndex makes the fixture more representative and avoids accidental reliance on invalid tree shapes in resolution/ordering logic.

[thiagobrez]

Addressed at exact head 14d3ef85bcae49471676401899344818e68edcd3.

• Android lowering is now piecewise across every canonical sample boundary, preserving source hold, movement, and destination hold. The regression uses non-cadence-aligned phase durations so the old [start, end] implementation fails it.
• Fixed both checked-in drag replays: .ad does not treat shell-style single quotes as quoting. A static regression now requires parseable source/destination selectors.
• The live iOS run exposed a separate false occlusion: Expo Router’s full-viewport structural Toolbar was classified as a floating overlay over every child. Full-viewport tab/toolbar/navigation containers are now excluded narrowly, with a regression; real smaller chrome remains an overlay.
• The fixture endpoints are real accessible buttons and live near the top of the card, so the scripts are independent of scroll state.

Exact-head live evidence: Replay Nightly run 30954741944:

• Android full-emulator job logged success: true, replayed: 8, and the destination canary passed. The job later failed in the pre-existing microphone-permission assertion (alertStatus returned no alert, expected permission), after the drag proof completed.
• iOS full-tier job has a green named Prove selector drag reaches its destination on iOS step; the same exact-head replay completed 7/7 steps with the drag completed yes guard.

Local gate is also green: 398 test files / 3,764 tests; changed-line coverage 126/133 (94.74%), changed-branch coverage 142/162 (87.65%).

[thymikee]

Final exact-head gate update: all regular PR checks are green, and the new direct selector-targeted drag replay passed on both platforms (Android: 8 steps, drag completed yes; iOS/XCTest: 7 steps, drag completed yes). The broader nightly run is nevertheless red from later native-suite scenarios unrelated to drag: Android full:lifecycle-system saw no microphone permission alert (also reproduced on the preceding run), and iOS smoke:automation-input timed out waiting for a native alert. These do not invalidate the drag implementation or its direct device evidence, but they are confirmed owner-action CI failures, so ready-for-human is being removed until the native-suite state is resolved or a clean rerun is available. Run: https://github.com/callstack/agent-device/actions/runs/30954741944

[thymikee]

Branch blocker update on unchanged head 14d3ef85bcae49471676401899344818e68edcd3: after the latest main merges, GitHub now reports this PR as DIRTY / conflicting. The existing regular PR checks are green, but they predate the new base and are no longer sufficient merge evidence. Rebase onto current main, resolve the conflicts, then rerun the affected gate and exact-head CI. The prior drag code-review verdict and device evidence do not need repeating unless the conflict resolution changes the relevant production path.

[Copilot]

Pull request overview

Copilot reviewed 84 out of 84 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/daemon/session-script-active-publication.ts:125

• assertPortableDragBindings currently only rejects session-local @refs, but it allows non-@ endpoints that are not valid selector expressions (it just continues). Since gesture drag endpoints must be selectors once portable, this can let an invalid/garbage endpoint slip through publication and only fail later at replay-time resolution.

[thiagobrez]

Addressed the branch blocker:

• rebased onto current main (4c7a899a0)
• resolved the replay-runtime conflicts while preserving the extracted @agent-device/ad-replay architecture and dual-endpoint drag verification
• updated the drag planner/tests to the current fixed-trajectory and temp-dir test APIs
• force-pushed exact head 7c5b98e8bdfb010b4936d7972a2275fad7123951; GitHub now reports the PR as mergeable

Validation on the exact rebased tree:

• pnpm check:affected --run: passed all runnable checks
• Vitest: 401 files / 3,827 tests passed
• changed-line coverage: 137/147 (93.20%)
• exact-head GitHub CI: 32 successful, 0 failing, 0 pending (including Android, iOS, macOS, Linux, and Web smoke checks)

The resolution preserves the previously reviewed drag production path, so I did not repeat the prior device evidence, per your note.

[thymikee]

Re-reviewed exact head 7c5b98e8bd after the rebase: clean and ready for human merge review. The prior timing, capability, guarantee-matrix, dual-endpoint replay, and Android phase-lowering findings remain closed. CLI/Node/MCP through gesture planning, platform execution, recording, and replay verification is coherent; ADR 0011–0013 contracts are covered. All exact-head checks are green, including Android/iOS/macOS/Linux/Web smoke, and the exact-head Android and iOS selector-drag replays both reached drag completed yes. No remaining code or evidence blocker.