feat(ios): usbmux runner route override + #1403 transport experiment evidence

Adds AGENT_DEVICE_IOS_RUNNER_ROUTE=usbmux, an experimental override that
routes coredevice-backend physical devices' runner commands through usbmux,
and simplifies the xctest branch that awaited a no-op resolveRunnerTransport.

Documents the live #1403 experiment (iPhone 17 Pro, USB + Wi-Fi legs):
steady-state is a wash, but the >30s-idle tax drops from ~4.5s (tunnel
re-probe + session re-establish) to ~440ms because the usbmux session stays
hot; CoreDevice Wi-Fi devices never appear in usbmuxd, so the verdict is
usbmux-primary with network fallback rather than tunnel-code deletion.
Also records the cable-out failure gap (2x45s retry hang swallowing the
usbmux DEVICE_NOT_FOUND hint), which affects today's xctest backend too.

Author: thymikee

docs/usbmux-runner-transport-experiment.md

@@ -0,0 +1,81 @@
+# #1403 experiment: usbmux as the primary physical-iOS runner transport
+
+Live results from 2026-07-30, thymikee-iphone (iPhone 17 Pro, iOS 26.5.2, UDID
+00008150-001849640CF8401C), macOS host with Xcode 26.2. Method: forced-route
+env override `AGENT_DEVICE_IOS_RUNNER_ROUTE=usbmux` in `runner-command-route.ts`
+(uncommitted experiment patch), isolated `--state-dir` daemons per leg, timings
+from per-request `--debug` ndjson plus wall-clock. Every timing sample was
+validated against real command output after an early instrument artifact (see
+Pitfalls).
+
+## Numbers (same build, same cable state, verified output)
+
+| Scenario | network (tunnel-IP) route | usbmux route |
+|---|---|---|
+| Steady-state snapshot, cable in | 350–450 ms | 440 ms |
+| Snapshot after 40 s idle | **4.5–4.6 s** (tunnel re-probe + session re-establish) | **440 ms** (unchanged) |
+| Runner killed, next snapshot | ~4.0 s self-heal | 4.7 s self-heal |
+| Steady-state snapshot, Wi-Fi only (no cable) | 425–495 ms | fails (see below) |
+| Full lifecycle open/snapshot/tap/screenshot/close | works | works |
+
+Older-build measurements (first pass) agreed in shape: first open pays one-time
+xctestrun prep (~13 s cold) and runner startup; fresh device resolution costs
+~6.7 s of devicectl listing either way.
+
+## Findings
+
+1. **The idle tax is the payoff, and it is bigger than expected.** Past the
+   30 s tunnel-IP cache TTL, the network route re-probes the tunnel and
+   re-establishes runner readiness: ~4.5 s per idle gap, every time. Over
+   usbmux the session stays hot indefinitely (40 s gaps measured at steady
+   440 ms). Agent workflows are exactly the idle-gappy workload that hits this
+   tax on almost every step.
+2. **Steady state is a wash.** The per-command usbmuxd handshake (fresh unix
+   socket, ListDevices, Connect) roughly cancels the network route's pooled-
+   connection advantage; ~±90 ms either way.
+3. **Wi-Fi deletion is off the table.** A CoreDevice-paired, Wi-Fi-reachable
+   iPhone (devicectl `available (paired)`, live tunnel) does NOT appear in
+   usbmuxd at all — modern CoreDevice Wi-Fi runs over `remoted`, invisible to
+   usbmuxd. usbmuxd listed the device only while cabled, `ConnectionType: USB`
+   only, and the entry disappears on unplug. The CoreDevice network route must
+   stay as the Wi-Fi fallback → this is a "usbmux-primary + network-fallback"
+   reshuffle, not a full tunnel-code deletion.
+4. **Failure semantics gap (also affects today's xctest backend).** With the
+   cable out, forced-usbmux commands hang for 2×45 s of retries and surface a
+   generic "Runner did not accept connection" — the usbmux client's precise
+   `DEVICE_NOT_FOUND` ("Connect the device by cable, trust this Mac…") is
+   swallowed by `shouldRetryRunnerConnectError`. An implementation should make
+   usbmux DEVICE_NOT_FOUND non-retryable (or fall back to the network route
+   immediately) and preserve the hint.
+5. **Lock behavior is transport-independent.** A locked phone wedges AX
+   capture/runner relaunch identically on both routes (SBMainWorkspace
+   "Locked" denial; xcodebuild "Unlock thymikee-iphone to Continue").
+6. **usbmux UDID matching is exact.** usbmuxd `SerialNumber` equals the
+   dashed hardware UDID that `DeviceInfo.id` already uses for devicectl-listed
+   devices; DeviceID (10) is the mux handle. Multi-device disambiguation is
+   structural. (Multi-attached-device matrix not exercised: one device only.)
+
+## Implied design (matches the issue's desired outcome, minus full deletion)
+
+- One private route resolver: physical devices try usbmux first; on usbmux
+  DEVICE_NOT_FOUND (not attached via USB) resolve the CoreDevice tunnel and
+  use the network route. Tunnel cache/invalidations stay but only run on the
+  fallback path, so cabled devices never pay the probe/TTL tax.
+- Failure semantics: usbmux DEVICE_NOT_FOUND → immediate fallback attempt;
+  if the fallback also fails, surface the usbmux hint (cable/trust/unlock).
+- The xctest backend keeps usbmux-only (CoreDevice unavailable by definition)
+  but needs the same non-retryable classification for DEVICE_NOT_FOUND.
+
+## Pitfalls for whoever implements/re-measures
+
+- `bin/agent-device.mjs` runs `dist/`, not `src/`. A stale dist silently ran
+  the first "usbmux leg" on stock network code while the env override sat
+  inert in the daemon (numbers looked plausible; only the cable-pull test
+  exposed it). Rebuild and verify the daemon process (`ps eww <pid>`, dist
+  mtime, and a behavioral probe) before trusting route attribution.
+- `/usr/bin/time` on a failing command still prints a small `real` — a fast
+  error latency is indistinguishable from a fast success unless output is
+  asserted. Runner-lease contention ("already owned by another daemon")
+  produced exactly this artifact.
+- Sessions are cwd-scoped; run every command of a leg from the same cwd.
+- Auto-Lock must be Never during measurement or the runner dies mid-series.

src/platforms/apple/core/__tests__/runner-transport.test.ts

@@ -60,6 +60,7 @@ beforeEach(() => {
 
 afterEach(() => {
   vi.unstubAllGlobals();
+  vi.unstubAllEnvs();
 });
 
 test('waitForRunner propagates request cancellation without fallback', async () => {
@@ -224,6 +225,36 @@ test('sendRunnerCommandOnce routes xctest physical devices through usbmux', asyn
   ]);
 });
 
+test('sendRunnerCommandOnce routes coredevice physical devices through usbmux when overridden', async () => {
+  vi.stubEnv('AGENT_DEVICE_IOS_RUNNER_ROUTE', 'usbmux');
+  const fetchMock = vi.fn();
+  vi.stubGlobal('fetch', fetchMock);
+
+  const response = await sendRunnerCommandOnce(iosDevice, 8100, { command: 'uptime' }, 5_000);
+
+  assert.equal(response.status, 200);
+  assert.equal(fetchMock.mock.calls.length, 0);
+  assert.equal(mockRunCmd.mock.calls.length, 0);
+  assert.deepEqual(mockUsbmuxPostCommand.mock.calls[0]?.slice(0, 3), [
+    iosDevice.id,
+    8100,
+    { command: 'uptime' },
+  ]);
+});
+
+test('waitForRunner routes coredevice physical devices through usbmux when overridden', async () => {
+  vi.stubEnv('AGENT_DEVICE_IOS_RUNNER_ROUTE', 'usbmux');
+  const fetchMock = vi.fn();
+  vi.stubGlobal('fetch', fetchMock);
+
+  const response = await waitForRunner(iosDevice, 8100, { command: 'snapshot' }, undefined, 5_000);
+
+  assert.equal(response.status, 200);
+  assert.equal(fetchMock.mock.calls.length, 0);
+  assert.equal(mockRunCmd.mock.calls.length, 0);
+  assert.equal(mockUsbmuxPostCommand.mock.calls.length, 1);
+});
+
 function stubSuccessfulFetch(): void {
   vi.stubGlobal(
     'fetch',

src/platforms/apple/core/runner/runner-command-route.ts

@@ -3,6 +3,13 @@ import { resolveIosPhysicalDeviceControl } from '../physical-device-control.ts';
 
 const RUNNER_DEVICE_TUNNEL_IP_CACHE_TTL_MS = 30_000;
 
+/**
+ * Experimental override for #1403: forces physical-device runner commands
+ * through usbmux regardless of backend. Read per-resolve so daemon restarts
+ * are not required between experiment runs.
+ */
+const RUNNER_ROUTE_OVERRIDE_ENV = 'AGENT_DEVICE_IOS_RUNNER_ROUTE';
+
 type DeviceTunnelIpCacheEntry = {
   ip: string;
   expiresAt: number;
@@ -33,13 +40,8 @@ export function createRunnerCommandRouteResolver(device: DeviceInfo, port: numbe
         return buildNetworkRoute(device, port, null, false);
       }
       const control = resolveIosPhysicalDeviceControl(device);
-      if (control.backend === 'xctest') {
-        const transport = await control.resolveRunnerTransport(device, timeoutBudgetMs);
-        return {
-          kind: transport.kind,
-          endpoints: [`usbmux://${device.id}:${port}/command`],
-          cachedTunnelIp: false,
-        };
+      if (control.backend === 'xctest' || readRunnerRouteOverride() === 'usbmux') {
+        return buildUsbmuxRoute(device, port);
       }
       if (!forceRefresh) {
         const cached = readDeviceTunnelIpCache(device.id);
@@ -50,11 +52,7 @@ export function createRunnerCommandRouteResolver(device: DeviceInfo, port: numbe
       }
       const transport = await control.resolveRunnerTransport(device, timeoutBudgetMs);
       if (transport.kind === 'usbmux') {
-        return {
-          kind: 'usbmux',
-          endpoints: [`usbmux://${device.id}:${port}/command`],
-          cachedTunnelIp: false,
-        };
+        return buildUsbmuxRoute(device, port);
       }
       const tunnelIp = transport.tunnelIp;
       requestTunnelIp = tunnelIp;
@@ -75,6 +73,18 @@ export function clearDeviceTunnelIpCache(): void {
   deviceTunnelIpCache.clear();
 }
 
+function readRunnerRouteOverride(): 'usbmux' | null {
+  return process.env[RUNNER_ROUTE_OVERRIDE_ENV]?.trim() === 'usbmux' ? 'usbmux' : null;
+}
+
+function buildUsbmuxRoute(device: DeviceInfo, port: number): RunnerCommandRoute {
+  return {
+    kind: 'usbmux',
+    endpoints: [`usbmux://${device.id}:${port}/command`],
+    cachedTunnelIp: false,
+  };
+}
+
 function buildNetworkRoute(
   device: DeviceInfo,
   port: number,