Skip to content

refactor(contracts): one viewport-root predicate for the whole repo - #1613

Merged
thymikee merged 4 commits into
mainfrom
claude/unify-snapshot-viewport-geometry
Aug 5, 2026
Merged

refactor(contracts): one viewport-root predicate for the whole repo#1613
thymikee merged 4 commits into
mainfrom
claude/unify-snapshot-viewport-geometry

Conversation

@thymikee

@thymikee thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member

"Is this the Application/Window root" was written nine times: three
spellings normalizing type|role|subrole, five lowercasing type alone,
and one comparing the normalized type for EQUALITY. Two of the nine sat
in contracts/snapshot-visibility.ts itself, disagreeing with each other.

Measured before collapsing, using #1592's method — ground the comparison
in what each backend ACTUALLY emits, not in fixture strings. Over the 31
names iOS's elementTypeName can return, the 18 fully-qualified class
names Android emits, and the 26 mapped/raw forms the macOS helper
produces, the nine agreed on 71 of 75. All four exceptions are macOS
windows, splitting two ways:

  • AXFloatingWindow / AXSystemFloatingWindow — only maestro's ===
    spelling misses them, and its platform union is android | ios, so it
    can never see a macOS node. Inert.
  • AXSystemDialog / AXUnknown — an AXWindow whose subrole is emitted
    AS the type, invisible to the six type-only spellings and named exactly
    by role. These two are the entire behavioral delta of this PR, and
    they are windows by role.

(Corrected from an earlier "73 names / 2 exceptions / 24 macOS forms":
that was measured against an incomplete macOS table, which omitted six
fixed outputs and most of the raw subrole ?? role fallback — including
the two rows that ARE the behavioral delta. Completing the table per
review moved the count, and the completeness claim is now derived from
the Swift emitter rather than a hand-kept twin.)

isViewportRootNode reads role and subrole because the macOS helper is
the only backend populating them and the only one able to emit a window
whose type does not say so: normalizedSnapshotType returns the raw
subrole for a non-standard window, so an AXWindow with subrole
AXSystemDialog or AXUnknown reads as neither from type alone. Those
two shapes are the whole behavioral delta of this change, at the six
call sites that were type-only, and they are windows by role.

snapshot-viewport-root.test.ts pins the predicate over those three
emitted vocabularies. Red evidence: reverting the canonical definition to
the type-only spelling fails 2 of 5 cells, to the equality spelling 4 of 5.

Also drops two kernel re-declarations this made visible: maestro's local
containsPoint and rectsOverlap were character-identical to
@agent-device/kernel/rect's containsPoint and isRectVisibleInViewport,
in a file that already imports from that module. And resolveViewportRect
loses three as Rect casts that only existed because .filter() cannot
narrow node.rect — one flatMap states the same thing honestly.

Deliberately NOT in this change: the three viewport RESOLVERS still
diverge, and on Android that is a live defect rather than duplication.
Filed separately with the measurement.

Gates: typecheck / lint / format / check:layering / check:production-exports green. test:unit 610 files / 5382 tests. check:affected --run green on this SHA: 438 files / 4055 tests, "all runnable checks passed".

Follow-up: #1609 (the three viewport resolvers still diverge; on Android that is a live defect, filed with measurements rather than folded in here).


Local gate note. pnpm check:affected --run is flaky on a loaded dev host. Across six runs on three branches — plus a reproduction on unmodified origin/main by a parallel agent — failures came only from the pool android-lifecycle / android-recording / doctor / input-actions / daemon-client, always as timeouts, with a different subset each run (1, 7, 0, 2, 1). All pass in isolation. Please treat the CI Integration Tests job on this head as authoritative for provider-integration, per AGENTS.md ("GitHub remains authoritative for reported device/toolchain lanes").

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.97 MB 1.97 MB -876 B
JS gzip 631.7 kB 631.5 kB -196 B
npm tarball 761.6 kB 761.5 kB -135 B
npm unpacked 2.67 MB 2.67 MB -876 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 21.6 ms 21.5 ms -0.1 ms
CLI --help 51.6 ms 51.0 ms -0.5 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/internal/daemon.js -526 B -124 B
dist/src/sdk-batch-runner.js +53 B +51 B
dist/src/session.js -58 B -32 B
dist/src/screenshot-geometry.js -103 B -31 B
dist/src/viewport-dimension.js -163 B -21 B

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Two blockers on f1aeb24: (1) snapshot-viewport-root.test.ts says it pins each backend’s actual emitted vocabulary, but the macOS table omits fixed normalizedSnapshotType outputs (TextField, TextArea, MenuBarItem, Menu, MenuItem) and the raw role/subrole fallback. Either enumerate the real vocabulary or narrow the test/comment claim; as written the promised emitter-drift protection is incomplete. (2) The branch conflicts with current main in snapshot-occlusion.ts; resolving it must preserve #1567’s full-viewport-chrome exclusion while replacing both local root checks with isViewportRootNode. The predicate consolidation otherwise looks sound. The Integration failure is the unrelated naïve-pipe flake being removed by #1610; iOS smoke is still running.

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Both blockers addressed on 78d9441, as two commits since they were two problems.

(1) macOS vocabulary — correct, and worse than reported. I had omitted six fixed outputs, not five (StaticText as well), plus AXSystemFloatingWindow and most of the raw subrole ?? role fallback. All three output classes of normalizedSnapshotType are now enumerated, and the table asserts its own completeness against the emitter's fixed-output set, so a role added to that switch without being added here fails — the drift protection the docblock claimed but did not provide.

Completing the table also falsified the header's own numbers. It said "71 of 73 agree, 2 disagree"; over the real 75 names it is 71 agree and 4 disagree, because AXSystemDialog and AXUnknown were the missing rows. Those two are exactly this PR's behavioral delta — an AXWindow whose subrole is emitted as the type, invisible to the six type-only spellings and named precisely by role. The incomplete table had been hiding the rows that justify reading role/subrole at all. The docblock now states the 4 with the split: 2 inert (only the === spelling misses them, and its platform union is android | ios), 2 real.

(2) Merge resolved preserving #1567. It added isFullViewportChromeContainer and a second isViewportRoot call site inside it; both now use isViewportRootNode, and the full-viewport-chrome exclusion is unchanged. Verified across occlusion, overlay, find, interaction-targeting and maestro: 21 files / 187 tests green.

Thanks for the #1610 pointer — matches my read, so I'm not chasing the naive-pipe failure. The remaining local check:affected red is the android-lifecycle timeout flake (passes in isolation, 12/12).

🤖 Addressed by Claude Code

@thymikee
thymikee marked this pull request as ready for review August 5, 2026 11:05
@thymikee
thymikee force-pushed the claude/unify-snapshot-viewport-geometry branch from 78d9441 to 2a7ee8d Compare August 5, 2026 11:09
@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Re-review of 2a7ee8d: the #1567 conflict is resolved correctly and all checks are green, but one test-validity blocker remains. The raw-fallback row { type: 'AXSearchField', role: 'AXTextField', subrole: 'AXSearchField' } cannot be emitted because the AXTextField switch arm always returns TextField; replace it with an actually unmapped role/subrole shape or narrow the claim. Also, MACOS_FIXED_OUTPUTS and the table are both hand-maintained in the same test, so comparing them cannot catch a new mapped Swift role—the promised emitter-drift enforcement is not real unless derived from/checking the emitter. No ready-for-human yet.

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Re-review of exact head 673713d: the prior raw-fallback and emitter-drift test findings are fixed. The AXCell/AXSortButton row is producible by the Swift default arm, and the fixed-output set is now derived from the actual Swift emitter rather than a hand-kept twin.

One P2 blocker remains: please restore the 44 unrelated scripts/maestro-conformance/corpus/** YAML files to main. This corpus is explicitly excluded by .oxfmtrc.json and serves as evidence data; even though the Maestro oracle confirms the quote/indent/EOF churn is semantic-neutral, it is out-of-scope noise in this contracts refactor.

P3: the PR body still reports 73 emitted names / 71 agreements / 2 exceptions (24 macOS forms), while the corrected current measurement is 75 / 71 / 4 (26 macOS forms). Please update it to match the validated table.

The iOS Smoke failure (testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden, partial text) is the known cross-branch XCTest text-entry flake also seen on an unrelated PR, not a code finding here. The exact head is still red, so no ready-for-human label until a rerun is green.

"Is this the Application/Window root" was written nine times: three
spellings normalizing `type|role|subrole`, five lowercasing `type` alone,
and one comparing the normalized type for EQUALITY. Two of the nine sat
in `contracts/snapshot-visibility.ts` itself, disagreeing with each other.

Measured before collapsing, using #1592's method — ground the comparison
in what each backend ACTUALLY emits, not in fixture strings. Over the 31
names iOS's `elementTypeName` can return, the 18 fully-qualified class
names Android emits, and the 24 mapped/raw forms the macOS helper
produces, the nine agreed on 71 of 73. The two exceptions are macOS
window subroles, and the only spelling that disagreed is maestro's `===`,
whose platform union is `android | ios` — so it can never see them. The
duplication was textual, not behavioral, which is what made the collapse
safe.

`isViewportRootNode` reads role and subrole because the macOS helper is
the only backend populating them and the only one able to emit a window
whose `type` does not say so: `normalizedSnapshotType` returns the raw
subrole for a non-standard window, so an `AXWindow` with subrole
`AXSystemDialog` or `AXUnknown` reads as neither from `type` alone. Those
two shapes are the whole behavioral delta of this change, at the six
call sites that were type-only, and they are windows by role.

`snapshot-viewport-root.test.ts` pins the predicate over those three
emitted vocabularies. Red evidence: reverting the canonical definition to
the type-only spelling fails 2 of 5 cells, to the equality spelling 4 of 5.

Also drops two kernel re-declarations this made visible: maestro's local
`containsPoint` and `rectsOverlap` were character-identical to
`@agent-device/kernel/rect`'s `containsPoint` and `isRectVisibleInViewport`,
in a file that already imports from that module. And `resolveViewportRect`
loses three `as Rect` casts that only existed because `.filter()` cannot
narrow `node.rect` — one `flatMap` states the same thing honestly.

Deliberately NOT in this change: the three viewport RESOLVERS still
diverge, and on Android that is a live defect rather than duplication.
Filed separately with the measurement.
Review found the table claimed to pin "the vocabulary each backend actually
emits" while omitting most of it. `normalizedSnapshotType` has three output
classes and only two were represented:

  1. thirteen roles mapped to fixed short names — six were missing
     (StaticText, TextField, TextArea, MenuBarItem, Menu, MenuItem);
  2. AXWindow, whose output is the SUBROLE unless it is AXStandardWindow;
  3. the `default:` arm, `subrole ?? role`, emitting the raw AX-prefixed
     value for every unmapped role.

All three are now enumerated, and the table asserts its own completeness
against the emitter's fixed-output set — a role added to that switch without
being added here fails, which is the emitter-drift protection the docblock
was promising but not delivering.

Re-measuring over the complete tables also corrected the header's own
numbers. The claim was "71 of 73 agree, 2 disagree"; over 75 names it is 71
agree and FOUR disagree, because AXSystemDialog and AXUnknown were absent
from the old table. Those two are the behavioral delta of this PR — an
AXWindow whose subrole is emitted as the type, invisible to the six
type-only spellings and named exactly by `role` — so the incomplete table
had been hiding the very rows that justify reading role/subrole. The other
two (AXFloatingWindow, AXSystemFloatingWindow) remain inert: only the `===`
spelling misses them and its platform union is `android | ios`.
Two test-validity defects from review, both real.

The raw-fallback row `{ type: 'AXSearchField', role: 'AXTextField', subrole:
'AXSearchField' }` was unreachable: the `AXTextField` arm returns `TextField`
whatever the subrole, so no emitter run can produce it. Replaced with
`{ type: 'AXSortButton', role: 'AXCell', subrole: 'AXSortButton' }` — a
subrole on a genuinely unmapped role, which is what the `subrole ?? role`
default arm actually emits.

`MACOS_FIXED_OUTPUTS` was a hand-kept twin compared against a hand-kept
table, which is circular: a new mapped Swift role is absent from BOTH, so
they agree and the gate stays green. The "emitter-drift protection" the
docblock promised did not exist. The set is now parsed out of
`normalizedSnapshotType` in SnapshotTraversal.swift, so the comparison is
against the emitter rather than against a copy of the table's own
assumptions. `case "AXWindow"` returns a subrole expression rather than a
literal and is deliberately outside the literal-return set.

Red evidence: adding `case "AXDisclosureTriangle": return "DisclosureTriangle"`
to the Swift switch fails with `expected [ 'DisclosureTriangle' ] to deeply
equal []`; 6 pass once reverted. The parser throws rather than silently
matching nothing if the function is renamed or moved.
@thymikee
thymikee force-pushed the claude/unify-snapshot-viewport-geometry branch from 673713d to 84fbb8a Compare August 5, 2026 14:21
@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Rebased onto d8b309c at 84fbb8a4. The conflict was the one #1614 created: this branch had added isViewportRootNode to facade-symbols.ts, and #1614 retired that table. Resolved by keeping it deleted — the symbol is named explicitly in facades/snapshot.ts, and #1614's new exhaustiveness gate passes on it (13/0).

Smoke Tests red is not this change. testBareTypeUsesTappedInputWhenSoftwareKeyboardIsHidden fails with typed text truncated: ("hardw") is not equal to ("hardware-keyboard"). This PR is TypeScript viewport-root predicates and cannot reach Swift text entry.

Worth noting the truncation length differs per run — you saw "h" on #1614, this run gives "hardw" (5 of 18 chars). A deterministic break would truncate identically; a changing prefix is a timing-dependent typing race on the simulator. #1614 merged with the same failure. Re-run requested.

That said, "typed text silently truncates on iOS" is worth its own issue independent of any of these PRs if it is not already tracked — it is a correctness symptom, not just CI noise.

🤖 Addressed by Claude Code

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Changed-head re-review of 84fbb8a4: the #1614 conflict resolution is correct—the retired façade pin table stays deleted, isViewportRootNode is explicitly exported, and the full-viewport-chrome exclusion is preserved.

The requested cleanup is still outstanding: the same 44 unrelated scripts/maestro-conformance/corpus/** formatting changes remain, and the PR body still reports 73 emitted names / 71 agreements / 2 exceptions (24 macOS forms) instead of the validated 75 / 71 / 4 (26 macOS forms). Please restore the corpus files to main and correct the body.

Android Smoke failed during emulator startup/initial capture with wait_capture_stalled and readableCaptures: 0; that is infrastructure, not a code finding in this downstream predicate refactor. iOS Smoke was cancelled before tests ran because of workflow concurrency, so it provides no result on this head. No ready-for-human until the requested code/body cleanup lands and the authoritative Android/iOS lanes are green.

@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Smoke red again, but a different failure from the last one, and this one is infrastructure.

14:24:02  ERROR | Unable to connect to adb daemon on port: 5037
14:25:46  ✖ live Android emulator fixture E2E (43235ms)
          AssertionError: step: wait for Automation lab
          "message": "wait timed out for text: Automation lab"

adb was unreachable 104 seconds before the wait failed — nothing was ever driven on the device, so the wait text timeout is a downstream symptom, not the cause. Note also the tail: ERROR | stop: Not implemented.

Cross-check: #1615 is on the same base, also touches daemon code, and has all five smoke checks green. So this is per-run, not per-branch.

That is now three distinct Smoke failures on this PR across three runs — iOS typing truncation ("h", then "hardw"), and now Android adb-daemon-unreachable. Three different subsystems, three different runs, on a change that only rewrites TypeScript viewport-root predicates. Re-run requested.

Re: the size report bot above — no action taken, it reports a decrease (JS raw −876 B, gzip −196 B), which is the nine deleted predicate copies. Nothing to address.

🤖 Addressed by Claude Code

45 corpus YAMLs carried an unrelated quote-style churn ("Button" ->
'Button'). They were already modified in the worktree when this branch
started and a `git add -A` swept them into the predicate commit. Nothing
in this PR reads them. Restored verbatim to main.
@thymikee

thymikee commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Both cleanups landed on 7e9e8bf3.

Corpus restored. You were right and the cause is worth naming: the 45 corpus YAMLs carried an unrelated quote-style churn ("Button"'Button') that was already modified in the worktree before this branch existed, and a git add -A swept it into the predicate commit. Nothing in this PR reads those files. Restored verbatim from maingit diff origin/main..HEAD -- scripts/maestro-conformance/** is now empty. Kept as its own commit rather than rewriting history, so the sweep is visible instead of hidden.

Body corrected, and recounted from the table rather than retyped: 31 iOS + 18 Android + 26 macOS = 75, 71 agreements, 4 exceptions. The body now also splits the four the way the test docblock does — two inert (AXFloatingWindow/AXSystemFloatingWindow, missed only by the === spelling that cannot see macOS) and two real (AXSystemDialog/AXUnknown, the behavioral delta). I left an explicit note that the old 73/2/24 figures came from the incomplete table, since those numbers appeared in the commit message history too and a reader will hit them.

On the CI lanes: agreed both were non-results — Android died in emulator startup with readableCaptures: 0, iOS was cancelled by workflow concurrency before tests ran. Re-runs are queued; I will not call this ready until both report on this head.

🤖 Addressed by Claude Code

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Aug 5, 2026
@thymikee
thymikee merged commit 8c800ae into main Aug 5, 2026
31 checks passed
@thymikee
thymikee deleted the claude/unify-snapshot-viewport-geometry branch August 5, 2026 16:17
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-05 16:17 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant