Skip to content

ci: device E2E smoke tests for every hosted-CI platform#371

Draft
latekvo wants to merge 2 commits into
mainfrom
e2e-device-smoke
Draft

ci: device E2E smoke tests for every hosted-CI platform#371
latekvo wants to merge 2 commits into
mainfrom
e2e-device-smoke

Conversation

@latekvo

@latekvo latekvo commented Jun 18, 2026

Copy link
Copy Markdown
Member

What

wayland-e2e.yml boots an Android AVD on Linux under headless Weston and asserts the full pipeline works without a display. It guards exactly one cell of the support matrix. This adds the same boot→screenshot→gesture-tap smoke test for every other (target × host) cell that can run on GitHub-hosted runners.

target host job status
Android emulator Linux (KVM) wayland-e2e.yml (existing)
iOS simulator macOS ios-sim-macos
Chromium / Electron Linux (Xvfb) chromium-linux
Chromium / Electron macOS chromium-macos

How

  • scripts/e2e/drive-device.sh — shared driver. Boots one device through the tool-server, then asserts booted:true → screenshot has real (non-blank) pixels → gesture-tap round-trips. Every tool takes the device id as udid and screenshots come back as the same data.image.hostPath envelope on all platforms, so the body is uniform; each job just supplies the cell-specific boot JSON + device id.
  • packages/tool-server/test/fixtures/electron-smoke-app/ — minimal self-contained Electron app the Chromium jobs point electronAppPath at. It vendors its own electron via its own lockfile (kept out of the repo root lockfile); CI npm cis it so boot-electron resolves ./node_modules/.bin/electron.
  • .github/workflows/e2e-device-smoke.yml — three jobs, workflow_dispatch + path-filtered pull_request. iOS downloads the darwin native binaries (injection must succeed for bootIos to report booted:true); Chromium-Linux runs under Xvfb with --no-sandbox. The tool-server is started with TS_NODE_TRANSPILE_ONLY=1 so cold ts-node startup doesn't flakily exceed the readiness poll on loaded macOS runners (types are still enforced by the typecheck job).

Known gaps (supported on real machines, not runnable on hosted runners)

  • Android emulator on macOS — the arm64 emulator needs HVF (-enable-hvf), but hosted macOS runners are themselves VMs with no nested virtualization. Confirmed by running it: qemu dies with HVF error: HV_UNSUPPORTED ~18s into boot, regardless of GPU mode or RAM. The Android boot/screenshot/tap path is already regression-guarded on Linux via KVM (wayland-e2e.yml), so macOS adds no runnable coverage.
  • Physical devices — real iOS (CoreDevice HID) and real Android (adb-USB) have no attached hardware on hosted runners.

Both would need self-hosted runners (a Mac exposing HVF / wired-up devices). Documented in the workflow header.

Verification

  • All checks green: the 3 device jobs + ESLint + Prettier + Unit tests.
  • Locally (before CI): ran drive-device.sh end-to-end against a real tool-server → boot booted:true, screenshot 142 KB (floor 20 KB), tap tapped:true, exit 0; and confirmed via the Argent stack that a tap at (0.5, 0.5) lands on the fixture's centered button and increments its counter.

@latekvo latekvo force-pushed the e2e-device-smoke branch 5 times, most recently from 05126ff to 2296695 Compare June 18, 2026 14:10
Extends the Android-on-Linux coverage in wayland-e2e.yml to the rest of the
supported (target x host) matrix that can run on GitHub-hosted runners:

  - iOS simulator on macOS
  - Chromium/Electron on Linux (Xvfb)
  - Chromium/Electron on macOS

Each job boots one device through the tool-server and asserts the headless
interaction pipeline via a shared scripts/e2e/drive-device.sh: booted:true ->
screenshot returns real pixels -> gesture-tap round-trips. The Chromium cells
point at a minimal self-contained Electron fixture that vendors its own
electron (kept out of the root lockfile).

Documented as known gaps (supported on real machines, not runnable on hosted
runners): Android emulator on macOS (the arm64 emulator needs HVF, but hosted
macOS runners are VMs with no nested virtualization -> HV_UNSUPPORTED; the
Android path is already guarded on Linux via KVM), and physical iOS/Android
devices (no attached hardware).
@latekvo latekvo force-pushed the e2e-device-smoke branch from 2296695 to bf27031 Compare June 18, 2026 14:15
@latekvo latekvo marked this pull request as ready for review June 18, 2026 15:41
@latekvo latekvo marked this pull request as draft June 18, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant