Run the real-resource tests on Linux only - #16
Conversation
Five continuous-integration failures since CI was added were all on Windows runners, all in tests that depend on a real child process, a real socket, or a real browser, a different test each time, every one passing on rerun: - e2e: launcher Search activates the offline Obstacle package (element absent) - e2e: triggered motion shell actions (1500 ms wait elapsed) - node: real-Chrome containment probe (10s DevTools bound), twice - native: host_api browser authorization preflight (socket write failed) These are not five defects. They are one property of the suite: integration tests that need real operating-system resources are only as stable as the runner they land on, and none of them ever caught a defect that Linux did not. The one real layout defect in that suite, a 720p text collision, reproduced only under Linux font metrics and was found by the Linux job. A required check that fails spuriously trains people to ignore it. Linux is the product target: the Raspberry Pi 5 and ordinary x86-64 Linux. Windows is the development host, and it keeps every deterministic check -- format, clippy, build, typecheck, and the computational test suite. That is where it earns its place: `clippy::large_enum_variant` on InstalledRuntime is invisible on Linux, where PathBuf is 24 bytes rather than 32. - The end-to-end suite now runs on Linux only. - The real-Chrome containment probes honour VCG_SKIP_REAL_BROWSER_TESTS, set only on the Windows node job. An opted-out run reports the test as skipped, so no assertion is ever disabled silently, and the probe still runs on Linux and on any developer machine. - `e2e / windows-latest` is removed from the branch-protection required checks, since that context no longer reports. Not addressed here: the host_api preflight socket failure. It flaked once in roughly six runs and I do not understand the cause, so it keeps running on both platforms rather than being retried or skipped. Adding a retry to a failure I cannot explain would hide it. If it recurs there will be more evidence to work from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 18 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
1 participant
Five CI failures since CI was added were all on Windows runners, all in tests that depend on a real child process, socket, or browser, a different test each time, every one passing on rerun:
host_apiauthorization preflightThese are not five defects. They are one property of the suite: integration tests needing real OS resources are only as stable as the runner they land on — and none ever caught a defect Linux did not. The one real layout defect in that suite, a 720p text collision, reproduced only under Linux font metrics and was found by the Linux job.
A required check that fails spuriously trains you to ignore it, which is worse than not having it.
Change
Linux is the product target (Pi 5, ordinary x86-64 Linux). Windows is the dev host and keeps every deterministic check — fmt, clippy, build, typecheck, and the computational suite. That is where it earns its place:
clippy::large_enum_variantonInstalledRuntimeis invisible on Linux, wherePathBufis 24 bytes rather than 32.VCG_SKIP_REAL_BROWSER_TESTS, set only on the Windows node job. An opted-out run reports the test as skipped, so no assertion is ever disabled silently — verified in both directions. The probe still runs on Linux and on any developer machine.e2e / windows-latestremoved from branch-protection required checks (now 7), since that context no longer reports. Done ahead of this PR, otherwise it could never satisfy protection.Deliberately not addressed
The
host_apipreflight socket failure. It flaked once in ~6 runs and I do not understand the cause, so it keeps running on both platforms rather than being retried or skipped. Adding a retry to a failure I cannot explain would hide it. If it recurs there will be more evidence to work from.Verified
Full Windows suite with the opt-out: exit 0. Opt-out honoured in both directions.
🤖 Generated with Claude Code