Skip to content

Diagnose macOS Intel E2E flake (Electron quitting mid-test)#1084

Closed
rly wants to merge 8 commits into
mainfrom
diagnose/e2e-intel-electron-quit
Closed

Diagnose macOS Intel E2E flake (Electron quitting mid-test)#1084
rly wants to merge 8 commits into
mainfrom
diagnose/e2e-intel-electron-quit

Conversation

@rly

@rly rly commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Temporary diagnostic branch (not for merge) to pin the root cause of the long-standing E2ELiveServices / macos-15-intel flakiness.

Background

The intel E2E job fails intermittently (~4/5 runs on the #1080 conda branch, and similarly on the uv migration branch #1082). It is pre-existing and dependency-independent. The failure mechanism: Electron quits gracefully mid-test (app.on("before-quit")exitPyProc, [electron] Exited with code 0), which tears down the Flask backend and cascades every remaining test to failure (Failed to fetch, dashboard.page undefined, etc.). It is not the backend hot-reload watcher (ruled out with a targeted diagnostic) and not electron-vite reload (Electron does not restart after the exit).

What this adds

Under the test suite only (runByTestSuite), logs:

  • which signal (SIGINT/SIGTERM/SIGHUP) the Electron main process receives, if any (while preserving graceful shutdown);
  • render-process-gone / child-process-gone events (renderer/GPU crashes);
  • a stack trace when before-quit fires.

Once CI on the intel runner reveals the trigger, the real fix will replace these diagnostics.

🤖 Generated with Claude Code

rly and others added 8 commits July 15, 2026 10:40
The E2ELiveServices suite flakes on the macOS x64 runner: Electron quits
gracefully mid-test (before-quit -> exitPyProc, exit code 0), tearing down the
Flask backend and cascading the remaining tests. This is pre-existing (it fails
on conda too) and unrelated to dependencies. Log the quit path under the test
suite: which signal (if any) Electron receives, render/child-process-gone
events, and a stack trace when before-quit fires, so CI reveals the trigger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…anch, do not merge)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-raise SIGINT/SIGTERM/SIGHUP after logging (instead of calling app.quit) so
the diagnostic does not alter the default behavior. Also log app will-quit /
quit / window-all-closed and window/webContents lifecycle events
(render-process-gone, unresponsive, destroyed, did-fail-load, close/closed) so
the next mid-test quit on the Intel runner is captured with full context.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@rly

rly commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: served its purpose. Diagnosis: the macos-15-intel E2E flake is Chromium/Electron subprocess instability on that runner (network-service and GPU-process crashes), producing two modes — app quits gracefully mid-test, or the window never opens ("Network service crashed" → beforeStart timeout). It is pre-existing (fails on conda and uv alike) and not dependency-related. The fix is a job-level E2E retry, which will be developed on a fresh branch off main.

@rly rly closed this Jul 15, 2026
@rly
rly deleted the diagnose/e2e-intel-electron-quit branch July 15, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant