Improve CI reliability and efficiency

[HamdaanAliQuatil]

CI is unreliable: tests fail randomly. Multiple issues:

• Tests run 2-3 times per job (VM twice, Chrome twice)
• Separate coverage jobs duplicate all test runs
• macOS times out after 15 minutes
• Mobile emulators are flaky

We plan to ship it in two parts:

Structural Cleanup:

• Add concurrency group (cancel superseded runs)
• Remove macos-14 (duplicate of macos-15)
• Merge linux-coverage and windows-coverage into main jobs
• Fix redundant test runs (VM/Chrome now run once per job)
• Make iOS/Android non-blocking (continue-on-error: true)

Emulator Reliability:
The goal is to make mobile tests deterministic by enabling retries and improving infrastructure.

Android

• Downgrade API 36 to 34 (36 has a known emulator issue; need to evaluate if it was upgraded for a specific reason)
• Cache AVD snapshots (actions/cache@v4)
• Add headless flags: -no-window -gpu swiftshader_indirect -noaudio
• Add 3-attempt retry loop

iOS

• Add 2-attempt retry wrapper (nick-fields/retry@v3)

Future Discussion:

• Debug macOS WASM hang (causes 15min timeout)
• Evaluate if Firefox is needed on all platforms
• Consider Linux-only coverage

Open for opinions

[harrshita123]

Hi ,
I am interested in working on this PR and would love to help move it forward. Please let me know if there is anything specific I can take up or assist with.

[HamdaanAliQuatil]

Hey @harrshita123, sure, we'd love some help!

For context, we are currently migrating to Dart-native build hooks. I've updated several tests in #236 that need to ship alongside those native hooks, so those specific test updates are currently blocked until the migration lands.

There are several CI infrastructure fixes we can land immediately. When I say the tests "fail randomly," here is exactly what's happening under the hood:

1. Test Timeouts: You might notice RSA-PSS: 4096/e3/sha-384/s0 -- generateKeyPair() timing out. RSA-4096 key generation is computationally expensive and frequently exceeds the default 30-second test timeout on loaded CI runners.
2. Emulator Flakiness: On iOS, the Xcode build will often succeed (say, after 60s), but the debug connection to the simulator takes too long to establish.

Both of these would immediately benefit from a retry policy in our workflow and from running the emulators in headless mode (experimental). If you want to tackle setting up the retries (like nick-fields/retry) or adding the headless flags to test.yaml, that would be a huge help.

[HamdaanAliQuatil]

Update on the Android API level: I traced the history and confirmed that API 36 was an intentional change.

#231 raised Android CI from API 28 to 36, following Flutter’s recommended setup.
#234 then kept API 36 and worked around the missing image issue by using the google_apis target, x86_64, and -d emulator.

The earlier note in this issue about “downgrade 36 to 34” should be treated as something to evaluate only if we find a current API-36-specific problem. At this point, keeping 36 looks like the better default.