Skip to content

[Enhancement]: Improve Instrumented Test Workflow Matrix & Failure Handling #24

@theMr17

Description

@theMr17

Summary of Improvement

Current Implementation

  • The current instrumented_tests job uses a hardcoded API level (api-level: [29]) without architecture control.
  • It uses reactivecircus/android-emulator-runner@v2 to spin up an emulator and run tests.
  • However:
    • There's no matrix for different arch/API combinations—limiting coverage.
    • API 33 is excluded, even though it's required for tests related to notifications.
    • Test command uses:
      ./gradlew connectedAndroidTest --daemon && killall -INT crashpad_handler || true
      The use of || true can silently ignore test failures, resulting in false-positive passes in CI.

Proposed Solution

  • Expand the matrix strategy with specific API/arch combinations using include, and drop reliance on a single hardcoded API level:
    strategy:
      matrix:
        include:
          - api-level: 29
            arch: x86
          - api-level: 33
            arch: x86_64
  • Ensure API 33 is part of the matrix to support notification-related test coverage.
  • Remove || true from the test command to allow failures to surface properly:
    ./gradlew connectedAndroidTest --daemon
  • Optional: switch to connectedDebugAndroidTest instead of the default connectedAndroidTest if that aligns with your Gradle configuration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementImprovements or additions to existing feature/code

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions