Skip to content

[Phase 7] Soft-launch E2E PR checks#364

Draft
kieran-osgood-shopify wants to merge 1 commit into
mainfrom
kieran-osgood/e2e-browserstack/phase-7-soft-launch
Draft

[Phase 7] Soft-launch E2E PR checks#364
kieran-osgood-shopify wants to merge 1 commit into
mainfrom
kieran-osgood/e2e-browserstack/phase-7-soft-launch

Conversation

@kieran-osgood-shopify

@kieran-osgood-shopify kieran-osgood-shopify commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Stack Context

This is Phase 7 of the BrowserStack Maestro E2E stack for Checkout Kit, stacked on GitHub E2E reporting.

Parent issue: https://github.com/shop/issues-checkout-kit/issues/1101
Epic: https://github.com/shop/issues-checkout-kit/issues/1084

What?

This PR adds the "pull_request" trigger to the bitrise config, launching the e2e pipeline when a PR is created
Skips draft PR's

  • E2E_STRICT=false means the failures are soft failures, when we're certain the pipeline is stable we can set E2E_STRICT=true, keeps assertion failures reportable without blocking initial PR progress
    (Technically even without E2E_STRICT they wouldn't be blocking because we havent marked the bitrise status checks as required, however it gives us an opt out in future too)
  • uses rolling builds to cancel BrowserStack builds on timeout or duplicate runs

Why?

The E2E pipeline should start non-blocking while the infrastructure and suite stabilize. Developers still get commit statuses, Check Runs, and actionable failure comments, but PR progress is not blocked until we are confident that the checks are free of false positives

Results

Topology on Bitrise
image.png

When one suite is failing
image

image

After fixing all the tests you'll see the workflow update the comment

image.png

@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from 33b81d9 to dc1c907 Compare June 29, 2026 16:04
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch 2 times, most recently from 6a6674a to 548ffe1 Compare June 29, 2026 16:38
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch 2 times, most recently from ea86286 to e791f00 Compare June 29, 2026 17:01
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch 3 times, most recently from 1f81b20 to c346dc8 Compare June 29, 2026 17:33
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from e791f00 to 3545355 Compare June 29, 2026 17:33
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from c346dc8 to c0cb990 Compare June 29, 2026 17:53
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch 2 times, most recently from 9bcd74d to 64cf7a4 Compare July 1, 2026 09:54
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch 2 times, most recently from 261b9df to 8cea139 Compare July 1, 2026 10:07
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch 2 times, most recently from 3895dbe to 9a25f02 Compare July 1, 2026 10:40
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch 2 times, most recently from 3baf4b0 to f219a0f Compare July 1, 2026 11:26
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from 9a25f02 to 8477810 Compare July 1, 2026 11:26
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from f219a0f to 9112818 Compare July 1, 2026 11:43
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from 8477810 to 3399b8d Compare July 1, 2026 11:43
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 9112818 to efb1d4f Compare July 1, 2026 13:41
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch 2 times, most recently from f29c0b4 to a9a9150 Compare July 2, 2026 13:25
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from f6f41e8 to d2ae60d Compare July 2, 2026 13:25
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from a9a9150 to 5cbc8cf Compare July 2, 2026 14:26
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch 2 times, most recently from 5b48786 to 7942882 Compare July 2, 2026 15:18
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 5cbc8cf to 219ee3f Compare July 2, 2026 15:18
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from 7942882 to 7d6f887 Compare July 2, 2026 15:35
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 219ee3f to 165945b Compare July 2, 2026 15:35
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from 7d6f887 to 06005bc Compare July 3, 2026 09:53
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 165945b to 98c8a2e Compare July 3, 2026 09:53
@bitrise

bitrise Bot commented Jul 3, 2026

Copy link
Copy Markdown

🤖 Bitrise Build Report > Checkout Kit E2E > e2e > #55: ✅ SUCCESS

  • Duration: 8m 48s
  • Workflows: 6 total / 6 executed / 0 aborted / 0 skipped / 0 failed / 6 successful

@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 98c8a2e to 8ce0b60 Compare July 6, 2026 08:33
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch 2 times, most recently from 8e949ef to df9ceff Compare July 6, 2026 11:18
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 8ce0b60 to 7e62db6 Compare July 6, 2026 11:18

Copy link
Copy Markdown
Contributor Author

^I have disabled these bitrise build report comments so its only the one we manually post above in future

@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from 7e62db6 to f214365 Compare July 6, 2026 12:33
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch from df9ceff to d2d5b3e Compare July 6, 2026 12:33
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch from f214365 to 8570f64 Compare July 6, 2026 15:16
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-6-github-reporting branch 2 times, most recently from 3d9c51f to ced0bde Compare July 6, 2026 16:27
@kieran-osgood-shopify kieran-osgood-shopify force-pushed the kieran-osgood/e2e-browserstack/phase-7-soft-launch branch 2 times, most recently from 9d963fc to bf49ea7 Compare July 6, 2026 16:36
Assisted-By: devx/6c1e3ad5-96c8-4972-b087-da7ff7b195c3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

#gsd:50662 Rebase Checkout Kit on UCP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant