Skip to content

E2E suite router#628

Merged
jasonmadigan merged 5 commits into
Kuadrant:mainfrom
Anton-Fil:e2e-suite-router
Jul 13, 2026
Merged

E2E suite router#628
jasonmadigan merged 5 commits into
Kuadrant:mainfrom
Anton-Fil:e2e-suite-router

Conversation

@Anton-Fil

@Anton-Fil Anton-Fil commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a changed-file inference suite router that maps source file changes to relevant e2e spec files. Instead of running all smoke tests on every PR (~11 min), the CI now detects which components changed and runs only the relevant spec files

Fixes #587

Type of change

  • [fix] Bug fix
  • [feat] New feature
  • [refactor] Refactor (no functional changes)
  • [test] Test updates
  • [chore] Dependency / config update

Changes made

  • Add build/suite-router.sh — bash script that maps changed source paths to relevant e2e spec files; outputs space-separated paths or empty string for full smoke fallback
  • Add route job to e2e.yaml that runs the suite-router and passes its output to the smoke job
  • Add optional specs input to e2e-common.yaml — if non-empty, runs only the specified spec files with --grep @smoke; falls back to all smoke tests when empty

Test plan

  • Tested build/suite-router.sh locally — correct spec files returned for apikey component changes
  • Tested fallback — empty output for src/utils/ and unknown paths
  • Verified route job output in GitHub Actions

Screenshots

i crated test pr in fork for show how its work

Anton-Fil#5

image

Summary by CodeRabbit

  • Tests

    • End-to-end smoke testing now selects only the relevant test specifications for each change, helping reduce validation time.
    • Changes affecting shared infrastructure or with no clear test mapping continue to run the complete smoke suite.
    • Nightly and full test runs remain unchanged.
  • Documentation

    • Added guidance on assigning smoke and nightly tags to end-to-end tests.
    • Documented how test selection is determined for pull requests and how new tests should be registered.

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Anton-Fil, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 349faaf6-84bd-4fac-8785-e570350665a2

📥 Commits

Reviewing files that changed from the base of the PR and between d05428e and eb93cd3.

📒 Files selected for processing (4)
  • .github/workflows/e2e-common.yaml
  • .github/workflows/e2e.yaml
  • CLAUDE.md
  • build/suite-router.sh
📝 Walkthrough

Walkthrough

Changes

E2E smoke routing

Layer / File(s) Summary
Changed-file suite routing
build/suite-router.sh
Detects changed files, applies full-smoke fallbacks for high-impact or unmapped changes, and emits deduplicated component-specific Playwright specs.
Workflow filtering and conventions
.github/workflows/e2e.yaml, .github/workflows/e2e-common.yaml, CLAUDE.md
Runs the router, passes its output to the reusable workflow, conditionally filters smoke tests, and documents the @smoke/@nightly tagging and routing rules.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • #584 — Covers the changed-file inference and smoke-test routing approach implemented here.

Possibly related PRs

Suggested reviewers: r-lawton, emmaaroche

Poem

A rabbit mapped the files with care,
Sending only the needed tests there.
Smoke tags sparkle, fallbacks guard,
Full suites wait when paths are hard.
Hops through CI, the checks now flow!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes implement changed-file inference, route to relevant smoke specs, and preserve full-smoke fallback as required by #587.
Out of Scope Changes check ✅ Passed The docs update in CLAUDE.md supports the new E2E routing flow and no unrelated changes are evident.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarises the main change: adding a suite router for E2E test selection.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jasonmadigan

Copy link
Copy Markdown
Member

nice

@emmaaroche

Copy link
Copy Markdown
Member
clock it

@Anton-Fil
Anton-Fil marked this pull request as ready for review July 13, 2026 14:24
@Anton-Fil
Anton-Fil requested review from R-Lawton and jasonmadigan and removed request for jasonmadigan July 13, 2026 14:24

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/e2e-common.yaml:
- Around line 78-83: Update the workflow step around SPEC_FILTER to receive the
input through the step’s env configuration, then reference the environment
variable in the shell rather than interpolating inputs.specs directly. Preserve
the existing conditional behavior and Playwright commands for empty and
non-empty filters.

In @.github/workflows/e2e.yaml:
- Around line 20-22: Update the actions/checkout@v4 step in the route job to set
persist-credentials to false, while retaining fetch-depth: 0 for git diff
access.
- Around line 15-29: Add an explicit permissions block to the route job,
granting only contents: read for checkout and repository diff access. Keep the
existing route steps, outputs, and token permissions otherwise unchanged.

In `@CLAUDE.md`:
- Line 247: Update the COMPONENT_MAP guidance in CLAUDE.md to match the bash
3.2-compatible implementation in build/suite-router.sh: replace
associative-array syntax with instructions and an example showing a new if/grep
-qE block that appends the relevant spec to SPECS, while preserving the
surrounding routing behavior and terminology.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c9b080f4-9e54-4153-b3b2-694dbf34c840

📥 Commits

Reviewing files that changed from the base of the PR and between 7162dc2 and d05428e.

📒 Files selected for processing (4)
  • .github/workflows/e2e-common.yaml
  • .github/workflows/e2e.yaml
  • CLAUDE.md
  • build/suite-router.sh

Comment thread .github/workflows/e2e-common.yaml Outdated
Comment thread .github/workflows/e2e.yaml
Comment thread .github/workflows/e2e.yaml
Comment thread CLAUDE.md
@jasonmadigan

jasonmadigan commented Jul 13, 2026

Copy link
Copy Markdown
Member

👀
👁️‍🗨️
👓

jasonmadigan
jasonmadigan previously approved these changes Jul 13, 2026

@jasonmadigan jasonmadigan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some small stuff, no biggies. would like to merge.

Comment thread CLAUDE.md
- Gateway and policy management
- Topology visualization

### E2E Test Tags

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread build/suite-router.sh Outdated
fi

if echo "$CHANGED" | grep -qE "^src/components/apiproduct/"; then
SPECS="$SPECS apiproduct-crud.spec.ts apiproduct-overview-tab.spec.ts api-product-list.spec.ts"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apiproduct-rbac.spec.ts missing?

Comment thread CLAUDE.md Outdated
```

**Rules:**
- Every test must have exactly one tag (`@smoke` or `@nightly`) — untagged tests are skipped in both runs

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

untagged tests are skipped in both runs

iirc nightly runs npx playwright test with no grep (e2e-common.yaml:88), so untagged tests do run nightly.

@Anton-Fil Anton-Fil changed the title E2e suite router E2E suite router Jul 13, 2026
Signed-off-by: Anton-Fil <a.filkach@gmail.com>
Signed-off-by: Anton-Fil <a.filkach@gmail.com>
Signed-off-by: Anton-Fil <a.filkach@gmail.com>
Signed-off-by: Anton-Fil <a.filkach@gmail.com>
…correct tag docs

Signed-off-by: Anton-Fil <a.filkach@gmail.com>
@jasonmadigan
jasonmadigan enabled auto-merge July 13, 2026 15:22
@jasonmadigan
jasonmadigan added this pull request to the merge queue Jul 13, 2026
Merged via the queue into Kuadrant:main with commit 09b1c97 Jul 13, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

e2e: changed-file inference — run only relevant e2e tests per PR

3 participants