chore(ci): consolidate the 3-shard test matrix into one unsharded validate-tests run#8514
Merged
Conversation
…idate-tests run At the one-shot gate's PR volume the 2 extra runners per PR were a major driver of org-wide Actions concurrency saturation (observed: 60-run queue), and the shard seams were a recurring flake/complexity source. One slower job (~10-15 min added PR latency, timeout raised to 45) buys: - 2 fewer runners per PR, every PR - ONE honest whole-suite lcov (the exact signal npm run test:coverage gives locally), replacing 3 additive shard uploads -- codecov.yml's after_n_builds drops 3 -> 1 (the new always-lands floor; the narrow rees/control-plane premature-window this reopens self-corrects and is documented in place) - the global 90% threshold runs INLINE in full-suite runs (scoped runs keep COVERAGE_NO_THRESHOLDS via the same expression the merge job used), so validate-tests-merge is deleted outright - the whole duration-aware sharding stack retires: compute-test-shards.ts, fetch-test-timing.ts, test-timing-refresh.yml (one fewer scheduled workflow), their suites, and the blob-report artifact plumbing Codecov flags: shard-N -> backend. codecov-policy pins updated (the merge job's absence is now itself pinned).
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
…s job Four tests across workflow-runner-labels, ci-skip-draft-prs, and ci-composite-setup-workspace still pinned the 3-shard shape (the merge job in validate's needs, the merge job's draft-skip condition, and its save-cache:false composite call). All now pin the single-job topology; codecov-policy already pins the merge job's absence.
loopover-orb
Bot
added
the
gittensor:bug
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8514 +/- ##
===========================================
+ Coverage 80.35% 92.53% +12.18%
===========================================
Files 793 793
Lines 79483 79527 +44
Branches 24018 24030 +12
===========================================
+ Hits 63867 73591 +9724
+ Misses 12673 4799 -7874
+ Partials 2943 1137 -1806
Flags with carried forward coverage won't be shown. Click here to find out more. |
JSONbored
added a commit
that referenced
this pull request
Jul 24, 2026
…o changes, make selfhost push-only, one-runner same-repo previews, merge the calibration advisories (#8523) Follow-up to the validate-tests unsharding (#8514), attacking the remaining per-PR runner fan-out at the source (observed 44-deep queue with 3 running): 1. The standalone `security` job folds into `changes` as a PR-only dependency-review step -- one fewer runner slot on EVERY PR at zero wall-clock cost (the API lockfile diff takes seconds). `validate`'s needs drops it; a dep failure still reddens `validate` via `changes`. 2. selfhost.yml drops its pull_request trigger (~99 x 5.2min/week -- migrations/** made nearly every backend PR pay a docker build+boot). ci.yml still validates migrations/schema/pg suites pre-merge; the boot smoke now catches a break minutes after merge, and self-host users only consume tagged releases, never main. 3. Same-repo UI PRs build AND deploy their preview in ONE job: the deploy half moves to a shared .github/actions/deploy-ui-preview composite (setup-workspace precedent), ui-preview.yml gains the inline build-deploy job, and ui-preview-deploy.yml becomes fork-only (its job skips runner-free for same-repo builds via head_repository comparison). The fork trust boundary is untouched: fork code still builds with zero secrets and deploys only through the trusted workflow_run half. Bonus: the deploy job now has a trusted default-branch checkout, retiring the hand-synced hardcoded Node version. 4. backtest-logic-check.yml + counterfactual-replay-check.yml merge into calibration-advisory.yml -- identical scaffolding, overlapping paths (ai-review.ts fired BOTH), so one job runs both halves for one npm ci and one runner slot. The replay half keeps its budget spend-guard and now self-gates on the canonical prompt actually differing (replacing its ai-review.ts-only path filter). Fail-open posture preserved verbatim on every advisory step. Net per-PR floor drops 5 -> 4 slots, UI PRs 7 -> 5, migration PRs lose the 5-minute docker job. Pinned tests updated (ci-skip-draft-prs, workflow-runner-labels); skill reference.md rows updated.
This was referenced Jul 24, 2026
JSONbored
added a commit
that referenced
this pull request
Jul 24, 2026
…toring both coverage attribution and subprocess artifacts (#8568) Second, complete fix for the 2026-07-24 codecov/patch (0%/99%) incident. The first fix (#8564) deleted the miner/MCP in-place build output before the coverage run, which restored .ts coverage attribution and --changed tracing -- but broke the other class of tests: subprocess-spawning ones (the CLI harnesses and MCP stdio tests) run under plain Node outside Vite's resolver and genuinely need the built lib/bin .js at spawn time (node's ESM resolver does not fall back from a .js specifier to a .ts sibling). Scoped post-fix runs failed in miner-discover-cli/--help, miner-init-wizard e2e, and all 14 mcp-feasibility-gate stdio tests. This replaces that approach: a resolve-stage vitest plugin rewrites relative .js-suffixed imports that land inside packages/loopover-{miner,mcp}/{lib,bin} to their .ts sibling when it exists, so in-process imports always carry the source identity (correct v8 attribution + import-graph tracing) while the built .js stays on disk for spawned subprocesses. The #8564 ci.yml surgery (pre-coverage clean, unconditional check-miner-package exclusion, trailing pack-integrity step) is reverted -- the workflow returns to the #8514 shape, with the plugin making it correct.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Queue-pressure + reliability consolidation (follows the CodeQL default-setup removal and #8512's batched cache cleanup): at the one-shot gate's PR volume, the 2 extra shard runners per PR were a major driver of org-wide Actions concurrency saturation (observed: 60-run queue), and the shard seams were a recurring flake/incident source. One slower job (~10–15 min added PR latency; timeout 45) buys:
npm run test:coveragegives locally — replacing 3 additive shard uploads.codecov.yml'safter_n_buildsdrops 3 → 1 (the new always-lands floor; a floor above it would leavecodecov/patchpermanently un-posted on non-rees/control-plane PRs). The narrow premature-window this reopens on rees/control-plane PRs self-corrects on the backend upload and is documented in place;require_ci_to_passstill holds the final verdict.COVERAGE_NO_THRESHOLDSvia the exact expression the merge job used) — sovalidate-tests-mergeis deleted outright, along with the blob-report artifact plumbing.compute-test-shards.ts,fetch-test-timing.ts,test-timing-refresh.yml(one fewer scheduled workflow), and their suites.shard-N→backend.codecov-policy.test.tspins updated — the merge job's absence is now itself pinned so a stale resurrection can't slip back in.validateaggregator'sneedsdrops the merge job; scoped-selection, harness-skip narrowing, no_tests_matched/no_src_coverage escapes, and the fork/tokenless upload split are all preserved verbatim.Validation
actionlintclean;codecov-policysuite 8/8 green against the new topology; stale-reference sweep clean (remaining "shard" mentions in ci.yml are historical rationale comments, with the job banner rewritten). This PR's own CI run is the live proof of the single-shard path end-to-end — including the first inline whole-suite threshold check and the singlebackend-flag Codecov upload.