Skip to content

feat(review): evidence-weighted reviewer routing, stage 1 — the report-only shadow#8301

Merged
JSONbored merged 4 commits into
mainfrom
feat/reviewer-routing-shadow
Jul 24, 2026
Merged

feat(review): evidence-weighted reviewer routing, stage 1 — the report-only shadow#8301
JSONbored merged 4 commits into
mainfrom
feat/reviewer-routing-shadow

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Advances #8229 (stage 1 of the issue's two-stage contract; stage 2 — actual weighting behind a default-off flag with hard floors — ships only against this stage's recorded evidence, per the issue).

After each ok block-mode dual review, the orchestration computes what evidence-weighted routing WOULD have preferred for this repo and records it, changing nothing about the review:

  • computeWouldHaveRouted (pure): repo-scoped track-record rows only, ROUTING_MIN_DECIDED = 10 per (provider, repo) — the AUTOTUNE_MIN_DECIDED never-on-noise bar at reviewer grain, satisfying the issue's floors clause — and null on ties, lone reviewers, or any missing density. An absent record MEANS "no measurable preference", so stage 2's eventual evidence read is never diluted by no-signal rows.
  • loadLiveProviderTrackRecords: stage-0 reviewer_vote rows joined to the labeled consensus corpus via computeProviderTrackRecords (calibration: per-provider reviewer track records from the consensus corpus #8228, zero new math). Reads only the live vote event type — replay-derived campaign data never enters (the calibration: seed provider track records via per-provider counterfactual replay campaigns #8278 segregation rule enforced from the consuming side). Fail-safe empty on any store error.
  • Orchestration hook: best-effort end to end — zero added AI calls (pure DB reads + one audit write), and a thrown read, a rejecting write, or a synchronously-throwing writer all reduce to "no record" with the review path byte-identical. Runs only when ≥2 reviewer votes exist.
  • Maintainer recap section: the window's shadow decisions read back off the audit trail at format time, grouped per (repo, preferred provider) with the mean precision edge, an explicit empty line, and the report-only footer; fail-safe to an absent section on a read blip. Spend note honored: per-review call count unchanged.

Validation

  • Full local gate npm run test:ci fully green — TEST_CI_EXIT=0, zero vitest failures, zero node:test failures, npm audit --audit-level=moderate clean.
  • 100% line+branch on every changed range across all four touched src files (lcov vs exact diff hunks): every refusal arm of the preference rule, the malformed-vote-row skips, both fail-safe catches (async rejection AND sync throw), the recap section's empty/grouped/malformed-basis arms, and both sides of the orchestration's vote-count guard.
  • 45 tests across the new suite + the extended vote-capture/recap suites.

…t-only shadow (#8229)

After each ok block-mode dual review, compute what routing WOULD have
preferred for this repo from the live per-provider track records (#8228
over the stage-0 reviewer_vote rows) and record it — one audit event
(reviewer_routing_shadow) plus a maintainer-recap section — changing
NOTHING about the review:

- computeWouldHaveRouted: repo-scoped rows only, ROUTING_MIN_DECIDED (10,
  the AUTOTUNE_MIN_DECIDED never-on-noise bar at reviewer grain) per
  (provider, repo), null on ties/lone reviewers/any missing density — an
  absent record MEANS no measurable preference, keeping stage 2's eventual
  evidence read undiluted
- loadLiveProviderTrackRecords reads ONLY the live vote event type (the
  replay-derived campaign data never enters — the #8278 segregation rule
  from the consuming side); fail-safe empty on any store error
- orchestration hook is best-effort end to end: zero added AI spend, a
  thrown read or a rejecting/throwing audit write reduces to no record
- the weekly recap gains a routing-shadow section (grouped per repo +
  preferred provider with the mean precision edge, explicit empty line,
  report-only footer), read back off the audit trail at format time and
  fail-safe to an absent section

Stage 2 (actual weighting behind a default-off flag with hard floors)
ships only against this stage's recorded evidence, per the issue.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.96%. Comparing base (9adcccb) to head (e9205b3).
⚠️ Report is 28 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8301      +/-   ##
==========================================
- Coverage   92.50%   91.96%   -0.54%     
==========================================
  Files         791      793       +2     
  Lines       79333    79403      +70     
  Branches    23960    23980      +20     
==========================================
- Hits        73386    73023     -363     
- Misses       4807     5306     +499     
+ Partials     1140     1074      -66     
Flag Coverage Δ
shard-1 54.04% <92.50%> (-1.47%) ⬇️
shard-2 48.74% <65.00%> (-1.15%) ⬇️
shard-3 58.16% <62.50%> (+1.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/queue/ai-review-orchestration.ts 100.00% <100.00%> (ø)
src/services/maintainer-recap-routing.ts 94.11% <ø> (ø)
src/services/maintainer-recap.ts 97.22% <100.00%> (+0.34%) ⬆️
src/services/reviewer-routing.ts 100.00% <100.00%> (ø)

... and 4 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 24, 2026
@loopover-orb

loopover-orb Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-24 15:00:34 UTC

6 files · 2 blockers · CI green · clean

⏸️ Suggested Action - Manual Review

  • AI review could not be completed: The dual-model AI review did not return a usable verdict for this change.

Review summary
AI review could not be completed for this PR head. LoopOver is holding this PR for manual review instead of relying on deterministic signals alone.

Nits — 1 non-blocking
  • AI review could not be completed — The gate is held for a human reviewer rather than passed automatically; it re-evaluates on the next update.

Concerns raised — review before merging

  • No linked issue detected: No closing reference or linked issue number was found in the PR metadata/body. — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue: This repo's maintainer focus manifest requires every PR to reference a tracked issue. — Link the relevant issue (for example Closes #123) before opening the PR.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. No linked issue detected: No closing reference or linked issue number was found in the PR metadata/body. — If this PR is intended to solve an issue, link it explicitly in the PR body.

2. Maintainer requires a linked issue: This repo's maintainer focus manifest requires every PR to reference a tracked issue. — Link the relevant issue (for example `Closes #123`) before opening the PR.

Decision drivers

  • ❌ Code review — 2 blockers (no AI review ran — blocker is from a non-AI gate check)
  • ❌ Gate result — Blocking (Repo-configured hard blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 14 registered-repo PR(s), 14 merged, 346 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 14 PR(s), 346 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 14 PR(s), 346 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Then work through the remaining 3 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 24, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 24, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
loopover-ui e9205b3 Jul 24 2026, 03:41 PM

@JSONbored
JSONbored merged commit 353d019 into main Jul 24, 2026
10 of 11 checks passed
@JSONbored
JSONbored deleted the feat/reviewer-routing-shadow branch July 24, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant