Skip to content

fix(review): unlinked-issue-match rate ceiling accumulates spend even when no AI binding is available to verify #8356

Description

@JSONbored

Context

src/review/unlinked-issue-guardrail.ts, resolveUnlinkedIssueMatchDisposition (lines ~222-260). For each candidate issue in the loop, line 245 calls await recordUnlinkedIssueVerifyAttempt(env, ...) — which increments the per-actor rate ceiling that isOverUnlinkedIssueVerifyRateCeiling reads — unconditionally, before line 246 even checks hasUnlinkedIssueVerifyAiBinding(env):

for (const candidate of candidates) {
  if (authorLogin) await recordUnlinkedIssueVerifyAttempt(env, input.repoFullName, input.pullNumber, authorLogin);
  const hadAiBinding = hasUnlinkedIssueVerifyAiBinding(env);
  const verdict = await verifyUnlinkedIssueMatch(env, { ... });
  ...

hasUnlinkedIssueVerifyAiBinding (lines 83-86) checks whether env.AI.run is a callable function — i.e. whether the Workers AI binding is actually configured. Self-host is this repo's primary shipped deployment target, and the AI binding is an optional/configurable piece of a self-host install, so an operator running without an AI binding configured is a real, reachable state. In that state verifyUnlinkedIssueMatch fails closed to NO_MATCH for every candidate (no binding to call), producing zero real verification value — yet recordUnlinkedIssueVerifyAttempt still records the attempt and consumes rate-ceiling budget as if a real AI call had been considered. A burst of unrelated PRs from the same contributor during a period with no AI binding configured can exhaust that contributor's hourly rate ceiling on pure no-op checks, so once the operator configures an AI binding (or a transient outage clears), that contributor's PRs get held on unlinkedIssueVerifyCapacityHold("rate") instead of getting a genuine verification pass — purely because of unrelated attempts that accomplished nothing.

Notably, the separate spend-budget call recordUnlinkedIssueVerifyUsage (line 255) already gets this right — it's gated on hadAiBinding with an explicit comment: "Record spend only after an actual verifier invocation was possible. Missing/no-op AI bindings should fail closed to NO_MATCH without burning the shared budget ledger as if an ok call occurred." The per-actor rate-ceiling attempt counter needs the same treatment.

Requirements

  • Move the hasUnlinkedIssueVerifyAiBinding(env) check to before recordUnlinkedIssueVerifyAttempt is called inside the candidate loop, so a no-AI-binding environment never increments the per-actor rate-ceiling counter.
  • When there is no AI binding, the loop must still produce its existing documented outcome for that candidate (fail closed to NO_MATCH) — just without recording a rate-ceiling attempt, mirroring the pattern already used for recordUnlinkedIssueVerifyUsage.
  • Preserve existing behavior for every candidate loop iteration when an AI binding is present — this change is scoped to not spending rate-ceiling budget on no-op attempts, not to altering any behavior in the normal (AI-available) path.

Deliverables

  • hasUnlinkedIssueVerifyAiBinding(env) is checked before recordUnlinkedIssueVerifyAttempt is called inside the candidate loop of resolveUnlinkedIssueMatchDisposition.
  • New test (in whichever test file covers this module — confirm via grep -rl "unlinked-issue-guardrail\|resolveUnlinkedIssueMatchDisposition" test/) asserting: with env.AI absent/non-callable, repeated calls to resolveUnlinkedIssueMatchDisposition for the SAME actor across many candidates never trip isOverUnlinkedIssueVerifyRateCeiling for that actor; and once a binding IS present, a genuine verification attempt is recorded normally.

Test Coverage Requirements

99%+ Codecov patch coverage, branch-counted, on every changed line/branch in src/review/unlinked-issue-guardrail.ts, per codecov.yml.

Expected Outcome

A self-host install running without an AI binding configured (or during a transient outage) no longer silently exhausts contributors' unlinked-issue-match rate-ceiling budget on attempts that could never have produced a real verification result, preserving that budget for genuine AI-backed checks once a binding is available.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions