[Improve] Surface routing failures instead of silently showing the picker - #1120
Merged
Conversation
…cker
Routing fallbacks caused by infrastructure errors (provider outages, key
limits) were indistinguishable from the router legitimately declining to
pick a workspace: the reason string was dropped, users just saw the manual
picker, and the router debug channel showed "(none)".
- Add RoutingFallbackCause ('exception' | 'model_decision') to fallback
decisions and set it at every construction site; include cause= in the
routing fallback logs
- Show a routing-unavailable warning above the Slack manual picker when the
fallback came from an exception (both the mention flow and channel
auto-start), keeping raw provider errors out of user threads
- Post a router-fallback diagnostic to the configured router debug channel
at fallback time, including the full failure reason
Contributor
|
No new code issues found. See task
Reviewed f548e7 |
Slack context setup failures (thread fetch, channel lookup) in the outer catch no longer claim the routing infrastructure is down; only a thrown router invocation maps to cause 'exception'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When the router's LLM call fails for infrastructure reasons (inference provider outage, key limit, timeout), the failure was invisible everywhere a human might look:
reasonwas dropped in the Slack start path, so users just got the manual workspace picker with no explanation — indistinguishable from an ambiguous request.Why this route: (none)because fallback decisions carryreason, notreasoning, and nobody passed it through.During a recent provider-key outage this made routing look "broken" with no user-facing signal about why, and the deploy that happened around the same time was a red herring.
Changes
Typed failure cause. Fallback decisions now carry
cause: 'exception' | 'model_decision'(RoutingFallbackCause). Catch blocks markexception; "router ran but declined to pick" paths markmodel_decision. The[LLM Router] Routing fallbacklog lines includecause=.User-visible warning on the picker. When the cause is an exception, the Slack manual picker now shows a notice that automatic routing is temporarily unavailable and an admin should check the deployment's inference provider. Wired through both entry paths: the mention flow (
showTaskConfiguration, including its catch block) and channel auto-start (startAutoRoutedSlackTask→showManualPickerForAutoRouteFallback). Raw provider errors are deliberately kept out of user threads since they can contain key identifiers; ambiguity-driven fallbacks still show the plain picker.Debug channel gets the real reason. New
postRouterFallbackDebugMessagefires at fallback time on every fallback, posting source, message, cause, and the full failure reason to the configured router debug destination (Slack blocks + plain-text variants for Discord/Telegram/Teams). Previously nothing was posted at fallback time at all.Tests
router-service.test.ts: exception fallback now assertscause: 'exception'show-task-configuration.test.ts: picker warning + fallback diagnostics for exception fallbacks; plain picker for model-decided fallbacks; notice passthrough withskipRoutingauto-route-fallback.test.ts: warning passthrough for exception cause, none for model decisionspnpm lint,pnpm check-types,pnpm knip, and the full@roomote/slacksuite pass.Follow-up (not in this PR)
Treating repeated non-task inference failures as provider health (web-app banner, Slack warning text) so this whole class of outage self-diagnoses across surfaces beyond routing.