Problem
The llm_classifier route consults a small LLM to decide which tier serves a request. That call
needs instructions — a rubric telling the model what to weigh and what format to answer in.
The rubric is compiled into the binary via include_str!, and there is no way to supply a different
one. It is written for general-purpose capability routing.
A deployment with a narrower task distribution routes measurably better with a rubric written for it.
Today that is simply unavailable: the operator's only options are to accept the built-in rubric or
not use the route.
Proposal
Two equivalent ways to supply a rubric, mutually exclusive:
[routes.<name>]
type = "llm_classifier"
prompt_file = "prompts/coding-agent.md" # resolved relative to the config file
prompt = "Route coding-agent turns. Answer with the routing JSON only."
prompt is the library-level field on TaskClassifierConfig; prompt_file is server-level sugar,
because a rubric is usually long enough that inlining it into TOML is awkward.- File reading stays in the server layer, which is the only one that knows the config's directory.
Relative paths resolve against it, so a deployment moves as a unit.
{{RESPONSE_SCHEMA}} substitution applies to an override exactly as to the packaged rubric, so a
custom rubric may embed the schema or omit it. The schema is still sent to the model either way.- An empty or whitespace-only rubric is rejected at load: it would leave the judge with no
instructions and fail as an unparseable verdict on every turn rather than loudly.
- Unset, the packaged rubric is used. No behaviour change for existing deployments.
Scope
6 files, +125/−12. The library change is a config field and one function parameter; the rest is
server plumbing (threading the config directory through build_algorithm), the PyO3 signature, two
tests, and the schema docs.
Validation
| Gate |
Result |
cargo test --workspace |
642 passed (baseline 640 + 2) |
cargo clippy --workspace --all-targets -- -D warnings |
clean |
cargo fmt --check |
clean |
uv run pytest tests/ |
unchanged |
Problem
The
llm_classifierroute consults a small LLM to decide which tier serves a request. That callneeds instructions — a rubric telling the model what to weigh and what format to answer in.
The rubric is compiled into the binary via
include_str!, and there is no way to supply a differentone. It is written for general-purpose capability routing.
A deployment with a narrower task distribution routes measurably better with a rubric written for it.
Today that is simply unavailable: the operator's only options are to accept the built-in rubric or
not use the route.
Proposal
Two equivalent ways to supply a rubric, mutually exclusive:
promptis the library-level field onTaskClassifierConfig;prompt_fileis server-level sugar,because a rubric is usually long enough that inlining it into TOML is awkward.
Relative paths resolve against it, so a deployment moves as a unit.
{{RESPONSE_SCHEMA}}substitution applies to an override exactly as to the packaged rubric, so acustom rubric may embed the schema or omit it. The schema is still sent to the model either way.
instructions and fail as an unparseable verdict on every turn rather than loudly.
Scope
6 files, +125/−12. The library change is a config field and one function parameter; the rest is
server plumbing (threading the config directory through
build_algorithm), the PyO3 signature, twotests, and the schema docs.
Validation
cargo test --workspacecargo clippy --workspace --all-targets -- -D warningscargo fmt --checkuv run pytest tests/