Skip to content

fix(libsy): reinforce the routing task after windowed conversation content #279

Description

@gburachas

Problem

llm_classifier puts its rubric in the leading system message. That works while the judged payload
is a single short user message — the default.

Once recent_turn_window is set, the request also carries real conversation: client instructions,
the opening task, and the trailing turns. The rubric is then far from the generation point, and the
judge sometimes answers the conversation instead of classifying it.

The failure is silent, and that is what makes it expensive:

  1. the judge replies with prose rather than the routing JSON;
  2. parse_json_verdict fails, so the verdict is unavailable;
  3. TaskClassifierPolicy::to_classification(None) returns Ambiguous;
  4. the cascade falls through to DefaultTarget — the capable tier.

So the turn is still served, no error surfaces, and the only symptom is that a route configured with
recent_turn_window quietly stops routing and sends everything to the strong tier. Exactly the cost
the feature exists to avoid.

Proposal

Append a short restatement of the task after the conversation content, on the windowed path only:

Route the conversation above. Output ONLY the routing JSON object, nothing else.
  • Windowed path only. The single-message path carries no conversation to be distracted by, so it
    is left byte-identical. No behaviour change for the default configuration.
  • End-positioned, not merely present. Reinforcement works because it is last; the test asserts
    position, not just inclusion.
  • No config surface. Not a knob — the windowed path is simply broken without it.

Scope

One file, crates/libsy/src/algorithms/llm_class.rs: a const, a four-line guard in
CapabilityJudge::build_request, and two tests. +63/−0.

Why this is its own PR

It is unrelated to the judge-deadline work it was originally written alongside: different failure
mode (prompt adherence vs liveness), different code path (build_request vs verdict), no shared
surface. The deadline change adds a field to the public TaskClassifierConfig and carries a
breaking-change discussion; this one has zero API surface and should not be blocked behind it.

Validation

Gate Result
cargo test --workspace 642 passed (baseline 640 + 2 new)
cargo clippy --workspace --all-targets -- -D warnings clean
cargo fmt --check clean
uv run pytest tests/ unchanged from baseline

Open question for review

The wording is carried over from the Python implementation, where it was validated on
coding-agent traffic. It has not been re-benchmarked against the Rust classifier prompt. A reviewer
may reasonably want an offload/quality comparison with and without it before merging.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions