Skip to content

fix(detect): price output tokens in cost estimate#115

Open
secxena wants to merge 1 commit into
stagingfrom
fix/phase1-output-token-pricing
Open

fix(detect): price output tokens in cost estimate#115
secxena wants to merge 1 commit into
stagingfrom
fix/phase1-output-token-pricing

Conversation

@secxena

@secxena secxena commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Problem

Phase 1.4 (proxy cost correctness). The proxy estimated request cost from input tokens only — output tokens (typically billed 3–5x the input rate) were never priced, so every downstream cost dashboard understated spend.

Origin: crates/soth-detect/src/engine.rsestimate_input_cost_usd (call site ~L447, definition ~L1156) multiplied only estimated_input_tokens by the input rate.

Change

  • Renamed estimate_input_cost_usdestimate_cost_usd and added an estimated_output_tokens: Option<u32> parameter; the parse-phase call site now threads normalized.estimated_output_tokens through.
  • Added extract_output_rate, mirroring extract_input_rate, covering output_*/completion_* per-million / per-1k / per-token keys plus the nested "output" object. No new hardcoded rates — output rate comes from the same per-model pricing registry (ProviderEntry.pricing) already used for input (e.g. output_per_million_usd alongside input_per_million_usd).
  • Parameterized extract_model_rate with the rate extractor so input and output share the same model → default → provider-level resolution.
  • Best-effort semantics preserved: prices whichever side has a known rate, returns None only when neither can be priced, and zero tokens still yields 0.0. Behavior is unchanged when output tokens are unknown (None). The value remains an estimate.

Tests

Added cost_tests in engine.rs:

  • prices_output_tokens_in_addition_to_input — gpt-4o (5.0/M in, 15.0/M out): input-only 1000 tok = $0.005; with 2000 output tok = $0.035; asserts output strictly raises the estimate.
  • output_only_is_priced_when_input_is_zero — 3000 output tok = $0.045.
  • missing_output_rate_still_prices_input_without_crashing — input-only pricing table still returns $0.005.
  • zero_tokens_yields_zero_cost.

cargo build -p soth-detect succeeds; cargo test -p soth-detect --lib123 passed, 0 failed. (The pre-existing realworld_domain_index_coverage corpus test fails identically on clean origin/staging — it needs corpus data absent from the CI/worktree checkout and is unrelated to this change.)

🤖 Generated with Claude Code

The proxy priced only input tokens, so every downstream cost dashboard
understated spend (output tokens are typically billed 3-5x the input
rate). Extend the cost estimator to also price output tokens using the
model's output/completion rate resolved from the same pricing registry
(no new hardcoded rates).

- Rename estimate_input_cost_usd -> estimate_cost_usd and thread
  normalized.estimated_output_tokens through the parse-phase call site.
- Add extract_output_rate mirroring extract_input_rate (output/
  completion per-million / per-1k / per-token, plus nested "output").
- Parameterize extract_model_rate with the rate extractor so input and
  output share model/default/provider resolution.
- Best-effort: price whichever side has a known rate; return None only
  when neither can be priced; zero tokens still yields 0.0. The estimate
  remains an estimate.

Add unit tests proving output tokens increase the estimate, output-only
pricing when input is zero, and graceful handling of a missing output
rate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant