fix(detect): price output tokens in cost estimate#115
Open
secxena wants to merge 1 commit into
Open
Conversation
The proxy priced only input tokens, so every downstream cost dashboard understated spend (output tokens are typically billed 3-5x the input rate). Extend the cost estimator to also price output tokens using the model's output/completion rate resolved from the same pricing registry (no new hardcoded rates). - Rename estimate_input_cost_usd -> estimate_cost_usd and thread normalized.estimated_output_tokens through the parse-phase call site. - Add extract_output_rate mirroring extract_input_rate (output/ completion per-million / per-1k / per-token, plus nested "output"). - Parameterize extract_model_rate with the rate extractor so input and output share model/default/provider resolution. - Best-effort: price whichever side has a known rate; return None only when neither can be priced; zero tokens still yields 0.0. The estimate remains an estimate. Add unit tests proving output tokens increase the estimate, output-only pricing when input is zero, and graceful handling of a missing output rate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Phase 1.4 (proxy cost correctness). The proxy estimated request cost from input tokens only — output tokens (typically billed 3–5x the input rate) were never priced, so every downstream cost dashboard understated spend.
Origin:
crates/soth-detect/src/engine.rs—estimate_input_cost_usd(call site ~L447, definition ~L1156) multiplied onlyestimated_input_tokensby the input rate.Change
estimate_input_cost_usd→estimate_cost_usdand added anestimated_output_tokens: Option<u32>parameter; the parse-phase call site now threadsnormalized.estimated_output_tokensthrough.extract_output_rate, mirroringextract_input_rate, coveringoutput_*/completion_*per-million / per-1k / per-token keys plus the nested"output"object. No new hardcoded rates — output rate comes from the same per-model pricing registry (ProviderEntry.pricing) already used for input (e.g.output_per_million_usdalongsideinput_per_million_usd).extract_model_ratewith the rate extractor so input and output share the same model →default→ provider-level resolution.Noneonly when neither can be priced, and zero tokens still yields0.0. Behavior is unchanged when output tokens are unknown (None). The value remains an estimate.Tests
Added
cost_testsinengine.rs:prices_output_tokens_in_addition_to_input— gpt-4o (5.0/M in, 15.0/M out): input-only 1000 tok = $0.005; with 2000 output tok = $0.035; asserts output strictly raises the estimate.output_only_is_priced_when_input_is_zero— 3000 output tok = $0.045.missing_output_rate_still_prices_input_without_crashing— input-only pricing table still returns $0.005.zero_tokens_yields_zero_cost.cargo build -p soth-detectsucceeds;cargo test -p soth-detect --lib→ 123 passed, 0 failed. (The pre-existingrealworld_domain_index_coveragecorpus test fails identically on cleanorigin/staging— it needs corpus data absent from the CI/worktree checkout and is unrelated to this change.)🤖 Generated with Claude Code