feat(cache-proposals): cost-weighted threshold optimization#227
Open
jamby77 wants to merge 10 commits into
Open
feat(cache-proposals): cost-weighted threshold optimization#227jamby77 wants to merge 10 commits into
jamby77 wants to merge 10 commits into
Conversation
05305d7 to
bbcd2c9
Compare
- Inject LicenseService into CacheReadonlyService constructor - Compute cost-weighted uncertain-hit and near-miss rates from per-sample cost_saved_micros - Use cost-weighted rates as trigger for paths 1 (uncertain_hits) and 3 (near_misses) on Enterprise tier - Fall back to count-based rates for Pro+/Community or when costed samples < minSamples - Populate cost_weighted_uncertain_hit_rate, cost_weighted_near_miss_rate, total_hit_cost_usd, uncertain_hit_cost_usd fields when cost-weighting was active - Add StubLicenseService, seedSamplesWithCost helper, and 3 new tests (23 total)
… correct fractionOfAll
d8438ad to
572b257
Compare
- Prune entries older than the 5-minute bound on every recordMissPending, so miss-only traffic cannot grow the bookkeeping zset unbounded - Delete the miss-pending key in flush() alongside stats and window keys - Add coverage for the record-time prune
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 394da28. Configure here.
KIvanow
approved these changes
Jun 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Stacked on #224 (confidence-score).
Summary
cost_saved_microson similarity-window entries in the semantic cache, with retroactive population from laterstore()calls.cost_saved_microsfrom similarity samples into the cache-proposals threshold recommendation engine.Closes the "Cost-Weighted Threshold Optimization" card on the BetterDB project board.
Test plan
store()→ similarity sample → recommendationNote
Medium Risk
Changes threshold tuning behavior for enterprise customers and adds best-effort Valkey bookkeeping on the hot check/store path; failures are swallowed but could skew cost metrics early in rollout.
Overview
Semantic cache now attaches
cost_saved_microsto rolling similarity-window events (hits from entrycost_micros, misses asnull), tracks pending misses in a__miss_pendingzset keyed by prompt hash, and backfills miss rows when a laterstore()supplies token cost—so threshold analytics can see dollar impact on near-misses, not only on hits.Cache proposals reads that field from the window, and on enterprise license only switches uncertain-hit / near-miss tighten and loosen decisions to cost-weighted rates (with optional USD fields and cost-aware reasoning); Pro and below keep count-based logic.
New unit tests cover instrumentation, pending-miss lifecycle, and tier-gated recommendations.
Reviewed by Cursor Bugbot for commit 394da28. Bugbot is set up for automated code reviews on this repo. Configure here.