Skip to content

feat: add FinOps model choice opportunity scoring#147

Open
ccf wants to merge 1 commit intomainfrom
feat/model-choice-opportunity-scoring
Open

feat: add FinOps model choice opportunity scoring#147
ccf wants to merge 1 commit intomainfrom
feat/model-choice-opportunity-scoring

Conversation

@ccf
Copy link
Owner

@ccf ccf commented Mar 26, 2026

Summary

  • score workflow-level model downshift opportunities inside FinOps cost modeling
  • surface potential savings and confidence in the FinOps modeling tab
  • true up the roadmap status for model-choice opportunity scoring

Testing

  • PYTHONPATH=src pytest tests/test_finops.py -q
  • ruff check src/primer/common/schemas.py src/primer/server/services/finops_service.py tests/test_finops.py
  • cd frontend && ./node_modules/.bin/vitest run --run src/components/finops/tests/workflow-cost-table.test.tsx src/components/finops/tests/workflow-compare-card.test.tsx src/components/finops/tests/model-choice-opportunity-table.test.tsx
  • cd frontend && ./node_modules/.bin/eslint src/components/finops/model-choice-opportunity-table.tsx src/components/finops/modeling-tab.tsx src/components/finops/tests/model-choice-opportunity-table.test.tsx src/types/api.ts
  • cd frontend && ./node_modules/.bin/tsc --noEmit

Note

Medium Risk
Adds new workflow/model scoring logic to the cost-modeling backend response and surfaces it in the FinOps UI; incorrect heuristics or missing facet/workflow data could yield misleading savings recommendations.

Overview
Extends FinOps cost modeling to compute workflow-level model downshift opportunities by comparing per-archetype success rates and average session costs across models, returning a ranked list plus total_model_choice_savings_monthly in CostModelingResponse.

Adds a new ModelChoiceOpportunityTable card to the Modeling tab to display opportunities, confidence, rationale, and projected monthly savings (with an empty-state), and updates API types/schemas and tests (backend + frontend). Also marks the roadmap item as shipped.

Written by Cursor Bugbot for commit 7de10e1. This will update automatically on new commits. Configure here.

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is ON, but it could not run because the spend limit has been reached. To enable Bugbot Autofix, raise your spend limit in the Cursor dashboard.

opportunities.sort(
key=lambda opportunity: (
-opportunity.monthly_savings_estimate,
{"high": 2, "medium": 1, "low": 0}.get(opportunity.confidence, 0),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confidence sort order is inverted, preferring low confidence

Medium Severity

The confidence tiebreaker in opportunities.sort() ranks low-confidence opportunities before high-confidence ones. The mapping {"high": 2, "medium": 1, "low": 0} is sorted ascending by default, so low (0) comes first. The value needs to be negated (like the savings field) so that high-confidence opportunities are preferred when monthly savings are equal. This can cause the [:limit] truncation to drop high-confidence opportunities in favor of low-confidence ones.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant