compiler: inject missing model pricing from models.dev at compile time

[Copilot]

Models absent from the embedded models.json catalog produce no pricing data in compiled lock.yml, causing the AWF proxy to mark them unknown_model_ai_credits=true at runtime. This PR makes the compiler fetch missing pricing from https://models.dev/catalog.json (once per process) and embed it in GH_AW_INFO_MODEL_COSTS so cost accounting works without a live catalog download in the agent job.

Approach

Dependency injection avoids the pkg/cli → pkg/workflow circular import: the pricing resolver is a func(ctx, provider, model) field on Compiler, set by the cli at compile time. WASM builds are safe since the field is never populated there.

Key changes

• pkg/cli/models_dev_sync.go — sync.Once download of models.dev/catalog.json; normalizes per-million → per-token float64; modelsDevCatalogURL and modelsDevHTTPClientFactory are var for testability
• pkg/cli/model_costs.go — FindOrFetchModelPricing: short-circuits with (nil, false) for models already in the embedded catalog (runtime already has them), falls through to models.dev for unknowns
• pkg/workflow/compiler_types.go — modelPricingResolver field + SetModelPricingResolver setter
• pkg/workflow/compiler_model_pricing.go — resolveModelPricingIfMissing (skips if frontmatter overlay already covers the model), resolveEngineProviderForPricing (LLMProvider → InlineProviderID → engine-ID heuristic), mergeModelPricingIntoModelCosts (shallow-clone; prices serialized as %g strings matching the models.json schema)
• pkg/workflow/workflow_builder.go — calls resolveModelPricingIfMissing after the frontmatter ModelCosts overlay is assembled
• pkg/cli/compile_compiler_setup.go — registers FindOrFetchModelPricing as the resolver in createAndConfigureCompiler

Behavior

Model state	Resolver action
In embedded catalog	(nil, false) — runtime models.json already has it
Unknown, found in models.dev	Pricing merged into GH_AW_INFO_MODEL_COSTS
Unknown, not in models.dev / network failure	Logged, compiler proceeds without pricing overlay
Frontmatter already provides pricing	Resolver skipped entirely

[pelikhan]

@copilot

• add cli argument to disable this lookup
• ignores if unreachable

[Copilot]

Pull request overview

Adds a compile-time fallback that injects missing model pricing into GH_AW_INFO_MODEL_COSTS by downloading and querying the models.dev catalog, so runtime cost accounting doesn’t rely on a live catalog fetch when the embedded models.json lacks a model.

Changes:

• Adds a Compiler-injected modelPricingResolver callback and wires it up from the CLI compile path.
• Implements compile-time resolution/merge logic to extend WorkflowData.ModelCosts only when pricing is missing.
• Adds a sync.Once-cached models.dev catalog downloader/parser plus unit tests for both CLI and workflow layers.

Show a summary per file

File	Description
pkg/workflow/workflow_builder.go	Calls the compiler’s pricing resolver after assembling the frontmatter ModelCosts overlay.
pkg/workflow/compiler_types.go	Adds modelPricingResolver and a setter to inject pricing resolution without circular imports.
pkg/workflow/compiler_model_pricing.go	New pricing resolution + merge helpers for injecting missing per-token costs into ModelCosts.
pkg/workflow/compiler_model_pricing_test.go	Unit tests covering provider resolution, merge behavior, and resolver integration.
pkg/cli/models_dev_sync.go	sync.Once download/parse of models.dev/catalog.json into a normalized per-token pricing cache.
pkg/cli/models_dev_sync_test.go	Unit tests for parsing and lookup logic against a local test server.
pkg/cli/model_costs.go	Adds FindOrFetchModelPricing to short-circuit embedded-catalog models and otherwise query models.dev.
pkg/cli/compile_compiler_setup.go	Registers FindOrFetchModelPricing as the compiler’s resolver during CLI compilation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 8/8 changed files
• Comments generated: 2
• Review effort level: Low

[Copilot]

@copilot

• add cli argument to disable this lookup
• ignores if unreachable

Implemented in commit 7cf566f.

• Added --no-models-dev-lookup to gh aw compile to disable models.dev pricing lookup.
• Wiring now skips registering the pricing resolver when that flag is set.
• Existing behavior for unreachable models.dev remains non-fatal (compile continues without injected pricing).

[gh-aw-bot]

@copilot please run the pr-finisher skill, check whether this branch needs a refresh from main, and rerun the relevant checks. The latest feedback requests are already implemented; if no further code changes are needed, post a short maintainer-facing summary of what changed and anything still blocking merge.

Run: https://github.com/github/gh-aw/actions/runs/28606537165

Generated by 👨‍🍳 PR Sous Chef · 25.6 AIC · ⌖ 15.6 AIC · ⊞ 3.1K · ◷

[Copilot]

@copilot please run the pr-finisher skill, check whether this branch needs a refresh from main, and rerun the relevant checks. The latest feedback requests are already implemented; if no further cod...

Ran a PR-finisher pass with no additional code changes required.

Maintainer summary:

• Branch freshness: compared with origin/main; this branch is ahead 3 / behind 2. No merge conflicts detected in a dry merge-tree check.
• Local checks rerun on current HEAD (100c9b7): make fmt, make lint, make test-unit, make test, and make recompile all passed.
• Prior CI inspected: latest PR workflow runs on this head are action_required with no jobs started yet (approval/retrigger needed). Earlier failure logs were reviewed, but current local validation is green.

Still blocking merge:

• CI on the current head is unverified/stale from the agent side and requires a maintainer-triggered run/approval.

[gh-aw-bot]

@copilot please run the pr-finisher skill, refresh this branch if needed, and address the remaining review feedback before rerunning checks. Please work through any outstanding review comments, resolve or reply on anything intentionally deferred, and leave the PR in a maintainer-ready state.

Run: https://github.com/github/gh-aw/actions/runs/28611171544

Generated by 👨‍🍳 PR Sous Chef · 19.4 AIC · ⌖ 16.2 AIC · ⊞ 6.4K · ◷

[Copilot]

@copilot please run the pr-finisher skill, refresh this branch if needed, and address the remaining review feedback before rerunning checks. Please work through any outstanding review comments, reso...

Completed. Addressed remaining review feedback and refreshed the branch from main.

Code updates landed in:

• a2812a3 normalize provider/model handling for compile-time pricing injection
• c320bd0 stabilize models.dev cache test cleanup
• 29811ce reject malformed qualified model IDs
• c40ea54 simplify split validation
• 1f7c131 merge origin/main into this branch

Local checks are green on current HEAD after refresh:

• make fmt
• make lint
• make test-unit
• make test
• make recompile
• make agent-report-progress

PR is maintainer-ready from the agent side; CI on this updated head still needs maintainer-triggered rerun/approval.

[pelikhan]

@copilot refactor models.dev support into separate package modelsdev to clearly separate the core functionality of this feature into a new package.

[Copilot]

@copilot refactor models.dev support into separate package modelsdev to clearly separate the core functionality of this feature into a new package.

Done in 0af4537.

I refactored the models.dev catalog fetch/parse/cache/lookup logic into a new pkg/modelsdev package and updated pkg/cli to delegate FindOrFetchModelPricing to it. Tests were moved accordingly (pkg/modelsdev/catalog_test.go), with CLI-specific behavior test kept in pkg/cli/model_costs_test.go.

[pelikhan]

@copilot use syncutil helper to cache catalog

[Copilot]

@copilot use syncutil helper to cache catalog

Implemented in 6aae2c1.

Switched pkg/modelsdev catalog caching to syncutil.OnceLoader[pricingCache] (replacing sync.Once + package cache globals), and updated the modelsdev tests to reset the shared loader via catalogCache.Reset() for isolation.

[pelikhan]

@copilot run pr-finisher skill

Fix https://github.com/github/gh-aw/actions/runs/28620919434/job/84877643669

[github-actions]

Hey @copilot-swe-agent 👋 — great work on the model pricing injection feature! The dependency-injection approach to avoid the pkg/cli → pkg/workflow circular import is clean, and the behavior table in the PR description makes the four model-state cases easy to follow at a glance.

One thing would help get this across the finish line:

• Split the PR — this mixes two distinct changes: (1) the compiler feature that injects missing model pricing from models.dev (13 Go source files), and (2) a bulk firewall version bump from v0.27.20 to v0.27.22 across 258 .lock.yml files. Reviewers would benefit from being able to evaluate the compiler logic change independently from the lock-file regeneration. Consider opening a dedicated PR just for the firewall bump, and keeping this PR focused on the pkg/modelsdev, pkg/cli, and pkg/workflow changes.

If you would like a hand splitting it up, you can assign this prompt to your coding agent:

Split PR #43003 in the github/gh-aw repository into two focused PRs:

1. **Compiler feature PR** — include only the Go source changes:
   - pkg/modelsdev/catalog.go + catalog_test.go
   - pkg/cli/model_costs.go + model_costs_test.go
   - pkg/cli/compile_compiler_setup.go + compile_compiler_setup_test.go
   - pkg/cli/compile_config.go
   - pkg/workflow/compiler_model_pricing.go + compiler_model_pricing_test.go
   - pkg/workflow/compiler_types.go
   - pkg/workflow/workflow_builder.go
   - cmd/gh-aw/compile_flags_test.go + main.go
   Keep the original PR description.

2. **Lock file bump PR** — include only the 258 .github/workflows/*.lock.yml files
   that bump gh-aw-firewall from v0.27.20 to v0.27.22.
   Title: "chore: bump gh-aw-firewall to v0.27.22 across all lock files"

Close PR #43003 once both replacement PRs are open.

Generated by ✅ Contribution Check · 184.6 AIC · ⌖ 12.9 AIC · ⊞ 6.3K · ◷

[Copilot]

@copilot run pr-finisher skill

Fix https://github.com/github/gh-aw/actions/runs/28620919434/job/84877643669

Fixed in 88ec024.

I reproduced the failing lint-go job (run 28620919434, job 84877643669) locally and fixed the custom linter failures by replacing case-insensitive strings.ToLower(...) == ... comparisons with strings.EqualFold(...) in:

• pkg/modelsdev/catalog.go
• pkg/workflow/compiler_model_pricing.go

Validation rerun locally passed:

• make fmt
• make golint-custom ... (same flags as CI)
• go test ./pkg/modelsdev ./pkg/workflow
• make lint
• make test-unit
• make test
• make recompile
• make agent-report-progress