Skip to content

Improve geothermal qualitative extraction + fix LiteLLM runtime bug - #521

Open
bpulluta wants to merge 2 commits into
mainfrom
geothermal-v222-litellm-runtime-fix
Open

Improve geothermal qualitative extraction + fix LiteLLM runtime bug#521
bpulluta wants to merge 2 commits into
mainfrom
geothermal-v222-litellm-runtime-fix

Conversation

@bpulluta

@bpulluta bpulluta commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR finalizes the geothermal qualitative extraction improvements and hardens LiteLLM/OpenAI runtime behavior for provider-specific unsupported kwargs.

Geothermal extraction updates

  • Tuned geothermal schema in v2.2.2
  • Keep scoped qualitative feature set aligned to required scope (removed extra district features)
  • Keep broadened qualitative definitions and applicability/instruction tuning for better recall
  • Added tuned collection prompt behavior in plugin_config.yaml

LiteLLM runtime hardening

  • Add adaptive retry in compass/services/openai.py for recoverable provider rejections of top-level kwargs
  • Automatically remove unsupported kwargs (e.g., temperature) and retry the same call
  • Preserve strict behavior for non-recoverable errors (no silent masking of real failures)

Tests

  • Add targeted service tests covering:
    • retry behavior when unsupported kwargs are rejected
    • non-recoverable bad request passthrough
    • fallback parsing of param from unstructured error text

Validation

  • pixi run -e pdev pytest -q tests/python/unit/services/test_services_openai.py -rapP
    • Passed (8 tests)

Restore geothermal schema/plugin to the latest tuned recall state (v2.2.2 + collection prompt tuning), and add adaptive OpenAI service retry behavior that removes provider-rejected call kwargs (e.g., temperature) and retries safely.

Also adds runtime-focused tests for unsupported-kwarg recovery and non-recoverable error passthrough.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 6, 2026 12:09
@codecov-commenter

codecov-commenter commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.07692% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.98%. Comparing base (77e7a53) to head (b61ab60).

Files with missing lines Patch % Lines
compass/services/openai.py 73.07% 7 Missing and 7 partials ⚠️

❌ Your patch status has failed because the patch coverage (73.07%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #521      +/-   ##
==========================================
+ Coverage   63.82%   63.98%   +0.15%     
==========================================
  Files          78       78              
  Lines        7437     7481      +44     
  Branches      755      766      +11     
==========================================
+ Hits         4747     4787      +40     
+ Misses       2529     2526       -3     
- Partials      161      168       +7     
Flag Coverage Δ
unittests 63.98% <73.07%> (+0.15%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves COMPASS’ geothermal electricity one-shot extraction configuration (schema + text-collection prompting) and hardens the OpenAI/LiteLLM call path to dynamically retry after provider-specific rejections of unsupported top-level kwargs.

Changes:

  • Added adaptive handling in OpenAIService to detect provider “unsupported parameter” style BadRequestErrors, drop the offending kwarg, and retry.
  • Updated geothermal extraction configuration: replaced boolean collection_prompts with an explicit prompt chain and tuned the geothermal qualitative schema (including removing out-of-scope district features).
  • Added unit tests covering the unsupported-kwarg retry behavior and kwarg detection logic.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
compass/services/openai.py Adds unsupported-kwarg detection and retry logic for resilient OpenAI/LiteLLM runtime behavior.
tests/python/unit/services/test_services_openai.py Adds targeted tests for retrying without unsupported kwargs and parsing unsupported params from error text.
compass/extraction/geothermal_electricity/plugin_config.yaml Switches to explicit collection prompt(s) for geothermal text retention decisions.
compass/extraction/geothermal_electricity/geothermal_schema.json Updates geothermal qualitative schema definitions and removes extra district features to match intended scope.

Comment thread compass/services/openai.py Outdated
Comment thread compass/services/openai.py
Address PR review feedback: correct the module docstring typo and
bound the number of unsupported top-level kwarg drops per request so
provider rejections cannot trigger unbounded paid retries.
@bpulluta bpulluta changed the title Improve geothermal qualitative extraction + LiteLLM runtime resilience Improve geothermal qualitative extraction + LiteLLM runtime bug Aug 6, 2026
@bpulluta bpulluta changed the title Improve geothermal qualitative extraction + LiteLLM runtime bug Improve geothermal qualitative extraction + fix LiteLLM runtime bug Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants