Feat: add LLM abstraction, inspect subsystem, and refinement support#12
Merged
fangchenli merged 1 commit intomainfrom Jan 26, 2026
Merged
Feat: add LLM abstraction, inspect subsystem, and refinement support#12fangchenli merged 1 commit intomainfrom
fangchenli merged 1 commit intomainfrom
Conversation
This PR adds three major features to Delibera: ## LLM Abstraction + Gemini Provider (PR #17) - Add `llm/` module with stable LLMClient protocol interface - Implement GeminiClient with SDK and HTTP fallback - Add LLM request/response types with structured tracing - Add redaction helpers for trace-safe logging - Implement ProposerLLM agent with JSON output - Add CLI flags: --use-llm-proposer, --llm-model, etc. - Add optional dependency: delibera[llm] LLM usage is restricted to WORK steps only (never CLAIM_CHECK). Stubs remain the default; LLM mode is opt-in. ## Inspect Subsystem (PR #15) - Add `inspect/` module for run inspection and reporting - Implement text and markdown renderers - Add summarize module for run analysis - Add CLI commands: inspect, report - Support inspection by run-id or path ## Refinement Support (PR #14) - Add refinement round support in protocol spec - Implement convergence criteria checking - Add refinement pipeline execution - Support multi-round deliberation ## Additional Improvements - Add CONTRIBUTING.md with contributor guidelines - Add docs/README.md documentation index - Add DocsSearch and DocsRead tools with capability classification - Add smoke tests for CLI and imports - Improve tool policy with capability-based access control - Add __version__.py for version tracking Tests: 373 passing Type check: mypy passes with no errors Lint: ruff passes with no errors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds three major features to Delibera:
- LLM Abstraction + Gemini Provider: Adds a stable LLM client interface with Gemini implementation, including request/response types, redaction helpers, and an LLM-backed proposer agent
- Inspect Subsystem: Adds run inspection and reporting capabilities with text and markdown renderers
- Refinement Support: Adds bounded refinement loops with convergence checking
Changes:
- New
llm/module with LLM client protocol and Gemini implementation - New
inspect/module for run analysis and reporting - New
DocsSearchToolfor evidence discovery - Protocol spec extensions for refinement rounds
- CLI commands:
delibera inspectanddelibera report
Reviewed changes
Copilot reviewed 40 out of 41 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Added dependencies for LLM support (google-generativeai, pydantic, etc.) |
| tests/llm/test_llm_proposer.py | Tests for LLM proposer with fake client |
| tests/inspect/test_inspect.py | Tests for inspect/report functionality |
| tests/test_refinement.py | Tests for refinement loop |
| tests/test_smoke.py | Smoke tests for CLI and imports |
| tests/tools_docs/test_docs_search.py | Tests for DocsSearchTool |
| src/delibera/llm/ | LLM abstraction layer with base types, Gemini client, prompts, redaction |
| src/delibera/inspect/ | Run inspection with summarize, text renderer, markdown renderer |
| src/delibera/agents/llm_proposer.py | LLM-backed proposer agent |
| src/delibera/tools/builtin/docs.py | DocsSearchTool implementation |
| src/delibera/protocol/spec.py | ConvergenceSpec with refinement parameters |
| src/delibera/engine/orchestrator.py | Refinement loop execution |
| src/delibera/cli.py | New CLI commands and LLM flags |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Returns: | ||
| Dict with refined artifact and refinement notes. | ||
| """ | ||
| node_id = context.get("node_id", "") # noqa: F841 |
There was a problem hiding this comment.
Variable node_id is not used.
fangchenli
added a commit
that referenced
this pull request
Jan 28, 2026
- Pass Gemini API key via x-goog-api-key header instead of URL query
parameter to prevent key leakage in logs and tracebacks
- Use Gemini SDK's native system_instruction parameter instead of
concatenating system messages into the user prompt
- Deduplicate _build_sdk_contents and _build_http_contents into a
single _build_contents method
- Redact error messages in llm_call_failed trace events using
redact_text() to prevent sensitive data in traces
- Fix _extract_protocol_info reading wrong key ("protocol" vs
"protocol_name") causing empty protocol name in inspect output
- Fix CLI catch-all error message referencing ANTHROPIC_API_KEY
instead of being provider-agnostic
- Wire check_llm_allowed_in_step runtime guard into the engine's
PROPOSE step to enforce the LLM-only-in-WORK invariant
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
4 tasks
fangchenli
added a commit
that referenced
this pull request
Jan 28, 2026
- Pass Gemini API key via x-goog-api-key header instead of URL query
parameter to prevent key leakage in logs and tracebacks
- Use Gemini SDK's native system_instruction parameter instead of
concatenating system messages into the user prompt
- Deduplicate _build_sdk_contents and _build_http_contents into a
single _build_contents method
- Redact error messages in llm_call_failed trace events using
redact_text() to prevent sensitive data in traces
- Fix _extract_protocol_info reading wrong key ("protocol" vs
"protocol_name") causing empty protocol name in inspect output
- Fix CLI catch-all error message referencing ANTHROPIC_API_KEY
instead of being provider-agnostic
- Wire check_llm_allowed_in_step runtime guard into the engine's
PROPOSE step to enforce the LLM-only-in-WORK invariant
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds three major features to Delibera:
LLM Abstraction + Gemini Provider
llm/module with stableLLMClientprotocol interfaceGeminiClientwith SDK and HTTP fallbackProposerLLMagent with JSON output--use-llm-proposer,--llm-model,--llm-temperature,--llm-max-output-tokenspip install delibera[llm]LLM usage is restricted to WORK steps only (never CLAIM_CHECK). Stubs remain the default; LLM mode is opt-in.
Inspect Subsystem
inspect/module for run inspection and reportingdelibera inspect,delibera reportRefinement Support
Additional Improvements
__version__.pyfor version trackingTest plan
delibera run --question "Test" --use-llm-proposer(requires GEMINI_API_KEY)delibera inspect --run-id <id>delibera report --run-id <id>🤖 Generated with Claude Code