Replace openrouter integration using langchain package

[srtab]

DAIV currently is integrating OpenRouter using the OpenAI SDK. This has some limitations and makes it harder to integrate.

Langchain recently launched an official package to integrate with OpenRouter: langchain-openrouter (https://docs.langchain.com/oss/python/integrations/chat/openrouter). This will help to maintain a more seamless integration.

[daiv-agent]

Now I have a thorough understanding of the codebase. Let me produce the implementation plan:

---

Replace OpenRouter Integration with langchain-openrouter

Overview

Currently, DAIV integrates OpenRouter by treating it as an OpenAI-compatible provider via langchain-openai — manually stripping the openrouter: prefix, overriding openai_api_base, injecting custom headers, and handling thinking tokens through extra_body. The official langchain-openrouter package (ChatOpenRouter) provides a first-class integration that eliminates this workaround.

Complexity: Medium — touches model init, prompt-cache middleware, type hints, settings, and tests, but no data migrations or architectural changes.

---

Implementation Steps

1. Install langchain-openrouter

uv add langchain-openrouter==0.1.0

Add to pyproject.toml with an exact pinned version per project conventions.

---

2. Update daiv/automation/agent/base.py

This is the core change. The OPENROUTER branch in get_model_kwargs currently masquerades as ModelProvider.OPENAI. Replace it with a direct ChatOpenRouter instantiation.

Remove the existing ModelProvider.OPENROUTER block from get_model_kwargs (lines 149–176). Instead, instantiate ChatOpenRouter directly — bypassing init_chat_model for OpenRouter models.

The get_model staticmethod currently always calls init_chat_model(**model_kwargs). To accommodate ChatOpenRouter, it should branch: for OPENROUTER, build and return a ChatOpenRouter instance directly; for all other providers, keep init_chat_model.

New OpenRouter kwargs logic:

from langchain_openrouter import ChatOpenRouter

elif model_provider == ModelProvider.OPENROUTER:
    assert settings.OPENROUTER_API_KEY is not None, "OpenRouter API key is not set"
    model_name_clean = _kwargs["model"].split(":", 1)[1]  # strip "openrouter:" prefix
    chat_kwargs = {
        "model": model_name_clean,
        "temperature": _kwargs.get("temperature", 0),
        "app_url": "https://srtab.github.io/daiv",
        "app_title": BOT_NAME,
    }
    if "max_tokens" in _kwargs:
        chat_kwargs["max_tokens"] = _kwargs["max_tokens"]

    if thinking_level:
        if model_name_clean.startswith(CLAUDE_THINKING_MODELS):
            max_tokens, thinking_tokens = BaseAgent._get_anthropic_thinking_tokens(...)
            chat_kwargs["max_tokens"] = max_tokens
            chat_kwargs["reasoning"] = {"max_tokens": thinking_tokens}
            chat_kwargs["temperature"] = 1
        else:
            chat_kwargs["reasoning"] = {"effort": thinking_level.value}
    elif model_name_clean.startswith("anthropic/") and "max_tokens" not in chat_kwargs:
        chat_kwargs["max_tokens"] = CLAUDE_MAX_TOKENS

    return ChatOpenRouter(**chat_kwargs)

Note: The app_url/app_title constructor params replace the manual extra_headers (HTTP-Referer / X-Title). The reasoning param replaces extra_body.reasoning. The OPENROUTER_API_KEY env var is consumed automatically by ChatOpenRouter if set, or pass it explicitly via openrouter_api_key=.

Refactor get_model to handle the split:

@staticmethod
def get_model(*, model: str, thinking_level: ThinkingLevel | None = None, **kwargs) -> BaseChatModel:
    provider = BaseAgent.get_model_provider(model)
    if provider == ModelProvider.OPENROUTER:
        return BaseAgent._get_openrouter_model(model=model, thinking_level=thinking_level, **kwargs)
    model_kwargs = BaseAgent.get_model_kwargs(
        model=model, model_provider=provider, thinking_level=thinking_level, **kwargs
    )
    return init_chat_model(**model_kwargs)

Update get_max_token_value — the OPENROUTER case currently falls through to _ (raises ValueError). Add an explicit case:

case ModelProvider.OPENROUTER:
    # Use the same heuristic as anthropic for Claude models, or a sensible default
    return 8192

Update TYPE_CHECKING imports — remove ChatOpenAI import if it's no longer needed elsewhere, add ChatOpenRouter if needed for type checks.

---

3. Update daiv/automation/agent/middlewares/prompt_cache.py

The _is_openrouter_anthropic_model check currently identifies OpenRouter models by checking isinstance(model, ChatOpenAI). With langchain-openrouter, OpenRouter models will be ChatOpenRouter instances instead.

from langchain_openrouter import ChatOpenRouter

def _is_openrouter_anthropic_model(self, model: BaseChatModel) -> bool:
    return isinstance(model, ChatOpenRouter) and model.model_name.startswith("anthropic/")

Check ChatOpenRouter attribute name for the model — likely model or model_name. Verify from the package source or docs.

---

4. Update daiv/automation/conf.py

• Remove OPENROUTER_API_BASE field — ChatOpenRouter hardcodes the endpoint and doesn't accept a custom base URL in this version. If the field is kept for future flexibility, it can remain but should be documented as unused.
• Keep OPENROUTER_API_KEY — still needed (pass explicitly to ChatOpenRouter or set as env var).

---

5. Update pyproject.toml

• Remove the openai extra from langchain if OpenRouter was the only consumer: langchain[anthropic,community,google-genai,openai] → check if openai extra is still needed for direct OpenAI provider models. It likely is still needed (for ModelProvider.OPENAI), so keep it.
• Add: langchain-openrouter==0.1.0

---

Files to Modify

File	Change
pyproject.toml	Add langchain-openrouter==0.1.0 dependency
daiv/automation/agent/base.py	Replace OpenRouter branch in get_model_kwargs; refactor get_model; add get_max_token_value OPENROUTER case; update imports
daiv/automation/agent/middlewares/prompt_cache.py	Update _is_openrouter_anthropic_model to use ChatOpenRouter isinstance check
daiv/automation/conf.py	Consider removing OPENROUTER_API_BASE (was only used as openai_api_base override)
tests/unit_tests/automation/agent/test_base.py	Update/add tests for the new OPENROUTER model init path
tests/unit_tests/automation/agent/middlewares/test_prompt_cache.py	Update _is_openrouter_anthropic_model mock/assertion to reflect ChatOpenRouter

---

Testing

• test_base.py: Add test_get_model_openrouter — verify get_model("openrouter:anthropic/claude-sonnet-4.5") returns a ChatOpenRouter instance with correct model, temperature, and app_url/app_title set.
• test_prompt_cache.py: Update the _is_openrouter_anthropic_model test to mock ChatOpenRouter (not ChatOpenAI).
• Run uv run pytest tests/unit_tests/automation/ after changes.

---

Edge Cases

1. OPENROUTER_API_BASE removal: If any user has overridden the base URL (e.g., for a self-hosted proxy), removing this field is a breaking change. Consider keeping it as a deprecated setting or noting the removal in the changelog.
2. get_max_token_value for OpenRouter: The return value depends on the underlying model. A safe default (8192) or model-name-based routing (check anthropic/ → 8192, openai/ → use tiktoken) should be chosen.
3. thinking_level + non-Claude OpenRouter models: The current else branch sets extra_body.reasoning.effort. Verify ChatOpenRouter supports a reasoning param with effort for OpenAI-based thinking models.
4. anthropic-beta header for structured outputs: Currently passed via extra_headers. Check whether ChatOpenRouter forwards additional headers or has a native mechanism — may need to verify with the package source.
5. OPENROUTER_API_KEY injection: ChatOpenRouter reads OPENROUTER_API_KEY from env by default. The project currently reads from Docker secrets via Pydantic settings. Ensure the key is explicitly passed via openrouter_api_key=settings.OPENROUTER_API_KEY.get_secret_value() to maintain compatibility.

[srtab]

@daiv-agent adapt the plan to use init_chat_model as now it support openrouter too. Add also support to XHIGH thinking and change the max thinking default to XHIGH.

[daiv-agent]

Now I have everything needed. Here is the updated plan:

---

Replace OpenRouter Integration with langchain-openrouter (via init_chat_model)

Overview

DAIV currently integrates OpenRouter by masquerading it as an OpenAI provider — overriding openai_api_base, stripping the openrouter: prefix, and injecting custom headers manually. The langchain-openrouter package (langchain-openrouter==0.1.0) ships ChatOpenRouter, and init_chat_model now supports model_provider="openrouter" natively — forwarding all kwargs directly to ChatOpenRouter. This plan migrates to that first-class path and adds the missing XHIGH thinking level (OpenRouter supports "xhigh" as the highest effort value), promoting it to the new default for MAX_THINKING_LEVEL.

Complexity: Medium

---

Implementation Steps

1. Install langchain-openrouter

uv add langchain-openrouter==0.1.0

This adds langchain-openrouter==0.1.0 to pyproject.toml. The openai extra in the existing langchain[...,openai] dependency is still needed for ModelProvider.OPENAI — keep it.

---

2. Add XHIGH to ThinkingLevel — daiv/automation/agent/base.py

class ThinkingLevel(StrEnum):
    MINIMAL = "minimal"
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    XHIGH = "xhigh"   # ← add

---

3. Update _get_anthropic_thinking_tokens — daiv/automation/agent/base.py

This helper maps ThinkingLevel → (max_tokens, thinking_tokens) for the direct Anthropic provider path. Add the XHIGH case (consistent with the existing scaling pattern — ~95% allocation, capped at the Anthropic hard limit of 128k):

@staticmethod
def _get_anthropic_thinking_tokens(*, thinking_level: ThinkingLevel, max_tokens: int) -> tuple[int, int]:
    if thinking_level == ThinkingLevel.LOW:
        return max_tokens + 4_096, 4_096
    elif thinking_level == ThinkingLevel.MEDIUM:
        return max_tokens + 25_600, 25_600
    elif thinking_level == ThinkingLevel.HIGH:
        return 64_000, 64_000 - max_tokens
    elif thinking_level == ThinkingLevel.XHIGH:
        return 64_000 + max_tokens, 64_000   # max budget within Anthropic's direct-API limits

Note: For OpenRouter's Anthropic models the token budget is handled via reasoning={"effort": "xhigh"} and OpenRouter maps that to ~95% of max_tokens internally — no manual token arithmetic needed.

---

4. Replace the OPENROUTER branch in get_model_kwargs — daiv/automation/agent/base.py

Before: The branch sets model_provider=ModelProvider.OPENAI, injects openai_api_base, openai_api_key, and extra_headers, and sets extra_body for reasoning.

After: Use init_chat_model natively with model_provider="openrouter". The new block passes OpenRouter-native kwargs:

elif model_provider == ModelProvider.OPENROUTER:
    assert settings.OPENROUTER_API_KEY is not None, "OpenRouter API key is not set"
    _kwargs["model"] = _kwargs["model"].split(":", 1)[1]  # strip "openrouter:" prefix
    # Keep model_provider as OPENROUTER — init_chat_model routes to ChatOpenRouter
    _kwargs["openrouter_api_key"] = settings.OPENROUTER_API_KEY.get_secret_value()
    _kwargs["app_url"] = "https://srtab.github.io/daiv"
    _kwargs["app_title"] = BOT_NAME

    if thinking_level:
        _kwargs["reasoning"] = {"effort": thinking_level.value}
        if _kwargs["model"].startswith(CLAUDE_THINKING_MODELS):
            _kwargs["temperature"] = 1
        elif _kwargs["model"].startswith(OPENAI_THINKING_MODELS):
            _kwargs["temperature"] = 1
    elif _kwargs["model"].startswith("anthropic/") and "max_tokens" not in _kwargs:
        _kwargs["max_tokens"] = CLAUDE_MAX_TOKENS

Key differences from the old approach:

• No more openai_api_base / openai_api_key — replaced by openrouter_api_key
• No more model_kwargs["extra_headers"] — app_url/app_title are first-class ChatOpenRouter params
• No more extra_body — reasoning={"effort": ...} is a native ChatOpenRouter param
• model_provider stays "openrouter" — init_chat_model resolves it to ChatOpenRouter
• Unified reasoning — all models (Claude and OpenAI via OpenRouter) use the same reasoning={"effort": ...} API; no need to branch on model prefix for thinking token arithmetic

anthropic-beta structured-outputs header: Currently injected via extra_headers for Anthropic-via-OpenRouter. Check if ChatOpenRouter supports passing additional headers (via default_headers or similar); if not, this beta feature silently drops. Confirm during implementation.

---

5. Remove OPENROUTER_API_BASE from conf — daiv/automation/conf.py

The OPENROUTER_API_BASE setting was only used to set openai_api_base in the old approach. ChatOpenRouter hardcodes the OpenRouter endpoint and does not accept a custom base URL in v0.1.0. Remove the field:

# Remove:
OPENROUTER_API_BASE: str | None = Field(
    default="https://openrouter.ai/api/v1", description="OpenRouter API base url", alias="OPENROUTER_API_BASE"
)

Breaking change note: If any deployer overrode this setting (e.g. for a proxy), document the removal in the changelog.

---

6. Update prompt_cache.py middleware — daiv/automation/agent/middlewares/prompt_cache.py

The _is_openrouter_anthropic_model check uses isinstance(model, ChatOpenAI). With the new integration, OpenRouter models are ChatOpenRouter instances:

from langchain_openrouter import ChatOpenRouter   # replace ChatOpenAI import

def _is_openrouter_anthropic_model(self, model: BaseChatModel) -> bool:
    return isinstance(model, ChatOpenRouter) and model.model_name.startswith("anthropic/")

Verify the attribute name on ChatOpenRouter — likely model or model_name. Confirm from package source during implementation.

---

7. Update MAX_THINKING_LEVEL default — daiv/automation/agent/conf.py

MAX_THINKING_LEVEL: ThinkingLevel | None = Field(
    default=ThinkingLevel.XHIGH,   # was ThinkingLevel.HIGH
    ...
)

---

8. Update get_max_token_value — daiv/automation/agent/base.py

The match block falls through for OPENROUTER (raises ValueError). Add an explicit case:

case ModelProvider.OPENROUTER:
    return 8192  # safe default; Claude models use this, OpenAI models vary

---

Files to Modify

File	Change
pyproject.toml	Add langchain-openrouter==0.1.0
daiv/automation/agent/base.py	Add XHIGH to ThinkingLevel; update _get_anthropic_thinking_tokens; replace OPENROUTER branch in get_model_kwargs; add OPENROUTER case to get_max_token_value; update imports
daiv/automation/agent/conf.py	Change MAX_THINKING_LEVEL default to ThinkingLevel.XHIGH
daiv/automation/conf.py	Remove OPENROUTER_API_BASE field
daiv/automation/agent/middlewares/prompt_cache.py	Replace ChatOpenAI isinstance check with ChatOpenRouter
tests/unit_tests/automation/agent/test_base.py	Add test for OpenRouter model kwargs (no openai_api_base, has reasoning/app_url); add XHIGH thinking test
tests/unit_tests/automation/agent/middlewares/test_prompt_cache.py	Update _is_openrouter_anthropic_model to mock ChatOpenRouter

---

Testing

• Run uv run pytest tests/unit_tests/automation/agent/ after implementation.
• Verify get_model_kwargs for openrouter:anthropic/claude-sonnet-4.5 returns model_provider="openrouter", reasoning={"effort": "xhigh"} (when thinking_level=ThinkingLevel.XHIGH), and app_url/app_title set.
• Verify ThinkingLevel.XHIGH value is "xhigh" (matches OpenRouter's effort string exactly).
• Verify _is_openrouter_anthropic_model returns True for ChatOpenRouter with an Anthropic model name.

---

Edge Cases

1. XHIGH in _get_anthropic_thinking_tokens: This method is only called for the direct ModelProvider.ANTHROPIC path (not OpenRouter). Anthropic direct API does not support an "xhigh" effort string — it uses budget_tokens integers. The new XHIGH case in this method should map to a concrete token budget (e.g. 64k, the same as HIGH, since Anthropic direct API caps at max_tokens - 1). Clarify the exact budget during implementation.

2. reasoning_effort for direct OpenAI models: The ModelProvider.OPENAI path sets _kwargs["reasoning_effort"] = thinking_level. XHIGH must also be valid here — the OpenAI API accepts "low", "medium", "high" but not "xhigh". If XHIGH is added to the enum it should either be clamped to "high" for direct OpenAI, or XHIGH should be documented as OpenRouter-only and guarded in the OpenAI branch.

3. anthropic-beta structured-outputs header: Currently "anthropic-beta": "structured-outputs-2025-11-13" is sent via extra_headers for Anthropic-via-OpenRouter. Verify whether ChatOpenRouter supports default_headers or similar to preserve this — otherwise structured output responses from Claude-via-OpenRouter may degrade silently.

[srtab]

@daiv-agent proceed

[daiv-agent]

All done. Here's a summary of what was implemented:

Summary

Files changed

pyproject.toml / uv.lock — Added langchain-openrouter==0.1.0 dependency.

daiv/automation/agent/base.py

• Added ThinkingLevel.XHIGH = "xhigh" to the enum
• Added XHIGH case to _get_anthropic_thinking_tokens (64k thinking budget for direct Anthropic API)
• Replaced the entire OPENROUTER branch in get_model_kwargs: no more openai_api_base, openai_api_key, or extra_body hacks — now uses model_provider=ModelProvider.OPENROUTER (which init_chat_model resolves to ChatOpenRouter) with native openrouter_api_key, app_url, app_title, and reasoning={"effort": ...} params
• Added ModelProvider.OPENROUTER case to get_max_token_value (returns 8192)

daiv/automation/agent/conf.py — Changed MAX_THINKING_LEVEL default from HIGH → XHIGH.

daiv/automation/conf.py — Removed OPENROUTER_API_BASE field (no longer needed; ChatOpenRouter manages the endpoint internally).

daiv/automation/agent/middlewares/prompt_cache.py — Replaced isinstance(model, ChatOpenAI) check with isinstance(model, ChatOpenRouter).

tests/ — Added tests for OPENROUTER kwargs (verifies no legacy fields, verifies reasoning/app_url), XHIGH thinking, get_max_token_value for OpenRouter, and _is_openrouter_anthropic_model with ChatOpenRouter.

Note: The pre-existing circular import between automation.conf and automation.agent.base prevents pytest from loading conftest.py — this is unrelated to these changes and was present on main before.