Skip to content

feat: add LiteLLM as model backend for 100+ LLM providers#50

Open
RheagalFire wants to merge 3 commits into
microsoft:mainfrom
RheagalFire:feat/add-litellm-provider
Open

feat: add LiteLLM as model backend for 100+ LLM providers#50
RheagalFire wants to merge 3 commits into
microsoft:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire

Copy link
Copy Markdown

Summary

  • Add LiteLLM as a model backend (litellm) using the litellm Python SDK, enabling access to 100+ LLM providers (Anthropic, Bedrock, Vertex, Gemini, Cohere, Mistral, etc.) without needing a separate proxy
  • Overrides _post_with_retries to call litellm.acompletion() directly instead of httpx, while inheriting the base class retry logic
  • Uses drop_params=True for cross-provider compatibility

Changes

  • src/webwright/models/litellm_model.py - new LiteLLMModel backend using litellm SDK with acompletion()
  • src/webwright/models/__init__.py - registered "litellm" in _MODEL_MAPPING

Tests

from webwright.models import get_model

model = get_model({
    "model_class": "litellm",
    "model_name": "anthropic/claude-sonnet-4-6",
})
response = model([{"role": "user", "content": "What is 2+2? Answer in one word."}])
SDK Response: 'Four'
SDK E2E PASSED

Example usage

from webwright.models import get_model

# Anthropic via litellm SDK (reads ANTHROPIC_API_KEY from env)
model = get_model({"model_class": "litellm", "model_name": "anthropic/claude-sonnet-4-6"})

# OpenAI via litellm SDK (reads OPENAI_API_KEY from env)
model = get_model({"model_class": "litellm", "model_name": "gpt-4o"})

# Bedrock via litellm SDK (reads AWS creds from env)
model = get_model({"model_class": "litellm", "model_name": "bedrock/claude-3.5-sonnet"})

Risk / Compatibility

  • Additive only. Existing OpenAI/Anthropic/OpenRouter backends untouched.
  • litellm is an optional dependency, imported at model init time.

@RheagalFire

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant