v1.8.0

Features

• benchmark_key, model_key_1, and model_key_2 parameters on comparisons.compare_models (sync + async). Address the benchmark and the two models by their unique key (e.g., aime2024, openai/gpt-4o) instead of by UUID; the existing *_id parameters keep working. Exactly one of *_id or *_key must be provided per entity — passing both, or neither, raises ValueError. Unknown keys raise ValueError with the offending key in the message.

v1.7.0

Features

• new extra_payload keyword argument to models.create_custom method
• new extra_payload keyword argument to models.update_custom method

v1.6.1

Features

• CLI authentication command (layerlens auth) (#72)
• models.update_custom(model_id, *, api_url, api_key, max_tokens) (sync + async) — repoint a custom model's mutable fields without recreating it (#169)
• models.delete_custom(model_id) (sync + async) — full teardown that disables the record, strips it from Project.Models, and releases the name for reuse (#169)
• 70+ production-ready SDK samples across 12 categories: core, industry, cowork, modalities, integrations, cicd, cli, openclaw, mcp, copilotkit, claude-code, data (#73)
• MCP server sample exposing LayerLens as tools
• CopilotKit sample with LangGraph CoAgents, React components, and hooks
• New trace samples (#144)

Bug fix

• models.add() / models.remove() now operate on the full project model list (public + custom). The previous type="public" filter silently dropped custom-model IDs from Project.Models on every call (#169)
• Expanded SDK documentation and README (#139, #167)

v1.6.0

Feature

• Prompts exposed on the private client.

v1.5.0

Features

• CLI - Full-featured command-line interface via layerlens / stratix.
• New client.scorers with full CRUD: create, get, list, update, delete.
• New client.evaluation_spaces with get, list, create, update, delete.
• New client.integrations with get, list, create, update, delete, and test.

Bug fix

• filter by categories/languages/companies/regions/licenses returns correct results

Docs

• Added CLI getting started guide, command reference, and examples
• Added Scorers API reference
• Updated evaluations, models & benchmarks, and public client docs with new parameters

v1.4.0

Features

• Unique evaluations parameter - Added unique parameter to evaluations.get_many() and public_evaluations.get_many() that deduplicates results by model+dataset pair, keeping only the latest evaluation per pair.

Bug fix

• Model comparison - Comparisons now pass unique=True when fetching evaluations, ensuring the correct (latest) evaluation is used for each model+benchmark pair instead of potentially picking up duplicates.

v1.3.3

Bug fix

• API naming convention - Fixed inconsistent naming across the SDK to follow a unified convention. Affected resources: comparisons, evaluations, judges, results, trace evaluations, traces, public benchmarks/evaluations/models. (#61)
• Benchmarks & Models resources - Added missing methods to benchmarks and models resources.
• Documentation - Fixed SUMMARY.md structure and updated examples to match new naming.

v1.3.2

Bug fix

• Fixed trace_evaluations.get_results() always returning empty/None results. The API returns evaluation data (score, passed, reasoning, steps) directly, but the SDK was looking for a non-existent results array. TraceEvaluationResultsResponse now correctly maps to the API response shape and inherits from TraceEvaluationResult.
• Fixed TraceEvaluationStep model to match actual API fields (tool, args, result) instead of the incorrect (step, reasoning).
• Added missing documentation pages for GitBook (getting-started, troubleshooting, security).

v1.3.1

Reliability improvements

• Added automatic retry with exponential backoff for transient errors (HTTP 429, 500, 502, 503, 504) in both sync and async clients.
• Up to 2 retries, respects Retry-After header, max 8s delay.
• Expanded documentation: updated README, added examples for models/benchmarks, public API, retrieving results.

v1.3.0

Enhancements

• Expanded model and benchmark result models with additional fields.
• Fixed CI/CD publish workflows.