Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/workflows/publish-to-aws.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,12 @@ jobs:
ls -la

echo "Checking if version file exists..."
if [ -f "src/atlas/_version.py" ]; then
if [ -f "src/layerlens/_version.py" ]; then
echo "Version file found:"
cat src/atlas/_version.py
cat src/layerlens/_version.py

# Extract version directly
VERSION=$(grep -E '^__version__\s*=' src/atlas/_version.py | grep -o '".*"' | tr -d '"')
VERSION=$(grep -E '^__version__\s*=' src/layerlens/_version.py | grep -o '".*"' | tr -d '"')
echo "Direct extraction result: '$VERSION'"
else
echo "Version file not found, trying script method..."
Expand Down Expand Up @@ -293,10 +293,10 @@ jobs:
<!DOCTYPE html>
<html>
<head>
<title>LayerLens Atlas SDK</title>
<title>LayerLens Stratix SDK</title>
</head>
<body>
<h1>LayerLens Atlas SDK</h1>
<h1>LayerLens Stratix SDK</h1>
<a href="${{ env.PACKAGE_NAME }}/">${{ env.PACKAGE_NAME }}</a><br/>
</body>
</html>
Expand Down
5 changes: 5 additions & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,14 @@
* [Client Configuration](docs/api-reference/client.md)
* [Models & Benchmarks](docs/api-reference/models-benchmarks.md)
* [Evaluations](docs/api-reference/evaluations.md)
* [Judges](docs/api-reference/judges.md)
* [Traces](docs/api-reference/traces.md)
* [Trace Evaluations](docs/api-reference/trace-evaluations.md)
* [Judge Optimizations](docs/api-reference/judge-optimizations.md)
* [Error Handling](docs/api-reference/errors.md)
* [Examples](docs/examples/README.md)
* [Running Evaluations](docs/examples/creating-evaluations.md)
* [Judges and Traces](docs/examples/judges-and-traces.md)
* [Troubleshooting](docs/troubleshooting/README.md)
* [Authentication Problems](docs/troubleshooting/authentication.md)
* [Error Codes Reference](docs/troubleshooting/error-codes.md)
20 changes: 10 additions & 10 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Layerlens Python SDK Documentation

Welcome to the official documentation for the Layerlens Python SDK for the atlas platform. This library provides convenient programmatic to the Atlas platform from any Python 3.8+ application.
Welcome to the official documentation for the Layerlens Python SDK for the Stratix platform. This library provides convenient programmatic to the Stratix platform from any Python 3.8+ application.

## What is Atlas?
## What is Stratix?

Atlas is an evaluation platform that allows you to benchmark AI models against various datasets and metrics. The Python SDK provides two HTTP clients (syncronous and asynchronous) powered by [httpx](https://github.com/encode/httpx) and [Pydantic](https://pydantic.dev/) models for type-safe API interactions.
Stratix is an evaluation platform that allows you to benchmark AI models against various datasets and metrics. The Python SDK provides two HTTP clients (syncronous and asynchronous) powered by [httpx](https://github.com/encode/httpx) and [Pydantic](https://pydantic.dev/) models for type-safe API interactions.

## Quick Start

Expand All @@ -16,27 +16,27 @@ Install the layerlens python sdk using the following command
pip install layerlens --index-url https://sdk.layerlens.ai/package
```

### Generate an api key on the atlas platform
### Generate an api key on the Stratix platform

Login to your organization at [app.layerlens.ai](https://app.layerlens.ai) to generate an api key. Admin users of organizations can generate a keys in the settings page.

Run this command to add your API key to your environment:

```bash
export LAYERLENS_ATLAS_API_KEY="YOUR_API_KEY"
export LAYERLENS_STRATIX_API_KEY="YOUR_API_KEY"
```

### Running an evaluation on the atlas platform
### Running an evaluation on the Stratix platform

Before triggering an evaluation using the sdk, login to your organization at [app.layerlens.ai](https://app.layerlens.ai) to ensure that the model and benchmark you are trying to evaluate has been added to your organizations dashboard.

#### Using synchronous client

```python
from layerlens import Atlas
from layerlens import Stratix

# Construct sync client
client = Atlas()
client = Stratix()

# --- Models replace with the model key you want to run
model = client.models.get_by_key("openai/gpt-4o")
Expand All @@ -61,11 +61,11 @@ from layerlens import Atlas

```python
import asyncio
from layerlens import AsyncAtlas
from layerlens import AsyncStratix

async def run_evaluation_async():
# Construct async client
client = AsyncAtlas()
client = AsyncStratix()

# --- Model to use
model = await client.models.get_by_key("openai/gpt-4o")
Expand Down
7 changes: 6 additions & 1 deletion docs/SUMMARY.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Table of Contents - Atlas Python SDK
# Table of Contents - Stratix Python SDK

* [Introduction](README.md)

Expand All @@ -12,10 +12,15 @@
* [Evaluations](api-reference/evaluations.md)
* [Results](api-reference/results.md)
* [Models & Benchmarks](api-reference/models-benchmarks.md)
* [Judges](api-reference/judges.md)
* [Traces](api-reference/traces.md)
* [Trace Evaluations](api-reference/trace-evaluations.md)
* [Judge Optimizations](api-reference/judge-optimizations.md)
* [Error Handling](api-reference/errors.md)

## Code Examples
* [Creating Evaluations](examples/creating-evaluations.md)
* [Judges and Traces](examples/judges-and-traces.md)
* [Retrieving Results](examples/retrieving-results.md)
* [Working with Timeouts](examples/timeouts.md)
* [Advanced Usage Patterns](examples/advanced-usage.md)
Expand Down
32 changes: 16 additions & 16 deletions docs/api-reference/client.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,44 @@
# Client Configuration

The `Atlas` (syncronous) and `AsyncAtlas` (asyncronous) classes are the main entry points for interacting with the LayerLens Atlas sdk. This page covers client initialization, configuration options, and usage patterns.
The `Stratix` (syncronous) and `AsyncStratix` (asyncronous) classes are the main entry points for interacting with the LayerLens Stratix sdk. This page covers client initialization, configuration options, and usage patterns.

## Basic Usage

### Syncronous Client

```python
from layerlens import Atlas
from layerlens import Stratix

# Construct syncronous client
# Loads for api key from the "LAYERLENS_ATLAS_API_KEY" enviornment variable
client = Atlas()
# Loads for api key from the "LAYERLENS_STRATIX_API_KEY" enviornment variable
client = Stratix()

# Explicit configuration
client = Atlas(api_key="your_api_key")
client = Stratix(api_key="your_api_key")
```

### Asyncronous Client

```python
import asyncio
from layerlens import AsyncAtlas
from layerlens import AsyncStratix

# Construct async client
# Loads for api key from the "LAYERLENS_ATLAS_API_KEY" enviornment variable
client = AsyncAtlas()
# Loads for api key from the "LAYERLENS_STRATIX_API_KEY" enviornment variable
client = AsyncStratix()

# Explicit configuration
client = AsyncAtlas(api_key="your_api_key")
client = AsyncStratix(api_key="your_api_key")
```

## Constructor Parameters

### `Atlas(api_key, base_url, timeout)` and `AsyncAtlas(api_key, base_url, timeout)`
### `Stratix(api_key, base_url, timeout)` and `AsyncStratix(api_key, base_url, timeout)`

| Parameter | Type | Required | Default | Description |
| ---------- | -------------------------------- | -------- | ------------- | ----------------------------- |
| `api_key` | `str \| None` | Yes\* | `None` | Your LayerLens Atlas API key |
| `base_url` | `str \| httpx.URL \| None` | No | Atlas API URL | Custom API base URL |
| `api_key` | `str \| None` | Yes\* | `None` | Your LayerLens Stratix API key |
| `base_url` | `str \| httpx.URL \| None` | No | Stratix API URL | Custom API base URL |
| `timeout` | `float \| httpx.Timeout \| None` | No | 10 minutes | Request timeout configuration |

\*Required unless set via environment variables
Expand All @@ -48,24 +48,24 @@ client = AsyncAtlas(api_key="your_api_key")
The client automatically loads configuration from these environment variables:

```bash
LAYERLENS_ATLAS_API_KEY="your_api_key_here"
LAYERLENS_STRATIX_API_KEY="your_api_key_here"
```

## Timeout Configuration

### Simple Timeout

```python
from layerlens import Atlas
from layerlens import Stratix

# 30-second timeout for all requests
client = Atlas(timeout=30.0)
client = Stratix(timeout=30.0)
```

### Per-Request Timeout Override

```python
client = Atlas()
client = Stratix()

# --- Models replace with the model name you want to run
models = client.models.get(type="public", name="gpt-4o")
Expand Down
Loading
Loading