Skip to content

Simplify docs #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Robert-H-Leonard merged 1 commit into from
Aug 25, 2025
Merged

Simplify docs #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 0 additions & 181 deletions docs/examples/retrieving-results.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,187 +127,6 @@ accuracy = correct / len(all_results) if all_results else 0
print(f"Overall accuracy: {accuracy:.1%} ({correct:,}/{len(all_results):,})")
```

### With Progress Tracking

```python
from atlas import Atlas

def get_all_results_with_progress(evaluation_id):
client = Atlas()
all_results = []
page = 1
page_size = 50

while True:
print(f"Fetching page {page}...")

results_data = client.results.get_by_id(
evaluation_id=evaluation_id,
page=page,
page_size=page_size
)

if not results_data or not results_data.results:
break

all_results.extend(results_data.results)

# Show progress
if page == 1:
total_count = results_data.pagination.total_count
total_pages = results_data.pagination.total_pages
print(f"Total results: {total_count:,} across {total_pages} pages")

progress = len(all_results) / results_data.pagination.total_count * 100
print(f"Progress: {progress:.1f}% ({len(all_results):,} collected)")

if page >= results_data.pagination.total_pages:
break

page += 1

return all_results

# Usage
results = get_all_results_with_progress("your_evaluation_id")
print(f"Done! Collected {len(results):,} results")
```

## Async Version

```python
import asyncio
from atlas import AsyncAtlas

async def get_results_async(evaluation_id):
client = AsyncAtlas()

all_results = []
page = 1

while True:
results_data = await client.results.get_by_id(
evaluation_id=evaluation_id,
page=page,
page_size=100
)

if not results_data or not results_data.results:
break

all_results.extend(results_data.results)
print(f"Page {page}: {len(results_data.results)} results")

if page >= results_data.pagination.total_pages:
break

page += 1

return all_results

# Run it
results = asyncio.run(get_results_async("your_evaluation_id"))
print(f"Total: {len(results)} results")
```

## Complete Workflow

```python
from atlas import Atlas

client = Atlas()

# 1. Create evaluation
models = client.models.get()
benchmarks = client.benchmarks.get()

evaluation = client.evaluations.create(
model=models[0],
benchmark=benchmarks[0]
)
print(f"Created evaluation: {evaluation.id}")

# 2. Wait for completion
print("Waiting for evaluation to complete...")
completed_evaluation = client.evaluations.wait_for_completion(
evaluation,
interval_seconds=30,
timeout_seconds=1800 # 30 minutes
)

# 3. Get all results
if completed_evaluation.is_success:
print("Getting results...")

all_results = []
page = 1

while True:
results_data = client.results.get_by_id(
evaluation_id=completed_evaluation.id,
page=page,
page_size=100
)

if not results_data or not results_data.results:
break

all_results.extend(results_data.results)
if page >= results_data.pagination.total_pages:
break
page += 1

# 4. Analyze results
correct = sum(1 for r in all_results if r.score > 0.5)
accuracy = correct / len(all_results)
avg_score = sum(r.score for r in all_results) / len(all_results)

print(f"Results: {len(all_results):,} total")
print(f"Accuracy: {accuracy:.1%}")
print(f"Average score: {avg_score:.3f}")
else:
print(f"Evaluation failed: {completed_evaluation.status}")
```

## Analyze Results by Subset

```python
from atlas import Atlas
from collections import defaultdict

client = Atlas()
evaluation_id = "your_evaluation_id"

# Get all results
all_results = []
page = 1

while True:
results_data = client.results.get_by_id(evaluation_id=evaluation_id, page=page)
if not results_data or not results_data.results:
break
all_results.extend(results_data.results)
if page >= results_data.pagination.total_pages:
break
page += 1

# Group by subset
subset_results = defaultdict(list)
for result in all_results:
subset_results[result.subset].append(result)

# Analyze each subset
print(f"Analysis by subset:")
for subset, results in subset_results.items():
correct = sum(1 for r in results if r.score > 0.5)
accuracy = correct / len(results)
avg_score = sum(r.score for r in results) / len(results)

print(f" {subset}:")
print(f" Cases: {len(results)}")
print(f" Accuracy: {accuracy:.1%}")
print(f" Avg Score: {avg_score:.3f}")
```

## Key Points

Expand Down
147 changes: 0 additions & 147 deletions docs/security/api-key-management.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,150 +95,3 @@ client = Atlas()
.env.*.local
*.env
```

### Advanced Credential Management

#### Using External Secret Managers

**AWS Secrets Manager**:

```python
import boto3
import json
from atlas import Atlas

def get_atlas_credentials_from_aws():
"""Retrieve Atlas credentials from AWS Secrets Manager"""
session = boto3.session.Session()
client = session.client('secretsmanager', region_name='us-east-1')

try:
response = client.get_secret_value(SecretId='layerlens/atlas/credentials')
secrets = json.loads(response['SecretString'])

return {
'api_key': secrets['api_key'],
}
except Exception as e:
print(f"Error retrieving secrets: {e}")
return None

# Usage
credentials = get_atlas_credentials_from_aws()
if credentials:
client = Atlas(**credentials)
```

## Environment-Specific Key Management

### Separating Development and Production Keys

**Use different API keys for different environments**:

```python
import os
from atlas import Atlas

def get_atlas_client():
"""Get Atlas client based on environment"""
environment = os.getenv('ATLAS_ENV', 'development')

if environment == 'development':
return Atlas(
api_key=os.getenv('DEV_ATLAS_API_KEY'),
base_url=os.getenv('DEV_ATLAS_BASE_URL') # Dev server if applicable
)
elif environment == 'staging':
return Atlas(
api_key=os.getenv('STAGING_ATLAS_API_KEY'),
)
elif environment == 'production':
return Atlas(
api_key=os.getenv('PROD_ATLAS_API_KEY'),
)
else:
raise ValueError(f"Unknown environment: {environment}")

# Usage
client = get_atlas_client()
```

**Environment-specific .env files**:

```bash
# .env.development
DEV_ATLAS_API_KEY=sk-dev-key-here
DEV_ATLAS_BASE_URL=https://dev-api.layerlens.com

# .env.production
PROD_ATLAS_API_KEY=sk-prod-key-here
```

### Container and Deployment Security

**Docker Secrets**:

```yaml
# docker-compose.yml
version: "3.8"

services:
atlas-app:
image: your-app:latest
secrets:
- atlas_api_key
environment:
- LAYERLENS_ATLAS_API_KEY_FILE=/run/secrets/atlas_api_key

secrets:
atlas_api_key:
file: ./secrets/atlas_api_key.txt
```

**Reading Docker secrets in Python**:

```python
import os
from atlas import Atlas

def read_docker_secret(secret_name):
"""Read secret from Docker secrets file"""
secret_file = f"/run/secrets/{secret_name}"
try:
with open(secret_file, 'r') as f:
return f.read().strip()
except FileNotFoundError:
return None

def get_atlas_client_from_docker_secrets():
"""Initialize Atlas client using Docker secrets"""
# Try Docker secrets first, fall back to environment variables
api_key = (read_docker_secret('atlas_api_key') or
os.getenv('LAYERLENS_ATLAS_API_KEY'))

return Atlas(api_key=api_key)

# Usage
client = get_atlas_client_from_docker_secrets()
```

## Security Checklist

### Development Security Checklist

- [ ] ✅ API keys stored in environment variables, not hardcoded
- [ ] ✅ `.env` files added to `.gitignore`
- [ ] ✅ Different API keys for development, staging, and production
- [ ] ✅ API key validation implemented before deployment
- [ ] ✅ Error handling doesn't expose API keys in logs
- [ ] ✅ Code review process includes credential security checks

### Production Security Checklist

- [ ] ✅ API keys stored in secure credential management system
- [ ] ✅ Key rotation schedule established and automated
- [ ] ✅ API usage monitoring and alerting configured
- [ ] ✅ Audit logging enabled for all API operations
- [ ] ✅ Network security controls (firewalls, VPNs) in place
- [ ] ✅ Least privilege access principles applied
- [ ] ✅ Incident response plan includes credential compromise scenarios
Loading
Loading