Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions samples/instrument/agno/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Agno instrumentation sample

End-to-end demo of `AgnoAdapter` — wraps a one-shot Agno `Agent`
backed by `OpenAIChat("gpt-4o-mini")`, runs a single `agent.run()`
call, and ships telemetry to atlas-app via `HttpEventSink`.

## Prerequisites

```bash
pip install 'layerlens[agno,providers-openai]'
```

Required environment:

- `OPENAI_API_KEY` — used by `OpenAIChat`. If unset the sample exits
with code `2`.
- `LAYERLENS_STRATIX_API_KEY` — your LayerLens API key (optional).
- `LAYERLENS_STRATIX_BASE_URL` — atlas-app base URL (optional).

## Run

```bash
uv run python -m samples.instrument.agno.main
```

## What this demonstrates

| Component | What it proves (source: `main.py`) |
|---|---|
| `AgnoAdapter(capture_config=CaptureConfig.standard())` | Adapter constructed with the standard capture profile. |
| `adapter.add_sink(HttpEventSink(...))` | Events batched to `/telemetry/spans` (`max_batch=10`, `flush_interval_s=1.0`). |
| `adapter.connect()` / `adapter.disconnect()` | Full lifecycle: instrument before invocation, restore on exit. |
| `adapter.instrument_agent(agent)` | Wraps `Agent.run` so `agent.input` + `model.invoke` + `agent.output` events fire on each call (per `main.py` module docstring). |
| Response print | The sample prints the `response.content` string returned by `Agent.run("What is 2 + 2?")`. |

## Expected output

When `OPENAI_API_KEY` is unset:

```text
OPENAI_API_KEY is not set; cannot run sample.
```

Exit code: `2`.

When `agno` is not installed:

```text
agno not installed. Install with:
pip install 'layerlens[agno,providers-openai]'
```

Exit code: `2`.

When the call succeeds the sample prints the model response and a
shipping confirmation:

```text
Response: <model reply>
Telemetry shipped. Check the LayerLens dashboard adapter health page.
```

## Multi-tenancy note

This sample does not pass `org_id` to the adapter constructor. The
`AgnoAdapter` constructor does not yet accept `org_id` — production
multi-tenant wiring lands with the PR #118 adapter-side contract
(currently DRAFT). In the interim, set `org_id` per-event through
`CaptureConfig` overrides or via `HttpEventSink` headers on the
caller side. Once PR #118 merges, pass `org_id="<your-org>"` to
`AgnoAdapter(...)` directly.
81 changes: 81 additions & 0 deletions samples/instrument/bedrock_agents/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# AWS Bedrock Agents instrumentation sample

End-to-end demo of `BedrockAgentsAdapter` — wraps a
`bedrock-agent-runtime` boto3 client, runs a single `invoke_agent`
call, drains the streamed response, and ships telemetry events to
atlas-app via `HttpEventSink`.

> **Live-only.** This sample requires a real Bedrock Agent ID; there
> is no mock mode. If `BEDROCK_AGENT_ID` is unset the sample exits
> cleanly with code `2`.

## Prerequisites

```bash
pip install 'layerlens[bedrock-agents]'
```

The `bedrock-agents` extra installs `boto3>=1.34`.

Required environment:

- `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` (or any standard
boto3 credential source — IAM role, profile, etc.).
- `AWS_REGION` — region your agent lives in (defaults to
`us-east-1`).
- `BEDROCK_AGENT_ID` — your Bedrock Agent ID (e.g. `ABCDEFGHIJ`).
- `BEDROCK_AGENT_ALIAS_ID` — alias to invoke (defaults to
`TSTALIASID`).
- `LAYERLENS_STRATIX_API_KEY` — your LayerLens API key (optional).
- `LAYERLENS_STRATIX_BASE_URL` — atlas-app base URL (optional).

## Run

```bash
uv run python -m samples.instrument.bedrock_agents.main
```

## What this demonstrates

| Component | What it proves (source: `main.py`) |
|---|---|
| `BedrockAgentsAdapter(capture_config=CaptureConfig.standard())` | Standard capture profile. |
| `adapter.add_sink(HttpEventSink(...))` | Batched HTTP transport. |
| `adapter.connect()` / `adapter.disconnect()` | Full lifecycle. |
| `adapter.instrument_client(client)` | Registers event hooks on the boto3 `bedrock-agent-runtime` client; the module docstring states this emits `agent.input` + `model.invoke` + `tool.call` + `agent.output` events. |
| `client.invoke_agent(agentId=..., agentAliasId=..., sessionId=uuid4(), inputText="What is 2 + 2?")` | Live invocation against your configured agent. |
| Streamed response drain (`for event in response["completion"]: ...`) | Trace events fire as each chunk is iterated; concatenated bytes are decoded and printed. |

## Expected output

When `BEDROCK_AGENT_ID` is unset:

```text
BEDROCK_AGENT_ID is not set; cannot run sample.
```

Exit code: `2`.

When `boto3` is not installed:

```text
boto3 not installed. Install with:
pip install 'layerlens[bedrock-agents]'
```

Exit code: `2`.

When the call succeeds:

```text
Response: <agent reply>
Telemetry shipped. Check the LayerLens dashboard adapter health page.
```

## Multi-tenancy note

This sample does not pass `org_id` to `BedrockAgentsAdapter`. The
constructor does not yet accept `org_id` — production multi-tenant
wiring lands with the PR #118 adapter-side contract (currently
DRAFT). Once PR #118 merges, pass `org_id` to the adapter so every
emitted event carries it.
71 changes: 71 additions & 0 deletions samples/instrument/benchmark_import/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Benchmark import sample

End-to-end demo of `BenchmarkImportAdapter` — writes a 3-row CSV to
a temp dir, calls `BenchmarkImportAdapter.import_csv` against it
with a schema mapping and tags, and prints the resulting
`ImportResult`.

> `BenchmarkImportAdapter` is a **data importer**, not a runtime
> trace adapter. It does not require any LLM credentials, does not
> instantiate an `HttpEventSink`, and does not emit runtime
> `agent.*` events. This is consistent with the adapter's design
> (per audit row for `benchmark_import`).

## Prerequisites

```bash
pip install 'layerlens[benchmark-import]'
```

The `benchmark-import` extra is intentionally empty — this sample
is replay-based and ships no additional runtime deps beyond
`layerlens` core.

No environment variables are required.

## Run

```bash
uv run python -m samples.instrument.benchmark_import.main
```

## What this demonstrates

| Component | What it proves (source: `main.py`) |
|---|---|
| `BenchmarkImportAdapter()` | Adapter constructs with no args — no sink, no capture config, no transport. |
| Temp CSV via `_write_sample_csv` | Three QA rows (math, geo, science) written via `csv.DictWriter`. |
| `adapter.import_csv(path=..., schema_mapping={"question": "prompt", "answer": "expected_output", "category": "category"}, tags=["sample", "qa"])` | Demonstrates the schema-mapping contract and tag attachment. |
| `result.success` / `result.errors` | Sample bails with exit code `1` and prints errors if the import fails. |
| `result.benchmark_id`, `result.records_imported`, `result.duration_ms`, `result.metadata.tags` | Sample prints every public field of the `ImportResult`. |

## Expected output

On success:

```text
Benchmark id: <uuid-or-id-string>
Records imported: 3
Duration: <float> ms
Tags: sample, qa
```

Exit code: `0`.

On failure:

```text
Import failed: [<error1>, <error2>, ...]
```

Exit code: `1`.

## Multi-tenancy note

This sample does not pass `org_id` to `BenchmarkImportAdapter`. The
constructor does not yet accept `org_id` — production multi-tenant
wiring lands with the PR #118 adapter-side contract (currently
DRAFT). For batch imports, the multi-tenant story is to scope the
`ImportResult.benchmark_id` to an `org_id` on the atlas-app side
when the record is persisted; the importer itself does not need
the org context to construct the dataset.
79 changes: 79 additions & 0 deletions samples/instrument/embedding/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Embedding instrumentation sample

End-to-end demo of `EmbeddingAdapter` — wraps an OpenAI client with
`EmbeddingAdapter.wrap_openai` and runs a single `embeddings.create`
call, emitting one `embedding.create` event per the module docstring.

## Prerequisites

```bash
pip install 'layerlens[embedding,providers-openai]'
```

> The `embedding` extra is intentionally empty — vector store hooks
> come from whichever store you actually use. This sample only
> exercises the OpenAI embedding path, so it needs the
> `providers-openai` extra for the `openai` SDK.

Required environment:

- `OPENAI_API_KEY` — your OpenAI API key. If unset the sample exits
with code `2`.
- `LAYERLENS_STRATIX_API_KEY` — your LayerLens API key (optional).
- `LAYERLENS_STRATIX_BASE_URL` — atlas-app base URL (optional).

## Run

```bash
uv run python -m samples.instrument.embedding.main
```

## What this demonstrates

| Component | What it proves (source: `main.py`) |
|---|---|
| `EmbeddingAdapter(capture_config=CaptureConfig.standard())` | Standard capture profile. |
| `adapter.add_sink(HttpEventSink(...))` | Batched HTTP transport. |
| `adapter.connect()` / `adapter.disconnect()` | Full lifecycle. |
| `adapter.wrap_openai(client)` | Wraps the `OpenAI()` client's `embeddings.create` path; module docstring states each call emits one `embedding.create` event. |
| `client.embeddings.create(model="text-embedding-3-small", input=[..., ...])` | Two input strings → two vectors. The sample prints vector count and dimension. |
| Token usage print | When `response.usage` is non-`None`, total tokens are printed. |

## Expected output

When `OPENAI_API_KEY` is unset:

```text
OPENAI_API_KEY is not set; cannot run sample.
```

Exit code: `2`.

When the `openai` SDK is not installed:

```text
openai not installed. Install with:
pip install 'layerlens[embedding,providers-openai]'
```

Exit code: `2`.

When the call succeeds:

```text
Embeddings: 2 vectors of dim 1536
Tokens: <n>
Telemetry shipped. Check the LayerLens dashboard adapter health page.
```

(The token line is conditional on `response.usage` being non-`None`.
The dim value is whatever `text-embedding-3-small` returns; 1536 is
the documented OpenAI dimension at the time of writing.)

## Multi-tenancy note

This sample does not pass `org_id` to `EmbeddingAdapter`. The
constructor does not yet accept `org_id` — production multi-tenant
wiring lands with the PR #118 adapter-side contract (currently
DRAFT). Once PR #118 merges, pass `org_id` to the adapter so every
emitted event carries it.
77 changes: 77 additions & 0 deletions samples/instrument/google_adk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Google ADK instrumentation sample

End-to-end demo of `GoogleADKAdapter` — builds a one-shot `LlmAgent`
backed by `gemini-2.0-flash`, attaches LayerLens callbacks, and runs
a single turn through the ADK `InMemoryRunner`.

## Prerequisites

```bash
pip install 'layerlens[google-adk]'
```

Required environment (one of):

- `GOOGLE_API_KEY` — used by the Gemini model against Google AI
Studio. **OR**
- `GOOGLE_GENAI_USE_VERTEXAI=true` — route through Vertex AI; supply
Application Default Credentials.

If neither is set the sample exits with code `2`.

Optional:

- `LAYERLENS_STRATIX_API_KEY` — your LayerLens API key.
- `LAYERLENS_STRATIX_BASE_URL` — atlas-app base URL.

## Run

```bash
uv run python -m samples.instrument.google_adk.main
```

## What this demonstrates

| Component | What it proves (source: `main.py`) |
|---|---|
| `GoogleADKAdapter(capture_config=CaptureConfig.standard())` | Standard capture profile. |
| `adapter.add_sink(HttpEventSink(...))` | Batched HTTP transport. |
| `adapter.connect()` / `adapter.disconnect()` | Full lifecycle. |
| `adapter.instrument_agent(agent)` | Attaches LayerLens callbacks to the `LlmAgent`; each callback emits a LayerLens event (per `main.py` module docstring). |
| `InMemoryRunner(agent=agent, app_name="layerlens-sample")` + `run_async` | Async iteration over the ADK event stream; text parts are concatenated and printed. |
| Pre-created session via `runner.session_service.create_session(...)` | Demonstrates the required ADK pattern for giving `run_async` a session to write into. |

## Expected output

When neither `GOOGLE_API_KEY` nor `GOOGLE_GENAI_USE_VERTEXAI=true` is
set:

```text
Neither GOOGLE_API_KEY nor GOOGLE_GENAI_USE_VERTEXAI is set; cannot run sample.
```

Exit code: `2`.

When `google-adk` is not installed:

```text
google-adk not installed. Install with:
pip install 'layerlens[google-adk]'
```

Exit code: `2`.

When the call succeeds:

```text
Response: <model reply>
Telemetry shipped. Check the LayerLens dashboard adapter health page.
```

## Multi-tenancy note

This sample does not pass `org_id` to `GoogleADKAdapter`. The
constructor does not yet accept `org_id` — production multi-tenant
wiring lands with the PR #118 adapter-side contract (currently
DRAFT). Once PR #118 merges, pass `org_id` to the adapter so every
emitted event carries it.
Loading