Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions docs/adapters/frameworks/autogen.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# AutoGen framework adapter

`layerlens.instrument.adapters.frameworks.autogen.AutoGenAdapter` instruments
[Microsoft AutoGen](https://github.com/microsoft/autogen) `ConversableAgent`
objects, capturing message exchange, LLM calls, code execution, and group-chat
turns.

## Install

```bash
pip install 'layerlens[autogen]'
```

Pulls `pyautogen>=0.2,<0.5`.

## Quick start

```python
from autogen import AssistantAgent, UserProxyAgent

from layerlens import LayerLens
from layerlens.instrument.adapters.frameworks.autogen import (
AutoGenAdapter,
instrument_agents,
)

client = LayerLens() # picks up LAYERLENS_STRATIX_API_KEY / _BASE_URL

adapter = AutoGenAdapter(stratix=client)
adapter.connect()

assistant = AssistantAgent(name="assistant", llm_config={"model": "gpt-4o-mini"})
user = UserProxyAgent(name="user", human_input_mode="NEVER", code_execution_config=False)

adapter.connect_agents(assistant, user)
user.initiate_chat(assistant, message="What is 2+2?", max_turns=1)

adapter.disconnect()
```

`instrument_agents(assistant, user, stratix=client)` is the one-line
equivalent of the `AutoGenAdapter(...).connect()` + `connect_agents(...)`
sequence above.

For an offline / no-API-key demonstration, see
`samples/instrument/autogen/main.py` — it wires the adapter against an
in-process `EventSink` and a duck-typed agent so you can inspect the
emitted event stream without any external services.

## What's wrapped

`adapter.connect_agents(*agents)` monkey-patches the following on each
`ConversableAgent`:

- `send` — emits `agent.handoff` for the outgoing message.
- `receive` — emits `agent.state.change` for the incoming message.
- `generate_reply` — emits `model.invoke` (with token usage when
available).
- `execute_code_blocks` — emits `tool.call` and `tool.environment` for
the code execution.

The originals are stashed on the adapter and restored on `disconnect()`.
A `GroupChatTracer` wires similar hooks onto `GroupChatManager`, and a
`HumanProxyTracer` traces `get_human_input` for human-in-the-loop
proxies (emitting `agent.input` events with `role: "HUMAN"`).

## Events emitted

| Event | Layer | When |
|----------------------|----------------|--------------------------------------------------------------------------|
| `environment.config` | L4a | First time each agent is seen by `connect_agents`. |
| `agent.input` | L1 | `on_conversation_start` and human-input requests. |
| `agent.output` | L1 | `on_conversation_end` and group-chat termination. |
| `agent.handoff` | cross-cutting | Every `send` (carries `from_agent`, `to_agent`, `message_seq`). |
| `agent.state.change` | cross-cutting | Every `receive` (carries `agent`, `from_agent`, `message_preview`). |
| `agent.code` | L2 | Group-chat speaker selection (via `GroupChatTracer.on_speaker_selected`).|
| `tool.call` | L5a | Each `execute_code_blocks` call (`tool_name: "code_execution"`). |
| `tool.environment` | L5c | Each `execute_code_blocks` call (execution context details). |
| `model.invoke` | L3 | Each `generate_reply` (with token usage / latency / messages). |

## AutoGen specifics

- **Multi-agent attribution**: `agent_name`, `recipient_name`, and
`message_seq` (a monotonic counter) are included on every event so the
full chat can be reconstructed in order.
- **Group chats**: `GroupChatTracer` registers as a callback on
`GroupChatManager`, capturing the speaker-selection turns. Pass a
`GroupChatManager` to `connect_agents` alongside the participants.
- **Code execution**: when an agent runs code blocks, the language and
truncated code body emit `agent.code` (only if
`CaptureConfig.l2_agent_code` is enabled).

## Capture config

```python
from layerlens.instrument.adapters._base import CaptureConfig

# Recommended.
adapter = AutoGenAdapter(capture_config=CaptureConfig.standard())

# Production-light: skip the verbose code-execution events.
adapter = AutoGenAdapter(
capture_config=CaptureConfig(
l1_agent_io=True,
l3_model_metadata=True,
l4a_environment_config=True,
l5a_tool_calls=True,
l2_agent_code=False,
),
)
```

## BYOK

AutoGen reads its `llm_config` to instantiate provider clients. The adapter
does not own those keys. For platform-managed BYOK see
`docs/adapters/byok.md` (atlas-app M1.B).
14 changes: 13 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ classifiers = [
[project.optional-dependencies]
cli = ["click>=8.0.0"]

# --- Instrument layer: framework adapters ---
# Adding any extra below MUST keep the default `pip install layerlens`
# install set unchanged. Verified by `tests/instrument/test_default_install.py`.
autogen = ["pyautogen>=0.2,<0.5; python_version >= '3.10'"]

[project.urls]
Homepage = "https://github.com/LayerLens/stratix-python"
Repository = "https://github.com/LayerLens/stratix-python"
Expand Down Expand Up @@ -139,14 +144,21 @@ known-first-party = ["openai", "tests"]
"tests/**.py" = ["T201", "T203", "ARG", "B007"]
"examples/**.py" = ["T201", "T203"]
"src/layerlens/cli/**" = ["T201", "T203"]
# Framework callbacks have signatures dictated by upstream — unused
# arguments are part of the contract, not a code smell.
"src/layerlens/instrument/adapters/frameworks/**.py" = ["ARG002"]

[tool.pyright]
include = ["src", "tests"]
exclude = ["**/__pycache__"]
reportMissingTypeStubs = false

# Less strict settings for tests and cli
# Less strict settings for tests, cli, and the dynamic-monkey-patching
# framework adapter code. mypy --strict stays strict for these dirs;
# pyright is relaxed here because it can't follow runtime attribute
# mutation that the framework instrumentation relies on.
executionEnvironments = [
{ root = "src/layerlens/cli", reportMissingImports = false, reportFunctionMemberAccess = false, reportCallIssue = false, reportArgumentType = false, reportAttributeAccessIssue = false },
{ root = "src/layerlens/instrument/adapters/frameworks", reportPossiblyUnbound = false, reportPossiblyUnboundVariable = false, reportCallIssue = false, reportAttributeAccessIssue = false, reportArgumentType = false, reportMissingImports = false, reportFunctionMemberAccess = false },
{ root = "tests", reportGeneralTypeIssues = false, reportOptionalSubscript = false, reportOptionalMemberAccess = false, reportUntypedFunctionDecorator = false, reportUnknownArgumentType = false, reportUnknownMemberType = false, reportUnknownVariableType = false, reportUnnecessaryIsInstance = false, reportUnnecessaryComparison = false, reportArgumentType = false, reportCallIssue = false },
]
Empty file.
145 changes: 145 additions & 0 deletions samples/instrument/autogen/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
"""Sample: instrument an AutoGen-style two-agent exchange with LayerLens.

This sample is fully self-contained — no network, no API keys, no
``pyautogen`` install required. It demonstrates the
``AutoGenAdapter`` lifecycle by:

1. Defining two minimal duck-typed agents that mimic AutoGen's
``ConversableAgent`` interface (``name``, ``send``, ``receive``,
``generate_reply``).
2. Connecting them through ``AutoGenAdapter.connect_agents``.
3. Driving a single send / generate_reply / receive round-trip.
4. Routing the resulting events through an in-process
:class:`EventSink` and printing the captured stream.

To wire this against the real AutoGen runtime, replace the
``_FakeAgent`` instances with ``autogen.AssistantAgent`` /
``autogen.UserProxyAgent`` and provide ``llm_config`` with real
``OPENAI_API_KEY`` / ``ANTHROPIC_API_KEY`` credentials.

Run::

python -m samples.instrument.autogen.main
"""

from __future__ import annotations

from typing import Any, Dict, List

from layerlens.instrument.adapters._base import CaptureConfig
from layerlens.instrument.adapters._base.sinks import EventSink
from layerlens.instrument.adapters.frameworks.autogen import (
AutoGenAdapter,
instrument_agents,
)


class _PrintSink(EventSink):
"""Sink that prints each event and keeps an in-memory log."""

def __init__(self) -> None:
self.events: List[Dict[str, Any]] = []

def send(self, event_type: str, payload: Dict[str, Any], timestamp_ns: int) -> None:
self.events.append({"event_type": event_type, "payload": payload, "ts_ns": timestamp_ns})
print(f" [{event_type}] {sorted(payload.keys())}")

def flush(self) -> None:
pass

def close(self) -> None:
pass


class _RecordingStratix:
"""LayerLens-shaped client stub that the adapter emits into."""

def __init__(self) -> None:
self.events: List[Dict[str, Any]] = []

def emit(self, *args: Any, **kwargs: Any) -> None:
if len(args) == 2 and isinstance(args[0], str):
self.events.append({"event_type": args[0], "payload": args[1]})


class _FakeAgent:
"""Minimal duck-typed AutoGen ``ConversableAgent`` for offline demos."""

def __init__(
self,
name: str,
system_message: str = "",
llm_config: Dict[str, Any] | None = None,
canned_reply: str = "",
) -> None:
self.name = name
self.system_message = system_message
self.llm_config = llm_config
self.human_input_mode = "NEVER"
self._canned_reply = canned_reply
self.received: List[Any] = []

def send(self, message: Any, recipient: Any, **_kwargs: Any) -> Any:
recipient.receive(message, sender=self)

def receive(self, message: Any, sender: Any, **_kwargs: Any) -> Any:
self.received.append({"from": getattr(sender, "name", "?"), "message": message})

def generate_reply(self, messages: Any = None, sender: Any = None, **_kwargs: Any) -> Any:
return self._canned_reply

def execute_code_blocks(self, code_blocks: Any) -> Any:
return f"executed {len(code_blocks)} blocks"


def main() -> int:
print("=== AutoGen adapter sample (mocked LLM) ===")

stratix = _RecordingStratix()
sink = _PrintSink()

adapter = AutoGenAdapter(stratix=stratix, capture_config=CaptureConfig.full())
adapter.add_sink(sink)
adapter.connect()

assistant = _FakeAgent(
name="assistant",
system_message="You are a concise assistant.",
llm_config={"model": "gpt-4o-mini", "temperature": 0},
canned_reply="2 + 2 is 4.",
)
user = _FakeAgent(name="user", canned_reply="thanks")

instrument_agents(assistant, user, stratix=stratix)
adapter.connect_agents(assistant, user)

print("-- agent.send round-trip --")
user.send("What is 2 + 2?", recipient=assistant)

print("-- generate_reply --")
reply = assistant.generate_reply(
messages=[{"role": "user", "content": "What is 2 + 2?"}],
sender=user,
)
print(f" reply: {reply!r}")

print("-- execute_code_blocks --")
assistant.execute_code_blocks([("python", "print('hi')")])

print("-- conversation lifecycle --")
adapter.on_conversation_start(initiator=user, message="What is 2 + 2?")
adapter.on_conversation_end(final_message=reply, termination_reason="auto_reply")

adapter.disconnect()

print(f"\nTotal events captured by sink: {len(sink.events)}")
print(f"Total events captured by stratix client: {len(stratix.events)}")
print("Event types:")
for ev in stratix.events:
print(f" - {ev['event_type']}")

return 0


if __name__ == "__main__":
raise SystemExit(main())
18 changes: 18 additions & 0 deletions src/layerlens/instrument/adapters/frameworks/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
"""Framework adapters for the LayerLens Instrument layer.

Each framework adapter wraps an agent / chain framework's lifecycle to
intercept agent runs, model invocations, tool calls, state changes, and
handoffs, emitting events through the LayerLens telemetry pipeline.

Adapters available (loaded on demand via :class:`AdapterRegistry`):

* ``autogen`` — Microsoft AutoGen (group chat + lifecycle)

Other framework adapters (LangChain, LangGraph, CrewAI, Agentforce,
Langfuse, Semantic Kernel, OpenAI Agents, etc.) ship in sibling M2
fan-out PRs.

Importing this package does NOT import any framework SDK.
"""

from __future__ import annotations
56 changes: 56 additions & 0 deletions src/layerlens/instrument/adapters/frameworks/autogen/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
"""
LayerLens AutoGen Adapter

Integrates LayerLens tracing with the Microsoft AutoGen framework.

Usage:
from layerlens.instrument.adapters.frameworks.autogen import (
AutoGenAdapter,
instrument_agents,
GroupChatTracer,
HumanProxyTracer,
)

adapter = AutoGenAdapter(stratix=stratix_instance)
adapter.connect()
adapter.connect_agents(agent1, agent2)
"""

from __future__ import annotations

from typing import Any

from layerlens.instrument.adapters.frameworks.autogen.groupchat import GroupChatTracer
from layerlens.instrument.adapters.frameworks.autogen.lifecycle import AutoGenAdapter
from layerlens.instrument.adapters.frameworks.autogen.human_proxy import HumanProxyTracer

# Registry lazy-loading convention
ADAPTER_CLASS = AutoGenAdapter


def instrument_agents(
*agents: Any, stratix: Any = None, capture_config: dict[str, Any] | None = None
) -> Any:
"""
Convenience function to instrument AutoGen agents with LayerLens tracing.

Args:
*agents: AutoGen ConversableAgent instances
stratix: LayerLens SDK / client instance to receive emitted events
capture_config: CaptureConfig to use

Returns:
List of instrumented agents
"""
adapter = AutoGenAdapter(stratix=stratix, capture_config=capture_config) # type: ignore[arg-type]
adapter.connect()
return adapter.connect_agents(*agents)


__all__ = [
"AutoGenAdapter",
"GroupChatTracer",
"HumanProxyTracer",
"instrument_agents",
"ADAPTER_CLASS",
]
Loading