feat(instrument): Typed Pydantic event foundation + agno reference (1/17 adapters)#129
Closed
mmercuri wants to merge 4 commits into
Closed
Conversation
Bootstraps the LayerLens instrument layer with the abstract base classes,
adapter registry, capture configuration, event sinks, vendored event
schemas, and pydantic v1/v2 compatibility shim that every concrete
adapter (frameworks, protocols, providers) will depend on.
Scope
-----
- src/layerlens/instrument/__init__.py: lean re-export surface
- src/layerlens/instrument/_vendored/: frozen ateam event schemas (no
runtime ateam dependency)
- src/layerlens/instrument/adapters/_base/: BaseAdapter, AdapterRegistry,
AdapterStatus, AdapterHealth, AdapterCapability, ReplayableTrace,
CaptureConfig, EventSink, TraceStoreSink, IngestionPipelineSink,
PydanticCompat
- src/layerlens/_compat/pydantic.py: model_dump/model_validate shim
spanning pydantic v1 + v2
- scripts/{port_adapter,port_protocol,emit_adapter_manifest,
regen_dep_baselines}.py: codegen helpers used to port the rest of M1
- tests/instrument/{test_base_layer,test_lazy_imports,
test_default_install,test_resolved_dep_tree}.py + _baselines/
- .github/workflows/dep-tree-guard.yaml: CI gate that locks the default
install footprint
- docs/adapters/: CONTRIBUTING, STATUS, pydantic-compatibility, testing,
PERSONA_REVIEW
Blast radius
------------
- Pure additions. No public surface changes outside the new
layerlens.instrument namespace.
- Default `pip install layerlens` install set is unchanged (verified by
test_default_install.py against the new baseline).
- Lazy adapter discovery: importing layerlens.instrument MUST NOT pull
in any optional adapter dep (verified by test_lazy_imports.py).
Test plan
---------
- uv run pytest tests/instrument/test_base_layer.py
tests/instrument/test_lazy_imports.py -x -> 45 passed
- The dep-tree-guard workflow exercises test_default_install.py and
test_resolved_dep_tree.py against the new baselines on every PR.
LAY-3400 umbrella: this PR is the prerequisite for the M1.B/M1.C/M1.D
adapter ports, M7 protocol certification, and M8 Cohere/Mistral.
Auto-fixed by 'ruff check --fix'. No behavior change.
…base Brings in the foundational instrument package (capture, registry, pydantic_compat, _vendored event models, _compat/pydantic shim) so this branch can build/test in isolation. PR #118 (feat/instrument-multitenancy-org-id-propagation) modifies _base files but does not include the foundation modules they import — a stacked-PR situation. Merging base-foundation here resolves the import chain and produces a self-contained branch. Conflict resolution used -X ours so the org_id propagation changes from PR #118 are preserved on every conflict. No functional code authored in this commit — pure dependency merge.
…/17)
Establishes the typed-event foundation for the instrument layer and
ports the agno framework adapter to the new emission contract as the
reference implementation. 16 framework adapters + provider/protocol
adapters remain on the legacy emit_dict_event path; see
docs/adapters/typed-events-followups.md for the per-adapter backlog.
Foundation changes
------------------
* New module src/layerlens/instrument/_compat/events.py vendors the
canonical event payload models from
ateam/stratix/core/events/{l1_io,l3_model,l4_environment,l5_tools,
cross_cutting}.py through layerlens.instrument._vendored. Exposes
ALL_TYPED_EVENTS registry (12 canonical event types), BaseEvent
Protocol, and validate_typed_event() / coerce_to_dict() helpers.
* New TypedEventValidationError raises when payloads fail canonical
schema validation. The validator REJECTS malformed inputs rather
than silently emitting them — CLAUDE.md "never silently skip
failing operations" applied to the emission boundary.
* BaseAdapter gains a per-adapter ALLOW_UNREGISTERED_EVENTS class
attribute (default False). Strict adapters reject unknown event
types; importer-style adapters (langfuse, benchmark_import) opt
into True for source data whose taxonomy genuinely diverges from
the canonical schema.
Dual-path emission contract
---------------------------
* emit_event(typed_payload) is the preferred path. It runs canonical
schema validation through validate_typed_event() before forwarding
the payload to the stratix client. Validation failures raise
TypedEventValidationError AND increment the circuit-breaker error
count (so persistent validation bugs eventually trip the breaker).
* emit_dict_event(event_type, dict) is the legacy path. It now emits
a DeprecationWarning on every call. It does NOT run canonical
validation — the 16 unmigrated adapters use adapter-specific dict
shapes that intentionally diverge from the canonical schema; the
warning is what keeps the gap visible in CI until each adapter
migrates. The org_id stamp still runs on this path so
multi-tenant scoping is preserved during the transition.
agno reference migration
------------------------
All 11 emit_dict_event call sites in
src/layerlens/instrument/adapters/frameworks/agno/lifecycle.py
replaced with emit_event(typed_payload) calls. Verified with
`grep -c emit_dict_event src/.../agno/` returning 0. Agno-specific
provenance fields (framework, agent_name, timestamp_ns) moved into
the canonical models' metadata / attributes / parameters slots.
The previously-emitted agent.state.change marker (which carried only
event_subtype with no real state hashes) is folded into the
AgentOutputEvent.metadata.run_status field — the canonical
AgentStateChangeEvent requires sha256:<hex64> before/after hashes,
which agno cannot surface without upstream instrumentation.
AgentHandoffEvent emissions now always carry a sha256
handoff_context_hash via a new _sha256_of helper.
Tests
-----
* tests/instrument/adapters/_base/test_typed_events.py — 23 tests
covering registry contract (2), validate_typed_event behaviour (7),
coerce_to_dict (2), and dual-path emission via BaseAdapter (12).
* tests/instrument/adapters/frameworks/test_agno_adapter.py extended
with 4 typed-event regression tests
(test_agno_emits_typed_payloads_only,
test_agno_emit_does_not_warn_after_migration,
test_agno_typed_handoff_validates_canonical_hash,
test_agno_typed_emission_records_org_id) and existing assertions
updated to read from canonical payload paths.
* No regression in 112 tests across the 16 unmigrated framework
adapters (validated via pytest tests/instrument/adapters/frameworks/).
Honest disclosure (CLAUDE.md item 11)
-------------------------------------
This PR delivers 1/17 framework adapters migrated. The remaining 16
adapters still emit via emit_dict_event and trigger
DeprecationWarning on every call site. Each unmigrated adapter is
listed with current emit_dict_event site count in
docs/adapters/typed-events-followups.md, including the recommended
migration order. The legacy path will be removed in the next major
SDK release (2.0.0) once all adapters have migrated; CI should run
pytest -W error::DeprecationWarning against post-migration adapters
to enforce no new emit_dict calls slip in.
Acceptance
----------
* `uv run pytest tests/instrument/adapters/_base/test_typed_events.py`: 23/23 pass
* `uv run pytest tests/instrument/adapters/frameworks/test_agno_adapter.py`: 16/16 pass
* `uv run pytest tests/instrument/adapters/frameworks/`: 128/128 pass (no regression)
* `grep -c emit_dict_event src/layerlens/instrument/adapters/frameworks/agno/`: 0
* `mypy --strict src/layerlens/instrument/_compat/events.py src/layerlens/instrument/adapters/_base/adapter.py`: clean
* `ruff check`: clean across all modified + new files
7 tasks
This was referenced May 10, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Establishes the typed-event foundation for the instrument layer and ports the agno framework adapter to the new emission contract as the reference implementation. 16 framework adapters + protocol/provider adapters remain on the legacy
emit_dict_eventpath — seedocs/adapters/typed-events-followups.mdfor the per-adapter backlog. This is a stacked PR over PR #118 (multi-tenancy org_id propagation).Honest disclosure (CLAUDE.md item 11)
This PR delivers 1/17 framework adapters migrated. The remaining 16 still emit via
emit_dict_eventand triggerDeprecationWarningon every call site. The foundation is what unblocks per-adapter migration PRs to land in parallel without coordinating on the dict-vs-typed shape. The legacy path is preserved (with deprecation warning) so existing customer apps don't break mid-rollout. Each unmigrated adapter is listed with current site count + recommended migration order in the followups doc.What's in this PR
Foundation
src/layerlens/instrument/_compat/events.py— vendors canonicalstratix.core.eventspayload models behind a Pydantic v1/v2-compatible facade. ExposesALL_TYPED_EVENTSregistry (12 canonical event types),BaseEventProtocol,validate_typed_event()/coerce_to_dict()helpers, andTypedEventValidationError.ALLOW_UNREGISTERED_EVENTStoggle onBaseAdapter. DefaultFalse(strict). Importer-style adapters opt intoTruefor source data whose taxonomy diverges from the canonical schema.Dual-path emission contract
emit_event(typed_payload)— preferred. Runs canonical schema validation that REJECTS malformed inputs by raisingTypedEventValidationError. Validation failures also increment the circuit-breaker error counter (CLAUDE.md "never silently skip failing operations").emit_dict_event(event_type, dict)— legacy. EmitsDeprecationWarningon every call. Does NOT run canonical validation (the 16 unmigrated adapters use adapter-specific dict shapes that intentionally diverge — running the validator would reject 100% of unmigrated emissions). Multi-tenantorg_idstamp still runs.agno reference migration
agno/lifecycle.pymigrated fromemit_dict_event→emit_event. Verified bygrep -c "emit_dict_event(" src/.../agno/returning 0.metadata/attributes/parametersslots.agent.state.change(no real state hashes) folded intoAgentOutputEvent.metadata.run_status. The canonicalAgentStateChangeEventrequiressha256:<hex64>before/after hashes that agno cannot surface honestly._sha256_of(value)helper produces canonicalsha256:<hex64>strings forAgentHandoffEvent.handoff_context_hash.Tests
tests/instrument/adapters/_base/test_typed_events.py— 23 tests (registry contract 2, validate_typed_event behaviour 7, coerce_to_dict 2, dual-path emission 12).tests/instrument/adapters/frameworks/test_agno_adapter.pyextended with 4 typed-event regression tests; existing assertions updated to read canonical payload paths.Docs
docs/adapters/typed-events.md— full migration guide with per-adapter checklist and worked example.docs/adapters/typed-events-followups.md— per-adapter backlog with current emit_dict_event site counts (honest, vs the spec's optimistic projections).Stacking
feat/instrument-multitenancy-org-id-propagation(PR fix(instrument): Propagate org_id through all event emissions (multi-tenancy CLAUDE.md fix) #118)feat/instrument-base-foundation(PR instrument: base foundation (M1.A port) #93) — merged into this branch viachore(instrument): merge feat/instrument-base-foundation as workable basebecause PR fix(instrument): Propagate org_id through all event emissions (multi-tenancy CLAUDE.md fix) #118 modifies_base/*files but does not include the foundation modules they import (capture, registry, pydantic_compat, _vendored). The merge used-X oursso PR fix(instrument): Propagate org_id through all event emissions (multi-tenancy CLAUDE.md fix) #118's org_id propagation changes are preserved on every conflict.Acceptance
uv run pytest tests/instrument/adapters/_base/test_typed_events.py -xuv run pytest tests/instrument/adapters/frameworks/test_agno_adapter.py -xuv run pytest tests/instrument/adapters/frameworks/grep -c "emit_dict_event(" src/layerlens/instrument/adapters/frameworks/agno/mypy --strict src/.../events.py src/.../adapters/_base/adapter.pymypy --strict src/.../adapters/frameworks/agno/lifecycle.pyruff check(all modified + new files)Test plan
tests/instrument/adapters/_base/test_typed_events.py)tests/instrument/adapters/frameworks/test_agno_adapter.py)emit_dict_event(call sites in agnoevents.py,adapter.py,agno/lifecycle.py