Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,3 @@ Brewfile.lock.json
.DS_Store
.coverage
docs/review/
marc-only/
2 changes: 1 addition & 1 deletion docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
* [LLM Provider Integrations](samples-guide.md#llm-provider-integrations-2-samples) -- OpenAI, Anthropic
* [OpenClaw Agent Evaluation](samples-guide.md#openclaw-agent-evaluation-10-demos--skill) -- Cage match, code gate, safety audit, red-team
* [MCP Server](samples-guide.md#mcp-server-1-sample) -- LayerLens as tools for Claude and other MCP clients
* [CopilotKit Integration](samples-guide.md#copilotkit-integration-2-agents--ui-components) -- LangGraph CoAgents, React components
* [CopilotKit Integration](samples-guide.md#copilotkit-integration) -- Next.js canvas + chat with create_agent and frontend HITL
* [Claude Code Skills](samples-guide.md#claude-code-skills-6-skills) -- Slash commands for CLI and desktop

## Troubleshooting
Expand Down
90 changes: 0 additions & 90 deletions docs/review/round-1.md

This file was deleted.

72 changes: 0 additions & 72 deletions docs/review/round-2.md

This file was deleted.

14 changes: 7 additions & 7 deletions docs/samples-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,16 +88,16 @@ Located in [`samples/mcp/`](../samples/mcp/). Expose LayerLens capabilities as t

See the [MCP README](../samples/mcp/README.md) for setup instructions.

### CopilotKit Integration (2 agents + UI components)
### CopilotKit Integration

Located in [`samples/copilotkit/`](../samples/copilotkit/). Full-stack integration with CopilotKit using LangGraph CoAgents and generative UI card components.
Located in [`samples/copilotkit/`](../samples/copilotkit/). A full-stack canvas + chat sample built on `langchain.agents.create_agent` + `CopilotKitMiddleware`, with a runnable Next.js 16 + Tailwind 4 + shadcn/ui demo app under `app/`. The pattern mirrors CopilotKit's own [`coagents-research-canvas`](https://github.com/CopilotKit/CopilotKit/tree/main/examples/v1/research-canvas) reference: state-driven cards on the host page, a chat sidebar with a frontend HITL widget, and out-of-band polling for long-running async work.

- [`agents/evaluator_agent.py`](../samples/copilotkit/agents/evaluator_agent.py) -- LangGraph CoAgent for evaluation workflows (human-in-the-loop judge confirmation via `interrupt()`)
- [`agents/investigator_agent.py`](../samples/copilotkit/agents/investigator_agent.py) -- LangGraph CoAgent for trace investigation
- [`components/*.tsx`](../samples/copilotkit/components/) -- React card components for rendering results
- [`hooks/*.ts`](../samples/copilotkit/hooks/) -- CopilotKit hooks for wiring LayerLens actions
- [`agents/evaluator_agent.py`](../samples/copilotkit/agents/evaluator_agent.py) -- LangGraph agent with four backend tools (`list_recent_traces`, `list_judges`, `run_trace_evaluation`, `get_evaluation_result`) and a frontend HITL tool (`confirm_judge`) for picking which judge to apply. The picker is a real React widget registered via `useCopilotAction({ renderAndWaitForResponse })`, bridged into the LLM's toolbelt by `CopilotKitMiddleware` -- no `interrupt()` call.
- [`agents/investigator_agent.py`](../samples/copilotkit/agents/investigator_agent.py) -- Standalone procedural `StateGraph` for trace investigation (errors / latency / cost hot spots). No HITL, no LLM. Reference for non-conversational agents.
- [`components/*.tsx`](../samples/copilotkit/components/) -- Five reusable SDK card components (`EvaluationCard`, `TraceCard`, `JudgeVerdictCard`, `MetricCard`, `ComplianceCard`) plus `MarkdownLite`, re-exported as `@layerlens/copilotkit-cards`.
- [`app/`](../samples/copilotkit/app/) -- Runnable Next.js + FastAPI demo. Real LayerLens only -- a missing `LAYERLENS_STRATIX_API_KEY` is a hard error at startup.

> **Checkpointer note:** Any LangGraph CoAgent that calls `interrupt()` (such as `evaluator_agent.py`) **must** be compiled with a checkpointer. Without one, the AG-UI stream ends without emitting `RUN_FINISHED` and CopilotKit blocks all subsequent messages. The sample ships with `InMemorySaver` for a zero-setup local run and documents Postgres / SQLite / Redis / LangGraph Platform alternatives for production in its [README](../samples/copilotkit/README.md#human-in-the-loop-checkpointers).
> **Checkpointer note:** The evaluator graph is compiled with `InMemorySaver` so `ag_ui_langgraph`'s endpoint can call `graph.aget_state(config)` per request -- without it the AG-UI handler errors with "No checkpointer set" before any tool runs. The sample ships `InMemorySaver` for zero-setup local development; production deployments should swap to a durable saver (Postgres / SQLite / Redis / LangGraph Platform). See the sample's [README](../samples/copilotkit/README.md) for the full architecture walkthrough.

See the [CopilotKit README](../samples/copilotkit/README.md) for the full list.

Expand Down
21 changes: 21 additions & 0 deletions tests/test_samples.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,32 @@
# Directories containing library/support modules (not standalone samples)
_LIBRARY_DIRS = {"judges", "lib", "components", "hooks"}

# Directories to skip entirely during sample discovery. The CopilotKit
# sample ships a Next.js app under ``app/frontend``; once a developer runs
# ``npm install`` there, dependencies like ``katex`` drop their own .py
# helper scripts into ``node_modules`` -- those are not LayerLens samples
# and must not be treated as such.
_SKIP_DIRS = {
"node_modules",
".next",
"__pycache__",
".venv",
"venv",
"dist",
"build",
".pytest_cache",
"test-results",
"playwright-report",
}


def _collect_samples():
"""Collect all sample .py files, excluding helpers and __init__."""
samples = []
for root, dirs, files in os.walk(SAMPLES_DIR):
# Mutate ``dirs`` in place so ``os.walk`` does not descend into
# build artefacts, virtualenvs, or vendored packages.
dirs[:] = [d for d in dirs if d not in _SKIP_DIRS]
for f in files:
if f.endswith(".py") and not f.startswith("_"):
rel = os.path.relpath(os.path.join(root, f), SAMPLES_DIR)
Expand Down
Loading