Engram (AE) ⚡ — Andy's Edition

Graph memory for OpenClaw agents.
Fork of Atomlaunch/engram with Kuzu backend (embedded), extraction policy enforcement, CLI tooling, and automated pipeline.

Engram (AE) is a temporal knowledge graph memory system for OpenClaw agents. It extracts entities, facts, relationships, and emotions from conversations and daily logs, stores them in a Kuzu embedded graph database, and injects relevant context into every agent turn.

What Makes AE Different

This fork diverges significantly from upstream Engram:

Extraction Policy

The core difference. AE enforces a strict extraction policy (EXTRACTION_POLICY.md) that ensures only durable, actionable knowledge enters the graph.

Store by default: Decisions, milestones, agent outcomes, todos, stable relationships, preferences, operating rules.

Never store: Reasoning/chain-of-thought, secrets/tokens, heartbeat wrappers, casual chatter, routine success spam.

Every fact passes a 5-gate pre-store test:

1. Durable next week?
2. Actionable or explanatory?
3. Specific enough to retrieve?
4. Safe (no secrets)?
5. Novel (not duplicate)?

Result: 90% noise reduction vs. unfiltered ingestion (3,590 → 354 nodes on identical source data).

Extraction Model

AE uses xAI grok-4-1-fast-non-reasoning exclusively for extraction. No Ollama, no local models. Local models lack the judgment needed for policy enforcement. Non-reasoning mode is used because extraction doesn't need chain-of-thought — it needs good judgment on a clear rubric.

Security

13 compiled secret-detection patterns catch and redact sensitive content before it reaches the LLM or the graph:

• API keys (sk-, ghp_, gho_, github_pat_, xai-, AKIA)
• Auth tokens (Bearer, quoted passwords/tokens)
• PEM private keys
• Content is stripped pre-extraction AND facts are re-scanned pre-storage (defense-in-depth)

AE-Only Components

• Engram CLI (cli.py) — 14 commands: search, entity, timeline, agent-history, facts, stats, briefing, health, dispatch, todos, todo-add, todo-done, contradictions, recent
• Batch extractor (batch_extract.py) — Catches missed messages from session logs
• Importance scoring — LLM assigns high/medium labels, converted to numeric scores with category bonuses
• Delta briefings — Show only what's new since your last session ("What did I miss?")
• Pre-dispatch context — Auto-pull agent + project history for agent prompts
• Todo tracking — Extract, manage, and auto-resolve todos as first-class facts
• Contradiction detection — Detect and auto-supersede stale/conflicting facts
• Temporal queries — --since / --until filtering on all search and fact commands

Modified from Upstream

• Dream consolidation (consolidate.py) — Reduced decay from 1%/day to 0.2%/day. Safe because extraction policy already filters noise at ingest time.
• Assembly cache — Cherry-picked from upstream v0.2.0. Session-scoped caching (3-min TTL for queries, 10-min for pinned facts).

Shared with Upstream

• Kuzu backend (embedded) — Zero-ops local graph DB, no server required
• Channel-scoped pinned context — Inject standing rules per-channel or per-session
• Briefings (briefing.py) — Generate session briefings from graph state
• Dashboard (dashboard/) — Sigma.js graph visualization (optional)
• Dream consolidation (consolidate.py) — Core consolidation pipeline (AE modifies the decay rate)

Architecture

  ┌─────────────────────────────────────────────────────────┐
  │                    Live Pipeline                         │
  │                                                          │
  │  User message → afterTurn hook → Regex + LLM extract     │
  │                                    ↓                      │
  │                         Pre-store test (5 gates)          │
  │                                    ↓                      │
  │                        Kuzu Graph DB (embedded)            │
  │                                    ↑                      │
  │  Agent turn  ← assemble hook ← Context query (cached)     │
  └─────────────────────────────────────────────────────────┘

  ┌─────────────────────────────────────────────────────────┐
  │                  Background Pipeline                     │
  │                                                          │
  │  Daily logs → ingest.py → grok-4-1-fast-non-reasoning    │
  │                             ↓                             │
  │                    Secret redaction + policy filter        │
  │                             ↓                             │
  │                     Kuzu Graph DB (embedded)               │
  │                                                          │
  │  Session JSONL → batch_extract.py (every 30 min)         │
  │  Graph DB → consolidate.py (daily)                       │
  │    → gentle decay, centrality boost, dedup               │
  └─────────────────────────────────────────────────────────┘

Components

Component	Description
cli.py	CLI — 14 commands including search, dispatch, todos, contradictions, recent
ingest.py	Bulk ingest — parallel LLM extraction with policy enforcement
context_query.py	Context queries — semantic + graph search, live extraction (regex + LLM)
batch_extract.py	Batch extractor — scans session logs, re-extracts missed messages
consolidate.py	Dream consolidation — gentle decay, centrality boost, dedup
schema_neo4j.py	Neo4j schema (fallback) — node/relationship definitions
briefing.py	Briefings — full + delta briefings from graph state
dispatch_context.py	Pre-dispatch — agent + project context for prompt building
todos.py	Todos — track, add, resolve todos as first-class facts
contradictions.py	Contradictions — detect and supersede stale/conflicting facts
EXTRACTION_POLICY.md	Policy — canonical reference for what gets stored, skipped, and scored
extensions/context-engine/	OpenClaw plugin — assemble() + afterTurn() hooks with assembly cache
dashboard/	Dashboard — Sigma.js graph visualization (optional)

Quick Start

1. Python Dependencies (Kuzu)

python3 -m venv .venv-memory
source .venv-memory/bin/activate
pip install -r requirements.txt
pip install kuzu

2. Configuration

cp config.json.example config.json
# Edit config.json:
#   backend: "kuzu"
#   xai_api_key: "your-xai-key"

3. Initial Ingest

# Run bulk ingest with parallel workers
python ingest.py --workers 6

# Check graph stats
python cli.py stats

# Check graph health
python cli.py health

4. Context Engine Plugin (OpenClaw)

Copy or symlink extensions/context-engine/ into your OpenClaw extensions directory, then configure the plugin in your OpenClaw config. See SKILL.md for detailed setup.

CLI Usage

# Core queries
engram search "CalCity Stripe"                  # Search everything
engram search "deploy" --since 2026-03-28       # Time-filtered search
engram entity "Woody"                           # Full context for an entity
engram timeline "CalCity" --days 30             # Project/entity timeline
engram agent-history "Buzz" --project CalCity   # What an agent built
engram facts --recent --days 7                  # Recent facts
engram facts --since 2026-03-27 --until 2026-03-29  # Date-range facts

# Session startup
engram briefing --delta                         # What's new since last session
engram briefing --save                          # Full briefing, save to BRIEFING.md
engram recent --hours 48                        # All activity in last N hours

# Agent dispatch
engram dispatch Buzz --project CalCity          # Pre-dispatch context for prompts

# Todo tracking
engram todos                                    # List open todos
engram todos --all                              # Include resolved
engram todo-add "Deploy Rosamond.info"          # Add a todo
engram todo-done fact_abc123                    # Resolve a todo

# Memory health
engram contradictions --days 7                  # Recent superseded facts
engram stats                                    # Graph statistics
engram health                                   # Graph health check

All commands support --json for structured output.

Extraction Policy

The extraction policy is the heart of AE. See EXTRACTION_POLICY.md for the full spec.

Goal: Engram should answer:

• What changed?
• Why did we decide that?
• Who worked on it?
• What's still pending?

Not: every intermediate thought or cron wrapper.

Importance scoring:

Level	Score	What qualifies
High	0.80–1.0	Decisions, milestones, durable preferences, major lessons
Medium	0.50–0.70	Agent outcomes, meaningful run summaries, useful todos
Low	Skip	Transient status, repetitive output, scratch text

Dream Consolidation

Runs daily. Maintains graph health:

• Gentle importance decay — 0.2%/day (~0.998^days). Facts stay near full strength for months, gently recede over a year.
• Centrality boost — highly-connected entities get importance bumps
• Deduplication — merge near-duplicate facts
• Relationship strengthening — reinforce frequently co-occurring entity links
• Emotional pattern tracking — track frustration, satisfaction, and other patterns over time

Schema

Nodes: Entity, Fact, Episode, Emotion, SessionState

Relationships: RELATES_TO, CAUSED, PART_OF, MENTIONED_IN, EPISODE_EVOKES, ENTITY_EVOKES, DERIVED_FROM, ABOUT

Every node carries an agent_id field for multi-agent memory isolation.

Requirements

• Python 3.10+
• Kuzu (pip package)
• xAI API key (for grok-4-1-fast-non-reasoning extraction)
• OpenClaw (for context engine plugin)
• Node.js 18+ (dashboard only)

Upstream

Fork of Atomlaunch/engram. Upstream uses Kuzu as the default graph backend and Ollama for extraction. AE uses Kuzu (embedded) and xAI exclusively.

License

MIT