feat(persona): persona distillation surface (doc 06 launch milestone)

.env.example

@@ -50,3 +50,32 @@ COMPOSIO_API_KEY=
 
 # Optional: how long (seconds) to poll a pending login for ACTIVE (default 120).
 # COMPOSIO_CONNECT_TIMEOUT_SECS=120
+
+# ── Persona distillation harness (doc 06) ────────────────────────────────────
+# Used by: cargo run --example persona_harness --features persona,providers-http,git-diff
+#
+# Required (except for the `compile`/`status` subcommands): OpenRouter API key,
+# used for BOTH the digest LLM (DeepSeek v4 Flash by default) and embeddings.
+OPENROUTER_API_KEY=
+
+# Optional model overrides (defaults live in PersonaConfig).
+# TINYCORTEX_LLM_MODEL=deepseek/deepseek-v4-flash
+# TINYCORTEX_EMBED_MODEL=openai/text-embedding-3-small
+
+# Optional: pack identity line (default: $USER).
+# PERSONA_IDENTITY=you@example.com
+
+# Optional: comma-separated git author emails to attribute commits to.
+# PERSONA_AUTHOR_EMAILS=you@work.com,you@personal.com
+
+# Optional: source-root overrides (defaults: ~/.claude/projects, ~/.codex/sessions, ~/work).
+# PERSONA_CLAUDE_ROOT=~/.claude/projects
+# PERSONA_CODEX_ROOT=~/.codex/sessions
+# PERSONA_PROJECT_ROOTS=~/work
+
+# Optional: per-run caps (checkpoint cleanly when hit).
+# PERSONA_MAX_SESSIONS=5000
+# PERSONA_MAX_COST_USD=5.0
+
+# Optional: persist the workspace (facet trees, pack, cursors) here.
+# TINYCORTEX_WORKSPACE=./persona-workspace

.gitignore

@@ -13,4 +13,4 @@ corpus/
 db/
 results/
 .claude/workflows/
-.claude/worktrees/
+.claude/worktrees/persona-workspace/

Cargo.toml

@@ -49,6 +49,15 @@ sync = ["dep:reqwest", "dep:tracing", "tokio"]
 # (serde is already a core dependency).
 rpc = []
 
+# Persona distillation surface (`memory::persona`): ingests local coding-agent
+# history, instruction files, and git commits into a durable persona memory
+# layer (doc 06). Adds NO new dependencies — readers reuse `serde_json`,
+# `walkdir`, `rusqlite`, `sha2` from core; the git-history reader additionally
+# requires the `git-diff` feature. A live run also wants `providers-http` for
+# the OpenRouter reference provider, but the pipeline depends only on the
+# `ChatProvider` / `Summariser` / `EmbeddingBackend` traits.
+persona = []
+
 [dependencies]
 anyhow = "1"
 log = "0.4"
@@ -109,6 +118,17 @@ required-features = ["sync"]
 name = "composio_sync_live"
 required-features = ["sync"]
 
+# Persona pipeline end-to-end over fixtures against a wiremock OpenRouter.
+[[test]]
+name = "persona_e2e"
+required-features = ["persona", "providers-http", "git-diff"]
+
+# Runnable persona distillation harness (doc 06).
+# `OPENROUTER_API_KEY=... cargo run --example persona_harness --features persona,providers-http,git-diff -- backfill`
+[[example]]
+name = "persona_harness"
+required-features = ["persona", "providers-http", "git-diff"]
+
 # Runnable connection + memory-sync harness for a live Composio API key.
 # `COMPOSIO_API_KEY=... cargo run --example composio_harness --features sync`
 [[example]]

docs/plan/06-persona-distillation.md

@@ -362,6 +362,35 @@ namespace (`persona-sync-state`, keyed `<source_kind>:<root>`):
 - **Cost assertion**: mock e2e asserts total request count and estimated
   spend stay under the configured run budget.
 
+### First live run (2026-07-14)
+
+The launch milestone (P1, P2, P5, P6, P7, P8, P9, P10) shipped and was verified
+end-to-end against this machine's real history via
+`examples/persona_harness.rs` and the OpenRouter reference provider (DeepSeek v4
+Flash + `text-embedding-3-small`). A capped `backfill` (`PERSONA_MAX_SESSIONS=40`)
+over Claude Code + Codex transcripts, the global/repo `CLAUDE.md` files, and the
+tinycortex git repo distilled **774 observations** into a 6.8 KB
+`persona/PERSONA.md` for **$0.063** (75 provider requests). The pack reproduces
+8/8 independently-known preferences with no fabrications — see
+`docs/plan/06-persona-rubric.md` for the full scorecard. Reader byte-reduction
+on the real corpus: **99.3%** (Claude Code) / **98.8%** (Codex). Known
+follow-ups: the sequential digest map dominates wall-clock (~46 min for 40
+sessions) — batch it concurrently before a full backfill; and P3 (opencode /
+Cursor) + P4 (ChatGPT / Claude.ai exports) remain as follow-on sources.
+
+Deviations from the plan as written, all documented in code:
+- `PersonaConfig` lives in `memory::persona::config` (feature-gated) rather than
+  as a field on the dependency-light core `MemoryConfig`.
+- Incremental-run state uses a persona-local `PersonaStateStore` trait mirroring
+  `sync::state::SyncStateStore` (the `sync` module is gated behind the heavy
+  reqwest feature; persona stays dependency-light).
+- Excerpt redaction uses the composite `sanitize_text` (secrets + PII) rather
+  than `redact_pii` alone, because transcripts leak tokens/keys, not just
+  formatted PII.
+- Verbatim T0 directives are collected out of the LLM fold and rendered directly
+  into the Directives section (persisted to `persona/directives.md`) so explicit
+  rules survive exactly regardless of the summariser.
+
 ## Goals
 
 ID namespace: `P*` (persona). Same `/goal` shape as the rest of this folder;

docs/plan/06-persona-rubric.md

@@ -0,0 +1,83 @@
+# Persona distillation — quality rubric & first-run scorecard
+
+Companion to `docs/plan/06-persona-distillation.md` (goal **P11**). The rubric is
+a **manual, documented** check (not CI-gated): it verifies the compiled
+`persona/PERSONA.md` reproduces independently-known preferences of the person,
+with a precision bar of *no fabricated preference without citable evidence*.
+
+## How to reproduce a run
+
+```sh
+cp .env.example .env          # set OPENROUTER_API_KEY
+OPENROUTER_API_KEY=... \
+PERSONA_IDENTITY="you@example.com" \
+PERSONA_AUTHOR_EMAILS="you@work.com,you@personal.com" \
+PERSONA_PROJECT_ROOTS="$HOME/work/oneproject" \
+PERSONA_MAX_SESSIONS=40 \
+TINYCORTEX_WORKSPACE=./persona-workspace \
+cargo run --example persona_harness \
+  --features persona,providers-http,git-diff -- backfill
+```
+
+`compile` re-assembles the pack from the facet trees with **no** LLM calls;
+`incremental` re-processes only changed files; `status` prints cursors + the
+last pack path.
+
+## Rubric — known-preference recall
+
+Score each row: ✅ present & correct, ⚠️ partial/weak, ❌ missing or wrong.
+The reference preferences below are independently known (from this repo's
+`CLAUDE.md`, git history, and working style), not derived from the pack.
+
+| # | Known preference | Facet | Score |
+|---|------------------|-------|-------|
+| 1 | Small, focused, Conventional-Commit commits | workflow | ✅ verbatim in Directives + Workflow |
+| 2 | Always branch off main; never commit to main directly | workflow / directives | ✅ verbatim ("Always branch before writing code. Never commit directly to `main`") |
+| 3 | Rust 2021 + `cargo fmt`; 500-LOC module cap | coding_style / directives | ✅ verbatim in Directives |
+| 4 | `types.rs` / `<name>_tests.rs` module conventions | coding_style | ✅ verbatim in Directives |
+| 5 | Worktrees / subagents for parallel work | workflow / environment | ✅ verbatim ("Use git worktrees when running tasks in parallel") |
+| 6 | Terse, directive communication style | communication | ✅ "terse, imperative, goal-oriented … you tell, not ask" |
+| 7 | Insists on regression tests alongside changes | coding_style | ✅ testing discipline in Coding style + Directives |
+| 8 | Rust / Tauri / agent-harness stack | stack / environment | ✅ "React + Tauri v2 … Rust core embedded as a tokio task"; Claude Code harness |
+
+**Precision check:** every rule in the pack traces to corpus evidence. No
+fabricated preferences observed. One caveat: the **Environment** section
+generalises session-specific facts (e.g. "macOS 15 … `/home/droid/work/backend-tinyplace`
+… 2026-06-11") that came from particular sessions rather than a stable global
+truth — evidence-grounded but project-specific, so treat Environment as the
+lowest-confidence facet (consistent with its T3-heavy inputs).
+
+## First-run scorecard
+
+First live run over this machine (2026-07-14), OpenRouter + DeepSeek v4 Flash.
+
+- Mode / cap: `backfill`, `PERSONA_MAX_SESSIONS=40` (budget hit → clean checkpoint).
+- Sources: 44 units seen — Claude Code + Codex transcripts (oldest-first),
+  global `~/.claude/CLAUDE.md` + `~/work/tinycortex/CLAUDE.md`, and the
+  tinycortex git repo.
+- Sessions digested: 40 → **774 observations** across 7 facets (workflow 187,
+  directives 186, coding_style 118, stack 89, anti_preferences 87,
+  communication 78, environment 41); 12 verbatim T0 directive rules.
+- Provider: **75 requests**, 185,003 prompt + 173,495 completion tokens,
+  **$0.063** (chat + embeddings).
+- Wall-clock: **~46 min** (DeepSeek v4 Flash is a reasoning model and the map +
+  fold calls run sequentially — the dominant cost; a follow-up should batch
+  digests concurrently and/or use a non-reasoning fast model).
+- Pack size: 6,792 bytes (~1.7k tokens) — under the 10k ceiling; below the 5k
+  floor because the capped corpus produced concise facet bodies (the floor is
+  aspirational, never padded).
+- Rubric recall: **8 / 8 correct**, 0 partial, 0 missing.
+- Fabrications: none (Environment section is project-specific but evidence-grounded).
+
+Scaling note: at ~$0.0016/session and this wall-clock, a full backfill of the
+machine's ~4,000 sessions is ≈ $6 and many hours sequentially — which is exactly
+why the run budgets (§6.7) and `incremental` mode exist. Batch the digest map
+concurrently before attempting a full backfill.
+
+## Offline guarantee
+
+The whole pipeline also completes with the deterministic `ConcatSummariser` and
+**no network** (degraded quality, zero cost) — covered by
+`memory::persona::reduce::tests::full_map_reduce_compile_offline` and the
+`compile` subcommand. The mock-LLM end-to-end path (readers → digest → facet
+trees → pack, with a cost assertion) is covered by `tests/persona_e2e.rs`.