Skip to content

fix(capture): correlate Stop turn meta with the flushed transcript record#325

Merged
efenocchi merged 1 commit into
mainfrom
fix/stop-capture-turn-meta-race
Jul 21, 2026
Merged

fix(capture): correlate Stop turn meta with the flushed transcript record#325
efenocchi merged 1 commit into
mainfrom
fix/stop-capture-turn-meta-race

Conversation

@efenocchi

@efenocchi efenocchi commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Problem

The Stop hook races the transcript writer: the turn's final assistant record is often not on disk when capture reads it. Observed in real interactive sessions (validated on this machine):

  • Turn 1 of every session captured model=null, token_usage=null even though the record landed ~ms later.
  • Worse, the scanner accepted ANY newest usage-bearing assistant record, so a turn could silently be credited with the PREVIOUS turn's tokens (off-by-one) — poisoning per-model rollups.

The #320 e2e could not catch this: it replays settled transcripts, never a mid-flush one.

Fix

parseClaudeTurnMetaLive (src/notifications/model-usage.ts):

  • Re-opens and re-scans the transcript across a bounded backoff (25/50/100ms — max +175ms in an async 30s hook, only on misses).
  • Correlates the newest text-bearing, non-sidechain assistant record with last_assistant_message: exact normalized match, or an anchored truncation prefix of at least 32 chars ("Done" can never match "Done — details…").
  • Never falls back to an older usage-bearing record; if the current turn's record doesn't materialize, the row is written unenriched with the reason logged. Empty last_assistant_message skips enrichment instead of bypassing correlation.
  • Sidechain records are skipped on main-session Stop and included on SubagentStop (agent_transcript_path).

scripts/trace-model-usage-e2e.mjs now replays each transcript's real final assistant text and selects transcripts with the same live semantics.

Validation (real harness, not just unit tests)

  • Real interactive claude session (PTY) with the fixed bundle via --plugin-dir, capture into a staging table: turn 1 row now carries model=claude-fable-5 + full token_usage (was null pre-fix on the same machine/flow); turn 2 carries its own distinct usage (23 vs 83 output tokens matching each reply's length) — no off-by-one.
  • scripts/trace-model-usage-e2e.mjs over real local transcripts: claude 3/3 models with tokens, codex/cursor/pi/openclaw unchanged, PASS.
  • Unit: 20 new tests simulating file growth between reads, partial trailing records, short-reply and unanchored-substring mismatches, textless trailing tool_use records, sidechain filtering. tsc clean, full suite green (pre-existing cli-bundle-runtime env flake aside).
  • codex review of the diff: initial round's 3 blockers (empty-message bypass, unanchored substring matching, e2e harness regression) all addressed as above.

Session Context

Session transcript

Summary by CodeRabbit

  • Bug Fixes
    • Improved Claude usage tracking to associate token and reasoning data with the correct assistant response.
    • Prevented stale or incomplete transcript data from being incorrectly attributed to a turn.
    • Capture continues safely without enrichment when transcript data is unavailable or incomplete.
    • Improved handling of main-session and subagent transcript records.
  • Tests
    • Added coverage for delayed, partial, missing, and sidechain transcript scenarios.

…cord

The Stop hook races the transcript writer: the turn's final assistant
record is often not flushed when capture reads the file, so the enrichment
either came back null (observed live: every first turn captured with null
model/token_usage) or - worse - silently attributed the PREVIOUS turn's
tokens to the current one, because the scanner accepted any usage-bearing
assistant record regardless of which turn it belonged to.

parseClaudeTurnMetaLive re-opens and re-scans the transcript across a
bounded backoff (25/50/100ms) and correlates the newest text-bearing,
non-sidechain assistant record with last_assistant_message (exact match,
or an anchored >=32-char truncation prefix). No fallback to older records:
if the current turn's record never materializes, the row is written
unenriched and the reason logged. An empty last_assistant_message skips
enrichment instead of bypassing the correlation.

The trace e2e harness now replays each transcript's REAL final assistant
text and selects transcripts with the same live semantics, instead of a
synthetic message that would never correlate.
@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9b542d4d-68a9-4b07-ae95-4552e82e0376

📥 Commits

Reviewing files that changed from the base of the PR and between 87c285d and 1c23a74.

📒 Files selected for processing (5)
  • scripts/trace-model-usage-e2e.mjs
  • src/hooks/capture.ts
  • src/notifications/model-usage.ts
  • tests/claude-code/capture-hook.test.ts
  • tests/shared/notifications-model-usage.test.ts

📝 Walkthrough

Walkthrough

Claude capture metadata now correlates hook messages with the newest relevant assistant transcript text. Live scanning retries during transcript writes, excludes or includes sidechains by context, and safely skips enrichment when no match appears. E2E selection and tests use the updated flow.

Changes

Claude live correlation

Layer / File(s) Summary
Transcript scanning and retry flow
src/notifications/model-usage.ts
Adds assistant content and sidechain data, text matching, bounded transcript rescans, and live metadata parsing.
Capture and E2E integration
src/hooks/capture.ts, scripts/trace-model-usage-e2e.mjs
Uses live parsing during capture and correlates E2E jobs with each transcript’s newest assistant text.
Correlation and asynchronous capture tests
tests/shared/notifications-model-usage.test.ts, tests/claude-code/capture-hook.test.ts
Covers matching, retries, sidechains, transcript races, and asynchronous hook assertions.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CaptureHook
  participant LiveParser
  participant TranscriptScanner
  participant ClaudeTranscript
  CaptureHook->>LiveParser: provide last_assistant_message and transcript path
  LiveParser->>TranscriptScanner: scan expected assistant text
  TranscriptScanner->>ClaudeTranscript: read transcript snapshot
  ClaudeTranscript-->>TranscriptScanner: matching metadata or retry
  TranscriptScanner-->>LiveParser: TraceModelMeta or no match
  LiveParser-->>CaptureHook: enrich event or continue without metadata
Loading

Possibly related PRs

Suggested reviewers: khustup2

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is detailed, but it does not follow the required template and omits Summary, Version Bump, and Test plan sections. Add the required Summary, Version Bump, and Test plan sections, including whether a version bump is needed and the local test checklist/results.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the capture/Stop transcript correlation fix and matches the main change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/stop-capture-turn-meta-race

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report

Scope: files changed in this PR. Enforced threshold: 90% per metric (per file via vitest.config.ts).

Status Category Percentage Covered / Total
🟢 Lines 97.28% (🎯 90%) 322 / 331
🟢 Statements 93.13% (🎯 90%) 393 / 422
🟢 Functions 100.00% (🎯 90%) 36 / 36
🔴 Branches 85.00% (🎯 90%) 289 / 340
File Coverage — 2 files changed
File Stmts Branches Functions Lines
src/hooks/capture.ts 🟢 94.3% 🔴 80.6% 🟢 100.0% 🟢 100.0%
src/notifications/model-usage.ts 🟢 92.8% 🔴 86.0% 🟢 100.0% 🟢 96.5%

Generated for commit 39e697d.

@efenocchi
efenocchi merged commit 18a7ebd into main Jul 21, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant