fix(autorun): reconcile CLI Auto Run summary with cumulative session stats - #1284
fix(autorun): reconcile CLI Auto Run summary with cumulative session stats#1284pedramamini wants to merge 1 commit into
Conversation
…stats The CLI batch processor's Auto Run summary counters (tasks, tokens, cost, duration) live only in memory, so they reset when a run spans an app/CLI restart, resume, or mid-run kill. The final summary and JSONL complete event then undercount cumulative work. Reconcile the in-memory counters against persisted on-disk history before building the summary: reconstruct totals from task entries written after the last final 'Auto Run ...' summary and take Math.max with the live counters, so stats survive restarts without absorbing earlier completed runs on the same session. History-read failure is expected/recoverable: fall back to in-memory counters and warn. The renderer path already gained this behavior (batchFinalSummary.ts) after the batch-processor rewrite that stranded #735. Extract the pure aggregate/ merge logic into src/shared/autoRunHistoryReconciliation.ts so the CLI reuses the exact same battle-tested logic instead of duplicating it; batchFinalSummary re-exports it for existing renderer importers. Also stamp completedTaskCount on CLI per-task history entries so reconciliation counts checkboxes exactly. Reimplements #735 on current main. Closes #734.
📝 WalkthroughWalkthroughChangesAuto Run reconciliation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant runPlaybook
participant readHistory
participant autoRunHistoryReconciliation
participant JSONL
runPlaybook->>readHistory: Read persisted AUTO history
readHistory-->>runPlaybook: Return history entries
runPlaybook->>autoRunHistoryReconciliation: Aggregate and merge totals
autoRunHistoryReconciliation-->>runPlaybook: Return reconciled totals
runPlaybook->>JSONL: Emit reconciled complete event
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds cumulative history reconciliation to CLI Auto Run. The main changes are:
Confidence Score: 4/5The cumulative CLI totals can still be wrong on halted, non-looping, and legacy-history paths.
src/cli/services/batch-processor.ts and src/shared/autoRunHistoryReconciliation.ts Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CLI Auto Run] --> B[Persist task history]
B --> C{Completion path}
C -->|Normal| D[Read session history]
D --> E[Aggregate after final boundary]
E --> F[Merge with runtime totals]
F --> G[Write summary and complete event]
C -->|Halt marker| H[Return with runtime totals]
H --> I[Persisted totals omitted]
G --> J{Final boundary written?}
J -->|Non-loop run| K[No boundary]
K --> L[Next run absorbs prior task rows]
Reviews (1): Last reviewed commit: "fix(autorun): reconcile CLI Auto Run sum..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| const reconciled = reconcileTotals(); | ||
|
|
||
| // Add total Auto Run summary (only if looping was used) | ||
| createAutoRunSummary(); | ||
| createAutoRunSummary(reconciled); |
There was a problem hiding this comment.
When a halt marker is detected during a resumed run, the earlier halt branch emits process-local totals and returns before reaching this reconciliation. The complete event then omits work persisted before the restart, and no final Auto Run boundary is written, so a later run can absorb the halted run's task rows.
Context Used: CLAUDE.md (source)
| break; | ||
| } | ||
| } | ||
| const currentRunEntries = orderedEntries.slice(previousFinalSummaryIndex + 1); |
There was a problem hiding this comment.
A successful CLI run with looping disabled persists task rows but createAutoRunSummary skips its final summary when loopIteration === 0. On the next run for that session, this slice includes the already completed run, so the reconciled summary and JSONL event overcount its tasks, tokens, cost, and duration.
Context Used: CLAUDE.md (source)
| return taskEntries.reduce<AutoRunHistoryTotals>( | ||
| (totals, entry) => { | ||
| const usageStats = entry.usageStats; | ||
| totals.totalCompletedTasks += Math.max(0, entry.completedTaskCount ?? 1); |
There was a problem hiding this comment.
CLI history created before this change has no completedTaskCount, so every legacy agent turn contributes exactly one task here. If a turn checked multiple boxes, a resumed run undercounts the completed tasks even though the row's token, cost, and elapsed totals are fully included.
Context Used: CLAUDE.md (source)
There was a problem hiding this comment.
Pull request overview
This PR fixes a long-running CLI Auto Run accounting bug where the final summary and complete JSONL event could undercount cumulative work when an Auto Run spans restarts or resumes. It does this by reconciling the CLI's in-memory counters against persisted on-disk history using a shared, pure reconciliation module that is also reused by the renderer path.
Changes:
- Added shared Auto Run history aggregation and merge helpers to reconcile runtime totals with persisted history totals.
- Updated the renderer batch final summary module to import the shared helpers and re-export them to avoid changing existing renderer import sites.
- Updated the CLI batch processor to (1) stamp
completedTaskCountper task entry and (2) reconcile totals before writing the final summary and emitting thecompleteevent, with a warn-and-fallback path on history read failure.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/shared/autoRunHistoryReconciliation.ts | New shared pure helpers to aggregate Auto Run totals from persisted history and merge with runtime counters. |
| src/renderer/hooks/batch/internal/batchFinalSummary.ts | Switched renderer to use the shared reconciliation helpers, while re-exporting to preserve existing imports. |
| src/cli/services/batch-processor.ts | Reconciles CLI summary and complete event totals against persisted history; stamps completedTaskCount for accurate cross-restart task counting. |
| src/tests/cli/services/batch-processor.test.ts | Adds CLI tests covering restart reconciliation, no double count across completed runs, no undercount when history lags, and fallback when history read throws. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Summary
Fresh reimplementation of #735 on latest
main. The original PR became unrebasable after the batch-processor rewrite (src/cli/services/batch-processor.ts+688/-573 and theuseBatchProcessor.tssplit +140/-1810). Closes #734.Bug still reproduces (verified on current main)
The renderer path already gained history reconciliation during the rewrite (
src/renderer/hooks/batch/internal/batchFinalSummary.tsviaaggregateAutoRunHistoryTotals+mergeFinalSummaryTotals), so the desktop half of #735 is already covered by a more robust implementation.The CLI path did NOT. On current main,
createAutoRunSummaryinsrc/cli/services/batch-processor.tsand thecompleteJSONL event were built purely from in-memory counters (totalCompletedTasks,totalInputTokens,totalOutputTokens,totalCost,Date.now() - batchStartTime). Those reset when a run spans an app/CLI restart, a resume, or a mid-run kill, so the final summary and thecompleteevent undercount cumulative work. That is the remaining live half of the bug.Fix
Reconcile the in-memory counters with persisted on-disk history before writing the summary and emitting the
completeevent:Auto Run ...summary (so it spans restarts but does not absorb earlier completed runs on the same session), and takeMath.maxof (in-memory, history-derived).logger.warn(does not bubble to Sentry).To avoid duplicating the renderer's battle-tested logic (per CLAUDE.md dedup rules), the pure aggregate/merge helpers were extracted from
batchFinalSummary.tsinto a new shared modulesrc/shared/autoRunHistoryReconciliation.ts. Both the renderer and the CLI now import it;batchFinalSummary.tsre-exports it so existing renderer importers and tests are unchanged. CLI per-task history entries now also stampcompletedTaskCountso the shared aggregation counts checkboxes exactly (matching desktop Auto Run), rather than approximating one task per run entry.Design deviations from #735
completedTaskCounton CLI task entries for exact cross-restart counts (a small accuracy improvement fix: reconcile Auto Run summary with cumulative session stats #735 lacked).Files changed
src/shared/autoRunHistoryReconciliation.ts(new) - shared pure reconciliation helperssrc/renderer/hooks/batch/internal/batchFinalSummary.ts- moved helpers to shared, re-exportsrc/cli/services/batch-processor.ts- reconcile before summary/complete; stampcompletedTaskCountsrc/__tests__/cli/services/batch-processor.test.ts- restart-reconciliation, no-double-count, no-undercount, read-failure-fallbackValidation
tscclean on all four configs:tsconfig.main.json,tsconfig.json,tsconfig.cli.json,tsconfig.lint.jsonReady for review. Do not merge.
Summary by CodeRabbit