Skip to content

Paginated history projection breaks after a growing rollout provider rewrite #24

Description

@Wangnov

Summary

With the latest Codex paginated-history format, a cold rollout rewrite that grows the first JSONL line invalidates the persisted byte-offset checkpoint.

codex-threadripper sync reports success and the thread can still resume, but Codex can no longer project newly appended turns into thread_history_1.sqlite.

Verified environment

  • Codex source: a30aee8d906c2ec4dfa07c794504d4ee7099f98a
  • codex-threadripper: v0.3.5
  • Isolated CODEX_HOME
  • Thread explicitly created with historyMode: paginated

Reproduction

  1. Create a paginated thread whose rollout provider is a short value such as L.
  2. Let Codex project the existing rollout:
    • rollout size: 22241
    • projection checkpoint: byte offset 22241, ordinal 3
  3. Make the rollout cold and run:
    codex-threadripper --provider openai sync
  4. The first line grows by 261 bytes and the rollout size becomes 22502, while the projection checkpoint remains 22241|3.
  5. Resume the thread and complete another turn.

Actual result

Codex logs projection failures:

expected value at line 1 column 1
expected ordinal 3, got 2

The runtime turn completes and is appended to the rollout, but it is not added to the paginated history projection.

Root cause

Codex now persists next_rollout_byte_offset in thread_history_1.sqlite. The cold rewrite path in src/rollout.rs replaces a shorter first line with a longer one, shifting every downstream byte offset without coordinating the projection checkpoint.

Acceptance criteria

  • Paginated threads must never be left with a stale rollout byte-offset checkpoint.
  • A provider sync that cannot safely preserve paginated history must fail closed for that thread instead of partially updating SQLite and rollout metadata.
  • Legacy history keeps the existing cold rewrite behavior.
  • In-place same-length rollout patches remain supported.
  • Regression tests cover short-to-long provider sync for both legacy and paginated threads.
  • cargo fmt --check, all Rust tests, and an end-to-end probe against the latest Codex pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions