Skip to content

feat: Tiered history retrieval (hot/cold layer pattern) #595

@tamirdresher

Description

@tamirdresher

Feature Proposal: Tiered History Retrieval

Problem

Agent history.md files grow unbounded. Currently Scribe summarizes at 12KB, but agents read everything — no distinction between recent context and archived knowledge. This wastes context window tokens and slows agent startup.

Proposed Solution

Formalize a two-tier retrieval pattern inspired by ClawMongo's tiered projection model:

Hot layer (history.md): Last ~20 entries. Always read at spawn time. Fresh, actionable context.
Cold layer (history-archive.md): Summarized older entries. Read only when agent explicitly needs historical context.

Implementation

  1. Spawn prompt change: agents read history.md (hot) by default. Only read history-archive.md when the task references past decisions or old patterns.
  2. Scribe summarization: when history.md exceeds threshold, move old entries to archive with summaries (already partially implemented).
  3. Search cold: add a 'search archive' pattern agents can invoke when they need historical context.

Benefits

  • Reduced context window usage (~60% less tokens for history)
  • Faster agent startup
  • No external DB required — pure file-based

Prior Art

Inspired by ClawMongo's tiered retrieval model (https://clawmongo.site) which uses ID-only projections for 10x context reduction. Adapted for Squad's file-based architecture.

Metadata

Metadata

Assignees

No one assigned

    Labels

    go:needs-researchNeeds investigationpriority:p2Next sprintsquadSquad triage inbox — Lead will assign to a membersquad:dskyAssigned to DSKY (TUI Engineer)squad:proceduresAssigned to Procedures (Prompt Engineer)type:featureNew capability

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions