summary-compression.md

Summary Compression Memory

Summarize older conversation turns into a compact representation. Keeps the essence of earlier context without consuming the full token budget.

How It Works

Window 1: Full conversation turns 1-10
Summary 1: "User asked about pricing, discussed enterprise plan, requested demo"

Window 2: Summary 1 + full turns 11-20
Summary 2: "User evaluated pricing, discussed enterprise plan, requested demo. 
            Then asked about security compliance, shared their SOC2 requirements."

Window 3: Summary 2 + full turns 21-30

Implementation

class SummaryCompression:
    def __init__(self, max_recent=5, summary_prompt=None):
        self.max_recent = max_recent
        self.summary = ""
        self.recent = []
        self.summary_prompt = summary_prompt or (
            "Summarize the key information from this conversation so far. "
            "Include: decisions made, user preferences, action items, "
            "and any facts the agent needs to remember."
        )

    def add_turn(self, turn, llm_summarize_fn):
        self.recent.append(turn)
        if len(self.recent) > self.max_recent:
            old = self.recent.pop(0)
            self.summary = llm_summarize_fn(
                f"{self.summary}\n{old}", self.summary_prompt
            )

    def get_context(self):
        parts = []
        if self.summary:
            parts.append(f"[Summary of earlier conversation]\n{self.summary}")
        parts.append("[Recent conversation]")
        parts.extend(self.recent)
        return "\n".join(parts)

Prompt

## Conversation Summary
{summary of earlier conversation}

## Recent Messages
{last N turns}

Use the summary for historical context.
Use recent messages for the current interaction.

Pros & Cons

Pros: Token-efficient, preserves key information, extends effective context.

Cons: Lossy compression (details lost), requires LLM call per summary, summary drift over time.