feat(realtime): conversation compaction (summarize-then-drop) + OpenAI item.delete/truncate/clear

[localai-bot]

Summary

Adds server-side conversation compaction to the realtime (voice) API so long sessions stay cheap on CPU without forgetting earlier context. Today a realtime session either feeds the whole growing buffer to the LLM (latency death-spiral on CPU) or, with max_history_items, silently drops old turns and forgets them. This change lets the server fold aged-out turns into a rolling summary instead.

Two layers:

1. OpenAI-parity conversation events

The realtime endpoint was missing client-side history management that the OpenAI Realtime API relies on. Now implemented:

• conversation.item.delete — was a not_implemented stub; now removes the item and emits conversation.item.deleted.
• conversation.item.truncate — clears an assistant item's text/transcript at a content index (discard an interrupted/barge-in tail).
• input_audio_buffer.clear — resets pending input audio.

2. Summarize-then-drop compactor

• A rolling Conversation.Memory summary, kept out of Items so trimRealtimeItems can't drop it, injected into the prompt right after the instructions.
• Filled by an async, post-turn compactor using a snapshot → summarize → commit pattern that never holds conv.Lock across the summarizer LLM call. Commit re-validates the head (prefixMatches) so a concurrent item.delete can't cause lost/misdropped data. Function-call/output pairs are never split across the boundary.
• A failed/empty summary leaves the buffer untouched — items are never evicted without a summary replacing them.
• This resolves the long-standing in-code TODO at the Conversation creation site.

Config (two-number model)

pipeline:
  max_history_items: 6        # live window — recent turns kept verbatim
  compaction:
    enabled: true
    trigger_items: 12         # high-water mark; summarize overflow back down to max_history_items
    summary_model: ""         # optional small/cheap model for the summary (CPU); default = pipeline LLM
    max_summary_tokens: 512

max_history_items = live window; compaction.trigger_items = high-water mark (must exceed it). A summary call runs roughly every (trigger_items - max_history_items) turns. The summarizer model is resolved lazily, inside the compaction goroutine (off the response path). With compaction absent/disabled, behavior is byte-for-byte unchanged.

Testing

• New Ginkgo/Gomega specs for every pure helper (resolveCompaction, itemID, deleteItem, truncateAssistantText, clearInputAudio, compactionCut, withMemory, renderItemsTranscript, buildSummaryMessages, prefixMatches, compact, summarizerModel), incl. the commit/abort and summarizer-error paths.
• go test -race ./core/http/endpoints/openai/... clean (the compactor spawns a goroutine).
• make lint reports 0 issues on the feature files.
• Folded into the existing TestOpenAI suite (single RunSpecs).

Notes / follow-ups (out of scope)

• max_summary_tokens is advisory (fed to the prompt) in this PR; a hard Predict-level cap is a follow-up.
• A small/fast summary_model for CPU loads via the realtime pipeline path and falls back to the pipeline LLM if it can't be constructed.
• Docs: new "Conversation compaction" section in docs/content/features/openai-realtime.md.

🤖 Generated with Claude Code

Assisted-by: Claude:claude-opus-4-8 [Claude Code]