Search perf + sessions corpus + background-work policy + oa→aix#60
Merged
Conversation
…ration Four cohesive changes (overlapping files, so one commit): Search latency quick-wins (#56) - Lazy embedder: open_corpus binds a _LazyEmbedder that resolves the model only when a query actually embeds. `ir ls` 12.7s -> 0.48s; ls/info/lexical no longer load the model. - Packed vector matrix: CorpusStore persists one normalized matrix (mmap reload), a write-invalidated read cache, replacing the per-record vector-file storm on reopen. Plus a vector-free metas() so mode="lexical" skips the matrix, and a single meta pass in matrix(). Claude Code sessions corpus (#57) - ClaudeTurn strategy (user_prompt + assistant_summary surfaces; optional assistant_full; session_title surface + filter field) + from_claude_sessions (90-day default window, ai-title indexed) + `sessions` preset. Records come from priv.claude_transcripts (injectable fetcher, mirrors from_skills). Per-corpus background-work policy (#58) - ir/policy.py (MaintenancePolicy + smart per-kind defaults + timing predicates), registry v2 (persists/reconstructs strategy + maintenance + storage; closes the gap where source_from_entry always reverted to the preset default), and an idempotent `ir maintain` (incremental reindex when due; synopsis only in its downtime window). ir holds declarative policy; the scheduler stays external. Route LLM calls through aix instead of oa (#59) - synopsis/select/formulate/eval_gen now use aix.prompt_func (multi-provider facade) instead of oa.prompt_function; llm extra oa -> aix. Lazy + injectable pattern preserved (import ir stays offline). Live-verified end to end. Full suite green (399). argh needed an explicit NameMappingPolicy for the new optional-positional `maintain` command.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements four issues from this session (overlapping files → one branch). Full suite green (399).
#56 — search latency quick-wins
open_corpus): model loads only when a query embeds.ir ls12.7s → 0.48s;ls/info/lexical never load it.CorpusStore): one mmap'd normalized matrix as a write-invalidated read cache (replaces the per-record vector-file storm);metas()lets lexical skip the matrix; single meta pass.#57 — Claude Code sessions corpus
ClaudeTurnstrategy (user_prompt+assistant_summarysurfaces; optionalassistant_full;session_titlesurface),from_claude_sessions(90-day window,ai-titleindexed),sessionspreset. Parsing lives inpriv.claude_transcripts(injectable fetcher).#58 — per-corpus background-work policy
ir/policy.py(MaintenancePolicy + smart per-kind defaults + timing), registry v2 (now persists/reconstructs strategy + maintenance + storage), idempotentir maintain. Declarative policy in ir; scheduler external (cron snippet in module docstring). No Settings singleton; one-way ir→raglab preserved.#59 — route LLM calls through aix (not oa)
synopsis/select/formulate/eval_gen→aix.prompt_func;llmextraoa→aix. Lazy + injectable preserved; live-verified end to end.Depends on thorwhalen/aix#30 (adds
egress/nametoprompt_func). Theaiximport is optional + lazy, so CI without it skips the eval_gen default-generator tests (importorskip).Closes #56, #57, #58, #59.