fix(vec): implement missing sample_db_embedding_widths (#160)#163
Closed
J0nd1sk wants to merge 1 commit into
Closed
fix(vec): implement missing sample_db_embedding_widths (#160)#163J0nd1sk wants to merge 1 commit into
J0nd1sk wants to merge 1 commit into
Conversation
`cmd_vec_reindex` imported `sample_db_embedding_widths` from `agentmemory.embeddings` (and called it for the audit-H4 width-consistency check), but the function was never defined — so `brainctl vec reindex` crashed with `ImportError` for every user. This is the documented path for migrating the vector index after a model switch, so the whole flow was dead. Implement the probe next to `get_db_embedding_dim`. It samples up to `sample_size` rows from `vec_memories`, measures each vector's float32 width (`len(blob) // 4`), and cross-checks against the dim declared in the table DDL. Returns `ok=False` with `sample_count=0` when the table is absent/unreadable so the reindex guard treats it as "nothing to check" and proceeds to build a fresh index; flags `consistent=False` only when sampled rows disagree with each other or the declared dim. Fixes #160 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #160.
Problem
cmd_vec_reindex(_impl.py:7855) importssample_db_embedding_widthsfromagentmemory.embeddingsand calls it at_impl.py:7873, but the function was never defined.brainctl vec reindextherefore crashed with:{"error": "ImportError: cannot import name 'sample_db_embedding_widths' from 'agentmemory.embeddings' ..."}for everyone.
pyproject.tomldocumentsvec reindexas the way to migrate the index after changingBRAINCTL_EMBED_MODEL, so that whole path was dead.Fix
Implement
sample_db_embedding_widths(conn, sample_size=8)next toget_db_embedding_dim, matching the contract the existing callsite already expects (ok,sample_count,consistent,declared_dim,observed_dims,message):sample_sizerows fromvec_memories, width =len(blob) // 4(float32);ok=False, sample_count=0when the table is absent/unreadable, so the guard skips and a fresh index is built;consistent=Falseonly on genuine disagreement (mixed widths, or uniform width ≠ declared dim).Tests
tests/test_embeddings_width_probe.py— 9 tests: import regression, missing/empty table, uniform-matches-declared, uniform-mismatch, mixed-width corruption, unknown declared dim, sample-size cap, and a realsqlite-vecvec0round-trip.Verification