Skip to content

fix(vec): implement missing sample_db_embedding_widths (#160)#163

Closed
J0nd1sk wants to merge 1 commit into
TSchonleber:mainfrom
J0nd1sk:fix/vec-reindex-width-probe-160
Closed

fix(vec): implement missing sample_db_embedding_widths (#160)#163
J0nd1sk wants to merge 1 commit into
TSchonleber:mainfrom
J0nd1sk:fix/vec-reindex-width-probe-160

Conversation

@J0nd1sk

@J0nd1sk J0nd1sk commented Jun 29, 2026

Copy link
Copy Markdown

Closes #160.

Problem

cmd_vec_reindex (_impl.py:7855) imports sample_db_embedding_widths from agentmemory.embeddings and calls it at _impl.py:7873, but the function was never defined. brainctl vec reindex therefore crashed with:

{"error": "ImportError: cannot import name 'sample_db_embedding_widths' from 'agentmemory.embeddings' ..."}

for everyone. pyproject.toml documents vec reindex as the way to migrate the index after changing BRAINCTL_EMBED_MODEL, so that whole path was dead.

Fix

Implement sample_db_embedding_widths(conn, sample_size=8) next to get_db_embedding_dim, matching the contract the existing callsite already expects (ok, sample_count, consistent, declared_dim, observed_dims, message):

  • samples up to sample_size rows from vec_memories, width = len(blob) // 4 (float32);
  • cross-checks observed width(s) against the DDL-declared dim (audit H4);
  • returns ok=False, sample_count=0 when the table is absent/unreadable, so the guard skips and a fresh index is built;
  • flags consistent=False only on genuine disagreement (mixed widths, or uniform width ≠ declared dim).

Tests

tests/test_embeddings_width_probe.py — 9 tests: import regression, missing/empty table, uniform-matches-declared, uniform-mismatch, mixed-width corruption, unknown declared dim, sample-size cap, and a real sqlite-vec vec0 round-trip.

Verification

$ pytest tests/test_vec.py tests/test_embeddings_width_probe.py -q
30 passed

# end-to-end, against a fresh brain.db with Ollama running:
$ brainctl vec reindex
{"ok": true, "model": "nomic-embed-text", "dim": 768, "embedded": 1, ...}

`cmd_vec_reindex` imported `sample_db_embedding_widths` from
`agentmemory.embeddings` (and called it for the audit-H4 width-consistency
check), but the function was never defined — so `brainctl vec reindex`
crashed with `ImportError` for every user. This is the documented path for
migrating the vector index after a model switch, so the whole flow was
dead.

Implement the probe next to `get_db_embedding_dim`. It samples up to
`sample_size` rows from `vec_memories`, measures each vector's float32
width (`len(blob) // 4`), and cross-checks against the dim declared in the
table DDL. Returns `ok=False` with `sample_count=0` when the table is
absent/unreadable so the reindex guard treats it as "nothing to check" and
proceeds to build a fresh index; flags `consistent=False` only when sampled
rows disagree with each other or the declared dim.

Fixes #160

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@J0nd1sk J0nd1sk closed this by deleting the head repository Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] vec reindex crashes — ImportError: cannot import name 'sample_db_embedding_widths' from agentmemory.embeddings

1 participant