perf(vectors): single-component bulk declare + MPS default (init ~143s → 44s)

[HumanBean17]

Summary

Cuts full init from ~143s (cold) / ~98s (warm) → ~44s on Shopizer (1167 files / ~3500 chunks) by eliminating a per-file Lance transaction storm in the vectors phase — the phase that was ~84% of init time. The fix stays entirely inside cocoindex (no bypass, no schema/search change, increment stays correct and proportional).

Root cause

cocoindex flushes target writes once per processing component, batching all declare_row calls inside a component into a single merge_insert. coco.mount_each creates one component per source file, so a 1167-file repo produced 1168 Lance fragments / 1170 manifest commits — one merge_insert (+ fsync) per file. Measured: the vectors phase spent 91.2s of kernel sys time on commit I/O alone. This is open upstream as cocoindex#2219 ("Support batching for target state sinks from different components").

Non-bottlenecks confirmed by measurement (don't chase): parse_java is Rust tree-sitter (~0.9s total), embedding is ~27s CPU / ~16s MPS, optimize is ~1.4s. The sink was purely the per-file commit granularity.

Changes (java_index_flow_lancedb.py only, +34/−19)

1. Single-component bulk declare. app_main now iterates walk_dir().items() in one async for loop instead of three coco.mount_each calls → all chunks flush as one merge_insert per table. process_*_file stay @coco.fn(memo=True), and _RowHandler.reconcile skips rows whose fingerprint is unchanged, so increment stays correct and proportional. Random uuid4() ids are only generated on a memo miss (changed file); on a memo hit coco replays the recorded target state (same ids/content) → reconcile skips.
2. MPS default. The lifespan defaults the embedder to Apple Metal when available (torch is already imported in this child process, so the check is free; the CLI parent never imports torch). Override with SBERT_DEVICE=cpu.

Measured (Shopizer)

metric	before	after
init end-to-end	~143s cold / 98s warm	43.8s
vectors phase	119.7s	40.4s (CPU) / 37.5s (MPS)
kernel I/O (sys)	91.2s	4.2s
Lance fragments	1168	1
no-change increment	—	2.4s, 0 new fragments
1-file-change increment	—	10.2s, +1 fragment

Correctness

• Empirical: no-change increment preserves all 3475 rows (0 new fragments); 1-file change updates only that file's rows (+1 fragment).
• Tests: full suite green (JAVA_CODEBASE_RAG_RUN_HEAVY=1, incl. cocoindex init/increment/reprocess lifecycle tests).

Notes / follow-ups

• This is the supported workaround for cocoindex#2219. One residual cost: each increment re-iterates all files so coco can fingerprint them for memo/reconcile (~3s on 1167 files). When #2219 lands we can revert to mount_each for per-file source-memo (no re-visit) while keeping bulk-load fast — tracked in perf(vectors): adopt cocoindex native cross-component batching (#2219) when available; revert single-component loop #380.
• Increment graph still re-parses the whole project for global pass 5/6 even when no Clients/Producers exist — tracked in perf(graph): skip full global pass-1 re-parse in increment when no Clients/Producers exist #379.
• "Migrate to Rust" was considered and rejected with data: the bottleneck was LanceDB's (Rust) per-file commit granularity — an architectural choice, not a language one.

🤖 Generated with Claude Code