feat: native single-hop Expand via CSR adjacency index (Phase 2, #159)

[jja725]

Summary

Phase 2 of #159: execute single-hop Cypher Expand natively using the Phase 1 CsrIndex instead of relationship-scan + two SQL joins. Implemented as custom DataFusion operators, with automatic fallback to the existing join path for anything not yet supported natively.

This is a DuckPGQ-style relational engine + CSR acceleration integration:

• CsrExpandExec — topology only: for each source row, look up neighbors in the CSR and emit one row per neighbor with the neighbor's row id appended.
• LanceTakeExec — materializes target node properties from those row ids via a RowMaterializer (in-memory arrow::compute::take now; a Lance-dataset take lands in Phase 4).
• CsrExpandNode / LanceTakeNode — logical extension nodes; CsrExtensionPlanner + CsrQueryPlanner build the CSR and materializer at physical-planning time.
• LanceNativePlanner overrides only Expand lowering and delegates everything else to DataFusionPlanner, so ExecutionStrategy::LanceNative is always correct — it uses CSR when it can and joins otherwise.

Design decisions

• Dense-ROWID model: the CSR vertex id is the node's row id (csr.neighbors(src_rowid) -> dst_rowids), mirroring how every Lance index works (key → row ids → take() to materialize). Generalizes to Lance stable row ids in Phase 4.
• Materialize all target columns via take() (schema parity with the join path) rather than analyzing which are referenced.

Full design and task breakdown: docs/superpowers/specs/2026-06-22-csr-native-expand-operator-design.md and docs/superpowers/plans/2026-06-22-csr-native-expand-operator.md.

Native vs. fallback

Served natively: exactly one single-hop Expand, single relationship type, Outgoing/Incoming, no inline relationship/target property filters, no bound relationship variable. Wrapping Project/Filter/Sort/Limit/Offset/Distinct run as normal DataFusion operators on the native stream.

Falls back to the DataFusion join path: variable-length / multi-hop, multiple relationship types, undirected, inline {k:v} filters, bound relationship variable, Join, Unwind.

Out of scope (later phases)

• Multi-hop / VariableLengthExpand, BFS/DFS/shortest-path operators (Phase 3)
• Persisting CSR as Lance datasets, incremental updates, stable row ids, LanceDatasetMaterializer, namespace native path (Phase 4)
• Hybrid CSR + vector search (Phase 5)

Test plan

• Unit tests for expand_batch, take_batch, InMemoryMaterializer, the CSR builder column generalization, and the planner native/fallback decision.
• End-to-end parity tests (tests/test_lance_native_expand.rs) asserting LanceNative returns identical results to DataFusion for: single-hop RETURN a.name, b.name, with WHERE b.age > 30, incoming direction, and a variable-length query (fallback).
• Full crate suite green: cargo test -p lance-graph (15 binaries, 0 failures); cargo clippy -p lance-graph --all-targets clean.

Known follow-ups (not blocking)

• Resolve the source-id column index at execution time (by name) rather than at plan time — robust against future multi-partition / column-pruning passes (safe under DataFusion 50 today).

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

❌ Patch coverage is 58.00604% with 417 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...rates/lance-graph/src/lance_native_planner/take.rs	53.08%	114 Missing ⚠️
...lance-graph/src/lance_native_planner/csr_expand.rs	49.09%	112 Missing ⚠️
crates/lance-graph/src/lance_native_planner/mod.rs	72.52%	100 Missing ⚠️
crates/lance-graph/src/query.rs	8.33%	55 Missing ⚠️
...raph/src/lance_native_planner/extension_planner.rs	0.00%	32 Missing ⚠️
crates/lance-graph/src/csr_index.rs	94.36%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!