feat: support lance vector search

[ChunxuTang]

This PR implements a vector search API that could be used for a multi-stage graphRAG pipeline: running a normal graph Cypher query to fetch the data, then applying a vector search to post-filering/reranking the returned result.

The vector search integrates both the existing in-memory UDF-based vector search and a new Lance ANN index-based optimized vector search (more suitable for large-scale vector search).

The execute_with_vector_rerank API allows users to query a dataset with a Cypher query first, then filter/rerank the returned dataset (but only with the in-memory vector search to be compatible with the Python execute API)

let query = CypherQuery::new(
        "MATCH (d:Document) \
         RETURN d.id, d.name, d.embedding",
    )?
    .with_config(config);

let results = query
    .execute_with_vector_rerank(
        datasets,
        VectorSearch::new("d.embedding")
            .query_vector(vec![1.0, 0.0, 0.0])
            .metric(DistanceMetric::L2)
            .top_k(3),
    )
    .await?;

Users currently can only run the Lance optimized vector search only from the Rust side, e.g.

let ann_results = VectorSearch::new("embedding")
    .query_vector(vec![1.0, 0.0, 0.0])
    .metric(DistanceMetric::L2)
    .top_k(5)
    .include_distance(true)
    .search_lance(&lance_dataset)
    .await?;

[codecov-commenter]

Codecov Report

❌ Patch coverage is 83.24873% with 66 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-graph/src/lance_vector_search.rs	79.62%	55 Missing ⚠️
...t/lance-graph/src/datafusion_planner/vector_ops.rs	91.30%	10 Missing ⚠️
rust/lance-graph/src/error.rs	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!