Skip to content

feat(search-v2): add hybrid search with BM25 fusion and Cohere reranking#140

Merged
DevanshuNEU merged 1 commit into
OpenCodeIntel:mainfrom
DevanshuNEU:feat/hybrid-search-reranking
Jan 4, 2026
Merged

feat(search-v2): add hybrid search with BM25 fusion and Cohere reranking#140
DevanshuNEU merged 1 commit into
OpenCodeIntel:mainfrom
DevanshuNEU:feat/hybrid-search-reranking

Conversation

@DevanshuNEU

Copy link
Copy Markdown
Collaborator

Summary

Adds hybrid search combining semantic embeddings with BM25 keyword matching, plus Cohere reranking for optimal result ordering.

Closes #70

Changes

File Description
hybrid_searcher.py BM25 + semantic fusion with RRF, Cohere reranking
indexer_optimized.py Added search_v2() method
requirements.txt Added rank-bm25, cohere

How It Works

Query → Semantic Search (top 50) → BM25 Scoring → RRF Fusion → Cohere Rerank → Top 10
  1. Semantic search: Get 50 candidates from Pinecone
  2. BM25 scoring: Apply keyword matching on candidates
  3. RRF fusion: Combine scores using Reciprocal Rank Fusion
  4. Cohere rerank: Final ordering with rerank-v3.5

Configuration

# Required for reranking (optional - gracefully skipped if missing)
COHERE_API_KEY=xxx

Test Results

8 passed (31 total for Search V2)

Part of Search V2 Epic

Closes OpenCodeIntel#70

Add hybrid search combining semantic and keyword matching:

- HybridSearcher with BM25 + semantic fusion (RRF algorithm)
- Cohere rerank-v3.5 for final result ordering
- search_v2() method in indexer
- Graceful fallback when Cohere unavailable
- 8 tests passing (31 total for Search V2)

Dependencies: rank-bm25, cohere

Part 3 of 4 in Search V2 epic.
@vercel

vercel Bot commented Jan 4, 2026

Copy link
Copy Markdown

@DevanshuNEU is attempting to deploy a commit to the Dev's projects Team on Vercel.

A member of the Team first needs to authorize it.

@vercel

vercel Bot commented Jan 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Review Updated (UTC)
opencodeintel Ignored Ignored Preview Jan 4, 2026 10:48pm

@DevanshuNEU DevanshuNEU merged commit 97d3913 into OpenCodeIntel:main Jan 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(search): Add confidence scores to search results

1 participant