Skip to content

locomo: add grep-allviews K200 retrieval results (best K=20/50 recall)#13

Open
tatavishnurao wants to merge 1 commit into
sochdb:mainfrom
tatavishnurao:feat/locomo-grep-allviews-k200
Open

locomo: add grep-allviews K200 retrieval results (best K=20/50 recall)#13
tatavishnurao wants to merge 1 commit into
sochdb:mainfrom
tatavishnurao:feat/locomo-grep-allviews-k200

Conversation

@tatavishnurao

Copy link
Copy Markdown
Contributor

Publishable Grep+Allviews Retrieval Pipeline (K=200)

Single-pass retrieval with 4 multiviews, BM25+SochDB vector+grep (trigram) RRF fusion, and local neighbor expansion.

Results

K=20

Category n Hit@20 Recall@20
adversarial 446 0.904 0.901
multi_hop 89 0.730 0.628
open_domain 841 0.954 0.946
single_hop 281 0.918 0.670
temporal 320 0.934 0.912
overall 1977 0.924 0.877

K=50

Category n Hit@50 Recall@50
adversarial 446 0.957 0.955
multi_hop 89 0.854 0.737
open_domain 841 0.985 0.982
single_hop 281 0.968 0.819
temporal 320 0.975 0.965
overall 1977 0.969 0.939

K=100

Category n Hit@100 Recall@100
adversarial 446 0.980 0.978
multi_hop 89 0.899 0.795
open_domain 841 0.994 0.993
single_hop 281 0.996 0.903
temporal 320 0.994 0.984
overall 1977 0.987 0.966

K=200

Category n Hit@200 Recall@200
adversarial 446 0.993 0.992
multi_hop 89 0.921 0.858
open_domain 841 0.995 0.995
single_hop 281 1.000 0.961
temporal 320 0.997 0.996
overall 1977 0.992 0.984

Pipeline Config

  • 4 multiviews: turn + event + entity + neighbor_window
  • BM25 + SochDB vector + Grep (trigram) RRF fusion (bm25=0.1, vec=3.0, grep=0.5, rrf_k=60)
  • Local neighbor expansion: ±2 turns from top-50 anchors
  • NVIDIA llama-nemotron-embed-1b-v2 (2048-dim)
  • K=200, candidate_k=400, single query mode

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant