Search results

[P1][Model Cleanup] Remove legacy qwen2_5_vl_interleave backend

enhancementNew feature or requestNew feature or request

Status: Open.

#1117 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

[P0][Infra] Unify response cache path and fix agentic cache-key collisions

enhancementNew feature or requestNew feature or request

Status: Open.

#1116 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

[P1][Infra] Benchmark parity dashboard vs VLMEvalKit/OpenCompass

enhancementNew feature or requestNew feature or request

Status: Open.

#1115 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

[P1][DX] Refresh Gemini 3.1 / Seed2.0 model defaults and judge configuration

enhancementNew feature or requestNew feature or request

Status: Open.

#1114 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

[P1][Benchmark] BrowseComp agentic search benchmark integration tracking

enhancementNew feature or requestNew feature or request

Status: Open.

#1113 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

[P0][Benchmark] MINERVA video reasoning benchmark integration tracking

enhancementNew feature or requestNew feature or request

Status: Open.

#1112 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

[P0][Benchmark] Neptune long-video benchmark integration tracking

enhancementNew feature or requestNew feature or request

Status: Open.

#1111 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Feb 20, 2026

Issues about MMVP Dataset

Status: Open.

#1018 In EvolvingLMMs-Lab/lmms-eval;

· Luodian opened on Jan 22, 2026

When I tested Qwen-2.5-vl-7B on the Videomme dataset, ValueErrorr: The decoder prompt (length 163270) is longer

Status: Open.

#988 In EvolvingLMMs-Lab/lmms-eval;

· LyuQi127 opened on Jan 12, 2026

MVBench dataset average result

Status: Open.

#925 In EvolvingLMMs-Lab/lmms-eval;

· ylllllll opened on Dec 8, 2025

用vllm和不用vllm结果差的很多

Status: Open.

#921 In EvolvingLMMs-Lab/lmms-eval;

· Fu-Fu-Fu-Fu opened on Dec 3, 2025

The LLM Judge function takes too long, resulting in low GPU utilization.

Status: Open.

#908 In EvolvingLMMs-Lab/lmms-eval;

· Fu-Fu-Fu-Fu opened on Nov 20, 2025