-
Notifications
You must be signed in to change notification settings - Fork 80
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Description
Add LongTextBench for evaluating text-to-image with long prompts.
Details
- Dataset:
X-Omni/LongText-Bench - Collate:
prompt_with_auxiliaries_collate
Implementation
- Add
setup_long_text_bench_datasetinsrc/pruna/data/datasets/prompt.py - Register in
base_datasets - Add
BenchmarkInfoentry with metrics:["text_score"] - Add test
Acceptance Criteria
-
PrunaDataModule.from_string("LongTextBench")works - Auxiliaries include
textfield (for evaluation) - Test passes
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request