-
Notifications
You must be signed in to change notification settings - Fork 80
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Description
Add HPS benchmark for text-to-image evaluation. Uses zhwang/HPDv2 dataset with 4 categories.
Details
- Dataset:
zhwang/HPDv2(benchmark subfolder contains JSON files) - Categories: anime, concept-art, paintings, photo
- Collate:
prompt_with_auxiliaries_collate
Implementation
- Add
setup_hps_datasetinsrc/pruna/data/datasets/prompt.py - Register in
base_datasetsinsrc/pruna/data/__init__.py - Add
BenchmarkInfoentry with metrics:["hps"] - Add test in
tests/data/test_datamodule.py
Acceptance Criteria
-
PrunaDataModule.from_string("HPS")works - Category filter:
PrunaDataModule.from_string("HPS", category="anime") - Auxiliaries include
categoryfield - Test passes
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request