-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Open
Labels
bugSomething isn't workingSomething isn't working
Description
软件环境
paddle2onnx 2.0.1
paddleformers 0.4.0
paddlefsl 1.1.0
paddlenlp 2.8.1
paddlepaddle-gpu 3.2.2
paddleslim 2.6.0
use_triton_in_paddle 0.1.0重复问题
- I have searched the existing issues
错误描述
使用paddlenlp的gptq量化ernie模型过程中,报错
cannot import name 'download' from 'aistudio_sdk.hub'
发现文件`aistudio_sdk/hub.py`中已无download方法,
文件`PaddleNLP/llm/run_quantization.py`中,24行,报错:
ImportError: cannot import name 'ZeroPaddingIterableDataset' from 'paddlenlp.datasets'
请问现在哪个版本可以使用gptq量化ernie-4.5-21b-a3b模型?稳定复现步骤 & 代码
gptq_config.json
{
"model_name_or_path": "/ernie45_21b_a3b_paddle",
"per_device_train_batch_size": 1,
"per_device_eval_batch_size": 1,
"eval_accumulation_steps": 8,
"src_length": 1024,
"max_length": 1024,
"bf16": false,
"fp16_opt_level": "O2",
"dataset_name_or_path": "data",
"output_dir": "./checkpoints/gptq_ckpts",
"do_eval": true,
"eval_with_do_generation": false,
"do_gptq": true,
"unified_checkpoint": true,
"gptq_step": 8
}
使用数据集
# 下载示例数据集
wget https://bj.bcebos.com/paddlenlp/datasets/examples/AdvertiseGen.tar.gz
tar -xzf AdvertiseGen.tar.gz -C data/
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working