Skip to content

[Bug]: 量化模型失败,对应包版本内有bug,不兼容 #11184

@ykallan

Description

@ykallan

软件环境

paddle2onnx                              2.0.1
paddleformers                            0.4.0
paddlefsl                                1.1.0
paddlenlp                                2.8.1
paddlepaddle-gpu                         3.2.2
paddleslim                               2.6.0
use_triton_in_paddle                     0.1.0

重复问题

  • I have searched the existing issues

错误描述

使用paddlenlp的gptq量化ernie模型过程中,报错

cannot import name 'download' from 'aistudio_sdk.hub'

发现文件`aistudio_sdk/hub.py`中已无download方法,
文件`PaddleNLP/llm/run_quantization.py`中,24行,报错:

ImportError: cannot import name 'ZeroPaddingIterableDataset' from 'paddlenlp.datasets'


请问现在哪个版本可以使用gptq量化ernie-4.5-21b-a3b模型?

稳定复现步骤 & 代码

gptq_config.json

{
    "model_name_or_path": "/ernie45_21b_a3b_paddle",
    "per_device_train_batch_size": 1,
    "per_device_eval_batch_size": 1,
    "eval_accumulation_steps": 8,
    "src_length": 1024,
    "max_length": 1024,
    "bf16": false,
    "fp16_opt_level": "O2",
    "dataset_name_or_path": "data",
    "output_dir": "./checkpoints/gptq_ckpts",
    "do_eval": true,
    "eval_with_do_generation": false,
    "do_gptq": true,
    "unified_checkpoint": true,
    "gptq_step": 8
}

使用数据集


# 下载示例数据集
wget https://bj.bcebos.com/paddlenlp/datasets/examples/AdvertiseGen.tar.gz
tar -xzf AdvertiseGen.tar.gz -C data/

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions