[Bug]: Qwen3-30B-A3B KeyError: 'blk.0.ffn_down_exps'

[XuehaoSun]

Problem Description

Traceback (most recent call last):
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/bin/auto-round", line 10, in <module>
    sys.exit(run())
             ^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/auto_round/__main__.py", line 811, in run
    start()
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/auto_round/__main__.py", line 533, in start
    tune(args)
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/auto_round/__main__.py", line 757, in tune
    run_model_evaluation(model, tokenizer, autoround, folders, formats, device_str, args)
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/auto_round/eval/evaluation.py", line 436, in run_model_evaluation
    model, tokenizer = load_gguf_model_for_eval(eval_folder, formats, args)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/auto_round/eval/evaluation.py", line 204, in load_gguf_model_for_eval
    model = AutoModelForCausalLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/auto_round/utils/common.py", line 102, in patched
    return underlying_func(klass, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py", line 374, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/transformers/modeling_utils.py", line 4080, in from_pretrained
    state_dict = load_gguf_checkpoint(checkpoint_files[0], return_tensors=True, model_to_load=dummy_model)[
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/transformers/modeling_gguf_pytorch_utils.py", line 567, in load_gguf_checkpoint
    result = processor.process(
             ^^^^^^^^^^^^^^^^^^
  File "/home/uttest/miniforge3/envs/autoround_v0.12.0_release/lib/python3.12/site-packages/transformers/modeling_gguf_pytorch_utils.py", line 143, in process
    self._set_moe_expert_tensor(weights, parsed_parameters, tensor_key_mapping[m["name"]], m["w"])
                                                            ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'blk.0.ffn_down_exps'

Reproduction Steps

auto-round --model_name /models/Qwen3-30B-A3B --format gguf:q4_k_m --iters 0

Environment Information

No response

Error Logs

https://inteltf-jenk.sh.intel.com/job/AutoRound_LLM_accuracy_test/418//artifact/logs/woq//models/Qwen3-30B-A3B/weight_only_quant_1.log

Additional Context

No response

[n1ck-guo]

#1575