Skip to content

ModelLoader: read failed for tensor lm.blk.0.attn_norm.weight with ASR q4_k.gguf #7

Description

Running vibevoice-cli on Windows always fails with ModelLoader: read failed for tensor lm.blk.0.attn_norm.weight immediately after the audio is loaded. The build itself succeeds without errors, only inference fails.
The issue reproduces with both vibevoice-asr-q4_k.gguf and vibevoice-asr-q8_0.gguf from the official mudler/vibevoice.cpp-models HF repository.

Reproduce

  1. Clone the repository and check out v0.1
  2. Build on Windows with MSYS2 / MinGW-w64 UCRT64:
    cmake -B build -G "Ninja" -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DGGML_USE_MMAP=OFF
    cmake --build build -j
    
  3. Copy required DLLs next to the executable:
    copy C:\msys64\ucrt64\bin\libwinpthread-1.dll build\bin\
    copy C:\msys64\ucrt64\bin\libgcc_s_seh-1.dll build\bin\
    copy C:\msys64\ucrt64\bin\libstdc++-6.dll build\bin\
    copy C:\msys64\ucrt64\bin\libgomp-1.dll build\bin\
    
  4. Download models:
    hf download mudler/vibevoice.cpp-models --local-dir models
    
  5. Run inference:
    .\build\bin\vibevoice-cli.exe asr --model ..\models\vibevoice-asr-q4_k.gguf --tokenizer ..\models\tokenizer.gguf --audio "audio.wav" --max-new-tokens 8192
    

Expected

The model loads successfully and transcribes the audio.

Actual

asr: loaded 142636 samples (5.94s)
[vv I] backend: CPU
[vv E] ModelLoader: read failed for tensor lm.blk.0.attn_norm.weight
asr: failed to load model

Environment

OS Windows 11
Compiler GCC 16.1.0 (MSYS2 UCRT64, MinGW-w64)
Branches tested master (commit ad856bd) and v0.1 tag (commit 8ffffa1)
ggml version 0.10.0 (commit 8be60f83)
Model vibevoice-asr-q4_k.gguf (10,392,063,296 bytes)
Model source hf download mudler/vibevoice.cpp-models

Additional context

  • The error occurs at the very first tensor (lm.blk.0.attn_norm.weight), which suggests a GGUF format mismatch between the uploaded HF models and the current model_loader.cpp parsing logic, or a Windows-specific file reading issue.
  • Disabling mmap (-DGGML_USE_MMAP=OFF) does not resolve the issue.
  • Building from both the latest master and the v0.1 tag (which was tagged at the same time the models were uploaded to HF) produces the same error.
  • The build completes without errors. The CLI starts correctly and loads the audio file successfully, only model loading fails.

Has this been tested on Windows by the maintainers? Is there a known workaround or a specific build configuration required for Windows?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions