Add MiniMax-M3 (MSA: MiniMax Sparse Attention) support - #24908

Merged

ngxson merged 64 commits into

ggml-org:masterfrom

Jul 26, 2026

Commits on Jun 22, 2026

Add preliminary MiniMax-M3 support

danielhanchen
authored and
timkhronos
committed
87cc29b
MiniMax-M3 vision tower (mmproj + clip graph)
timkhronos
committed
53c81dd
Delete m3_vision_ref.py
timkhronos
committed
f07f1d4
Update clip.cpp
timkhronos
committed
4a3206f
MSA
timkhronos
committed
bbf1a80
Update constants.py
timkhronos
committed
09657dd
Update minimax.py
timkhronos
committed
8fe2e01
Cache creation. Working withotu flash attention
timkhronos
committed
8c953a9
Added flash attention for sparse layers
timkhronos
committed
ea6fbd6
Decomposed slow cpu OP into GPU + CPU ops. Massive speedup over long ctx
timkhronos
committed
2e82759
Rewrote indexer op to be cuda native. Modified flash attention to match per group block picking
timkhronos
committed
0152226
Implement sparse attention calc out of stock ops.
timkhronos
committed
d1a04f7
Fix a cache allocation and cont issue
timkhronos
committed
b1b174e
Fixed -fa auto crash, flagged debug spots
timkhronos
committed
cea714a
Delete vocab.json
timkhronos
authored
69c958d
Delete model.safetensors.index.json
timkhronos
authored
afc09f1
Delete generation_config.json
timkhronos
authored
f40d0f5
Delete Minimax directory
timkhronos
authored
714bbe9

Commits on Jun 23, 2026

Commits on Jun 25, 2026

Changed 3 Gelu Ops for vision into Gelu_erf ops
timkhronos
committed
35990be
Assert that n_kv is multiple of 128
timkhronos
committed
fa15850

Commits on Jun 26, 2026

Rename MSA index tensors to indexer convention
timkhronos
authored
eae55e2
Fix incorrect Assert
timkhronos
authored
2bb7eeb

Commits on Jun 28, 2026

Review driven changes (#3)
timkhronos
authored
d6f9426
Remove comment from conversion minimax.py

timkhronos
and
CISC
authored
0a7b2dc
Remove whitespaces from constants.py

timkhronos
and
CISC
authored
636143a
Tighten comment in minimax.py

timkhronos
and
CISC
authored
7b7ff65
inherit MiniMax-M3 from MiniMax-M2
timkhronos
committed
1cd03ef
drop dead text_config fallbacks
timkhronos
committed
c70c8a9
Add indexer writer methods
timkhronos
committed
25199fa
Reuse LLM_FFN_SWIGLU_OAI_MOE
timkhronos
committed
d54b4eb
Remove duplicate indexer setters, add only block_size/local_blocks, follow value naming convention
timkhronos
committed
f57efce
Fix conversion error /gguf_writer.py

timkhronos
and
CISC
authored
618e145
Update gguf-py/gguf/gguf_writer.py

timkhronos
and
CISC
authored
5a24782
Update gguf-py/gguf/tensor_mapping.py

timkhronos
and
CISC
authored
5dfe838
Update conversion/minimax.py

timkhronos
and
CISC
authored
b99a8d5
Update conversion/minimax.py

timkhronos
and
CISC
authored
56ba541
Remove whitespace in src/llama-kv-cache.cpp

timkhronos
and
CISC
authored
044391a
Remove Whitespace in Update src/llama-model.h

timkhronos
and
CISC
authored
c314faf
Remove whitespace in src/llama-hparams.h

timkhronos
and
CISC
authored
5015a6b
remove multimodal code upon maintainer request. Will be made as a separate PR
timkhronos
committed
c97a2ac

Commits on Jun 29, 2026

Whitespace clean in tensor_mapping.py
timkhronos
authored
fcb9ed2

Commits on Jun 30, 2026

Merge branch 'master' into MSA
timkhronos
authored
51e7bf8

Commits on Jul 1, 2026

Merge branch 'ggml-org:master' into MSA
timkhronos
authored
6b14fea

Commits on Jul 3, 2026

Log cache size on launch, block ctx shift, support prompt caching
timkhronos
authored
d2fe995

Commits on Jul 10, 2026

Merge branch 'ggml-org:master' into MSA
timkhronos
authored
9ea2aa1
Update minimax-m3.cpp
timkhronos
authored
aa73ec3

Commits on Jul 11, 2026

Optimize implementation, add multi stream support.
timkhronos
authored
cacc42f

Commits on Jul 14, 2026

Commits on Jul 19, 2026

Add Minimax eos to llama vocab
timkhronos
authored
f020121
Guard edge case where idx cache can become stale after a tail trim
timkhronos
authored
da2c168

Commits on Jul 23, 2026

Merge branch 'ggml-org:master' into MSA
timkhronos
authored
297a179

Commits on Jul 25, 2026

Update llama-kv-cache.cpp
timkhronos
authored
e53544b
Merge branch 'master' into MSA
timkhronos
authored
954598b
Review driven changes
timkhronos
committed
d6cc337

Commits on Jul 26, 2026

Merge branch 'ggml-org:master' into MSA
timkhronos
authored
8bfbead
style fix
Xuan Son Nguyen
committed
285df2d
indexer hparams are required
Xuan Son Nguyen
committed
a56fd29
fix tests
Xuan Son Nguyen
committed
296e8e4
fix lint
Xuan Son Nguyen
committed
5d5ed01