Model: Minimax M2 - chat format

[pwilkin]

Add support for Minimax M2 chat format, minimal goal: handle tool calls.

[codersx]

Is there a problem with the chat format? It doesn't start with <think> but it ends with it.

[aaronnewsome]

I tried using the official chat_template.jinja serving unsloth/unsloth/MiniMax-M2-GGUF MiniMax-M2-UD-Q5_K_XL with llama-cpp:b6931, using the following command to launch

llama-server \
  --model ~/MiniMax-M2-UD-Q5_K_XL/MiniMax-M2-UD-Q5_K_XL-00001-of-00004.gguf \
  --alias MiniMax-M2-UD-Q5_K_XL \
  --log-verbosity 1 \
  --threads -1 \
  --ctx-size 131072 \
  --n-gpu-layers 99 \
  --temp 1.0 \
  --min-p 0.0 \
  --top-p 0.95 \
  --top-k 40 \
  --repeat-penalty 1.05 \
  --context-shift \
  --host 0.0.0.0 \
  --reasoning-format auto \
  --flash-attn off \
  --jinja --chat-template-file ~/MiniMax-M2-UD-Q5_K_XL/chat_template.jinja

The model does load and I can chat with it, getting 70-90 tokens/s generation with a simple prompt (it's FAST!). Tool calling is a complete failure though. I see the <minimax:tool_call> in the llama.cpp console verbose logs, but none of the coding agents I've tried work, Opencode, Qwen Code, Claude Code, Claude code with CCR (claude code router), Claude Code with Minimax M2 proxy, Crush, etc. Nothing I've tried has been able to understand the Minimax tool usage.

[jmfirth]

I understand that this is a WIP and not working, but adding a different experience than @aaronnewsome : I've been testing with https://huggingface.co/noctrex/MiniMax-M2-MXFP4_MOE-GGUF, and the first tool call works but then the engine crashes.

[pwilkin]

#16946 should resolve this, but there are also independent errors with running the model, see #16945 .

[hksdpc255]

Could we take a look at PR #16932? I’ve already implemented Minimax M2 tool calling there. I'm testing it with zed editor, MiniMax M2 performs impressively well on fairly complex tasks (full fill 65536 context length without any problem in a single task!). unsloth/MiniMax-M2-GGUF/MiniMax-M2-UD-Q3_K_XL and DevQuasar/MiniMaxAI.MiniMax-M2-GGUF/MiniMaxAI.MiniMax-M2.MXFP4_MOE has been tested.

[pwilkin]

@hksdpc255 I've looked at your code and I agree yours is the more complete solution, but please take a look at the remarks I made.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.