[AMD] [DNM, still merge in 0.18 as trust_remote_code=True is not passed to quark] Add MiniMax M2.5 MXFP4 benchmark for MI355x vLLM v0.17.1 (TP=2,4)#827
Conversation
|
@claude can u comment out TP=4 and only do TP=2 for now since TP=4 is bugged vllm-project/vllm#35637 |
|
Claude finished @functionstackx's task in 53s —— View job Commented out TP=4 for MiniMax M2.1 MXFP4Due to vllm-project/vllm#35637, TP=4 has been disabled. Only TP=2 will run.
Changes made:
|
|
@chunfangamd i am seeing good mxfp4 performance when comparing to FP8. just waiting for AMD to release mxfp4 version of MiniMax M2.5 now. currently AMD only have mxfp4 version of MiniMax M2.1.
|
bd10495 to
e849d65
Compare
86cc700 to
b82116b
Compare
b82116b to
7dd6063
Compare
|
@adibarra do u wanna take over this PR too? upgraading this PR to v0.18.1, this might work |
|
Sure, I'll give it a shot |
ddd4f96 to
1f44d49
Compare
|
@adibarra Hi, I think we need wait for vllm v0.19.1 for this. The fix PR can be found in vllm 0.19.1rc0. 0.19.0 still don't have the fix. |
|
Sounds good, we'll wait till then! |
|
@adibarra Run passed. Wait for VLLM v0.19.1's release. |
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you |


Add MiniMax M2.5 MXFP4 benchmark config for MI355x with vLLM v0.17.1, now that AMD's MXFP4 checkpoint is out: https://huggingface.co/amd/MiniMax-M2.5-MXFP4
Closes #826
Generated with Claude Code