ggml-webgpu: fix minimax-m3 crash on webgpu by adding F16 REPEAT - #26307
Conversation
nikwen
left a comment
There was a problem hiding this comment.
I didn't see your PR earlier, so I spent a few hours investigating this bug and came to the same solution. Implementing F16 REPEAT fixes the issue.
The crash happens because tensor->buffer->buft->iface.get_name(tensor->buffer->buft) is "CPU" here. We incorrectly cast tensor->buffer->context to ggml_backend_webgpu_buffer_context *. The program then crashes when we try to return ctx->buffer because we attempt an invalid memory access during the copy constructor of wgpu::Buffer.
fairydreaming
left a comment
There was a problem hiding this comment.
Tested this, it fixes the problem with crashing test-llama-archs minimax-m3 test.
|
There seems to be a similar potential bug in ggml scheduling that causes a few other models in I noticed that the output buffer for a |
|
The minimax-m3 tests were disabled in #26415. They should probably be re-enabled after this PR was merged. (I've hit the open PR limit so can't open another PR myself right now.) |
Overview
This PR fixes the error of webgpu ci error for minimax-m3 introduced by #24908.
minimax-m3 graph calls
ggml_tensor * bmx = ggml_repeat_4d(ctx0, ...);followed byggml_add_inplace(ctx0, bmx, ...), but existing webgpu backend doesn't have F16 repeat, so it crashes withBus error, because the GPU ADD op ends up with an inplace destination on CPU buffer. So it fixes the error by adding F16 repeat support.Additional information
The root cause seems to be in ggml scheduler, so I'll investigate it and follow up on it in separate PR.
Requirements