Skip to content

Fix/hip apu host buffer#25863

Open
Victor-Loos wants to merge 1 commit into
ggml-org:masterfrom
Victor-Loos:fix/hip-apu-host-buffer
Open

Fix/hip apu host buffer#25863
Victor-Loos wants to merge 1 commit into
ggml-org:masterfrom
Victor-Loos:fix/hip-apu-host-buffer

Conversation

@Victor-Loos

Copy link
Copy Markdown

Overview

Fixes a regression after #24233, which restored HIP integrated GPU detection. On my AMD APU, this also made the ROCm host-buffer path available for HIP integrated GPUs. With that path enabled, prompt input appears corrupted: chat templates and system prompts can be ignored, and generated output becomes garbage.

This change keeps integrated GPU detection/reporting intact, but disables host-buffer exposure for HIP integrated GPUs.

Additional information

Tested locally on gfx1151 / RDNA3.5 APU with ROCm 7.14. The corrupted input/output issue is fixed with this patch.
The built-in web UI did not reproduce the issue when used first, but external harnesses like Pi and OpenCode do.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES - I used AI assistance to review and patch the relevant code. I tested and reviewed the final change myself.

@Victor-Loos
Victor-Loos requested a review from a team as a code owner July 18, 2026 15:22
@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning CUDA Related to the CUDA backend labels Jul 18, 2026
@pwilkin

pwilkin commented Jul 19, 2026

Copy link
Copy Markdown
Member

@liminfei-amd could you please take a look?

@liminfei-amd

Copy link
Copy Markdown
Contributor

Thanks for the ping. I tested this on gfx1151 / ROCm 7.2.1. Before #24233, the host-buffer capability and factory were already enabled; the newly reachable behavior was supports_buft(ROCm_Host): false -> true.

The current patch fixes this conservatively, but disabling the factory reduced pp512 from 5857 to 1502 t/s. I suggest a narrower fix:

caps/factory = GGML_CUDA_NO_PINNED is not set
supports_buft(ROCm_Host) = caps/factory enabled && not a HIP integrated device

This keeps pinned staging/output buffers but prevents direct HIP APU computation on ROCm_Host. It retained 5725 t/s (97.7% of baseline). Could you test this variant with the exact Pi/OpenCode reproduction?

@Victor-Loos
Victor-Loos force-pushed the fix/hip-apu-host-buffer branch from 8a02d7f to fdc1260 Compare July 20, 2026 20:05
@Victor-Loos

Copy link
Copy Markdown
Author

Thanks @liminfei-amd for the testing and for clarifying the narrower fix.

I tested your suggested variant with my setup, and it still fixes the issue. I've updated the PR accordingly.

As for performance, on my setup I didn't see any meaningful difference. Here are my llama-bench results with a Qwen3.6-35B-A3B model:

Version pp512 (t/s) tg128 (t/s)
Before 1165.42 ± 29.48 67.57 ± 0.87
Fix v1 1148.87 ± 23.64 67.76 ± 0.70
Fix v2 1144.09 ± 20.46 68.11 ± 0.74

Could you retest the updated version on your side and confirm whether it preserves the pp512 result you mentioned? If there's still a difference, could you share the exact command and model you used so I can try to reproduce it?

@flashburns

Copy link
Copy Markdown

Hello I ran into issues on my strix halo box and this PR fixes it. Step 3.7 Flash IQ4_XS will spam < in a loop at 4k+ context.
I had an agent bisect the issue and it traced it down to the same commit & PR. c7d8722

I have now tested this pull request on my box and its solved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CUDA Related to the CUDA backend ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants