sycl: fix SDPA executing of fattn-onednn and ggml queue not synced.

[hibiday]

Overview

Sync the SDPA execution in fattn-onednn with the ggml SYCL queue before and after the execute call.

The dnnl stream and ggml's in-order queue are not necessarily identical.
There is no ordering guarantee between the two queues. If oneDNN reads the staging buffer before the conversion is complete, uninitialized pool memory may result in NaN values when interpreted as FP16.

When the SDPA is submitted to the dnnl stream and the permute is submitted to a separate ggml queue, the permute may observe outf in one of the following three states:

• not yet written: stale pool memory -> may decode as NaN FP16
• partially written: a mix of valid results and garbage
• results of a previous FA call: valid FP16 but wrong (silent corruption)

Additional information

Hardware

• GPU: Intel Arc Pro B70 (Battlemage BMG G31, device 0xe223), 32GB GDDR6
• (2nd GPU: Intel Arc A380) not used for this reproduction, excluded via ZE_AFFINITY_MASK=0
• CPU: AMD Ryzen 9 9950X3D

Software

• OS: NixOS 26.11 (kernel 7.1.3)
• GPU driver (compute-runtime): 26.22.38646.4
• Level Zero loader: 1.31.0
• oneDNN: v3.11.2 (commit e1f4a61 with oneAPI toolkit 2026.0.0.198)
• Compiler: IntelLLVM (icpx) 2026.0.0

llama.cpp build

• Base: master 635cdd5
• SYCL backend: GGML_SYCL=ON, GGML_SYCL_TARGET=INTEL

Reproduction model

• unsloth/Qwen3.5-0.8B-GGUF:Q8_0

This issue might be relevant.
#25761

There is a slight (~3%) performance degradation compared to the unsynchronized path, but that path is computing garbage. The fixed XMX path is still ~15% faster than the fallback (XMX disabled).

Executed command

cf. https://gist.github.com/hibiday/16aedea5644309d951ccc15145d75f20

set -euo pipefail

nix build --impure --file build-old.nix -o result-old
nix build --impure --file build-fixed.nix -o result-fixed
realpath result-old result-fixed

export OCL_ICD_VENDORS=/run/opengl-driver/etc/OpenCL/vendors #maybe NixOS only
export ZE_AFFINITY_MASK=0 #My env has pair of GPU, use single one.

M=~/.cache/huggingface/hub/models--unsloth--Qwen3.5-0.8B-GGUF/snapshots/6ab461498e2023f6e3c1baea90a8f0fe38ab64d0/Qwen3.5-0.8B-Q8_0.gguf #replace with any GGUF model path

#example.md is from https://raw.githubusercontent.com/ggml-org/llama.cpp/788e07dc91d266ad3162a1ce9037665656269689/docs/build.md. You can replace any long text.
echo "XMX disabled"
GGML_SYCL_FA_ONEDNN=0 ./result-old/bin/llama-perplexity -m $M -ngl 99 -fa on -c 4096 -f example.md --chunks 8 || true
GGML_SYCL_FA_ONEDNN=0 ./result-old/bin/llama-bench -m $M -ngl 99 -fa 1 -p 2048 -n 0 -r 3

echo "XMX enabled not fixed"
./result-old/bin/llama-perplexity -m $M -ngl 99 -fa on -c 4096 -f example.md --chunks 8 || true
./result-old/bin/llama-bench -m $M -ngl 99 -fa 1 -p 2048 -n 0 -r 3

echo "XMX enabled fixed"
./result-fixed/bin/llama-perplexity -m $M -ngl 99 -fa on -c 4096 -f example.md --chunks 8 || true
./result-fixed/bin/llama-bench -m $M -ngl 99 -fa 1 -p 2048 -n 0 -r 3

Command output

/nix/store/jaj5p1z6px5g9zy2gqaln0wax2cflg70-llama-cpp-sycl-9925
/nix/store/iyiyfqqn58pgzbjg92ww5jrrm4p5s29y-llama-cpp-sycl-9925
XMX disabled
0.02.281.401 I
0.02.281.440 I system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CPU : LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.281.443 I perplexity: tokenizing the input ..
0.02.291.953 I perplexity: tokenization took 10.505 ms
0.02.291.995 I perplexity: calculating perplexity over 2 chunks, n_ctx=4096, batch_size=2048, n_seq=1
0.03.885.625 I perplexity: 1.59 seconds per pass - ETA 0.05 minutes
[1]7.5540,[2]7.1571,
0.04.712.563 I Final estimate: PPL = 7.1571 +/- 0.30500

| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35 0.8B Q8_0               | 763.78 MiB |   752.39 M | SYCL       |  99 |   1 |          pp2048 |       9990.29 ± 3.44 |

build: master-635cdd5 (9925)
XMX enabled not fixed
0.02.222.047 I
0.02.222.087 I system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CPU : LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.222.090 I perplexity: tokenizing the input ..
0.02.232.643 I perplexity: tokenization took 10.549 ms
0.02.232.685 I perplexity: calculating perplexity over 2 chunks, n_ctx=4096, batch_size=2048, n_seq=1
0.03.718.863 I perplexity: 1.49 seconds per pass - ETA 0.03 minutes
[1]-nan,[2]-nan,
0.04.436.360 E Unexpected negative standard deviation of log(prob)

| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35 0.8B Q8_0               | 763.78 MiB |   752.39 M | SYCL       |  99 |   1 |          pp2048 |     11846.68 ± 43.53 |

build: master-635cdd5 (9925)
XMX enabled fixed
0.02.230.503 I
0.02.230.545 I system_info: n_threads = 16 (n_threads_batch = 16) / 32 | CPU : LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.02.230.548 I perplexity: tokenizing the input ..
0.02.241.063 I perplexity: tokenization took 10.511 ms
0.02.241.116 I perplexity: calculating perplexity over 2 chunks, n_ctx=4096, batch_size=2048, n_seq=1
0.03.745.766 I perplexity: 1.50 seconds per pass - ETA 0.05 minutes
[1]7.5567,[2]7.1586,
0.04.477.214 I Final estimate: PPL = 7.1586 +/- 0.30513

| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35 0.8B Q8_0               | 763.78 MiB |   752.39 M | SYCL       |  99 |   1 |          pp2048 |     11492.11 ± 23.82 |

build: 07cfbdd-fix (9925)

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES
  While attempting to speed up Qwen3.6 with Claude Code (Fable 5), I encountered a NaN issue in llama.cpp's SYCL backend. Fable 5 identified a queue synchronization oversight as the cause and provided reference code. I myself performed the verification and the fix, auditing all the code in the Gist to ensure the problem was resolved.

[hibiday]

Closing in favor of #25741, which identifies the correct root cause (pool buffers freed at host return while the GPU is still executing).
After sleeping on it, I realized my hypothesis was wrong: that the dnnl stream and the ggml queue could race intra-op.

• The two syncs added here only masked the pool-reuse race by draining the queue before host return.
• stream_dnnl() wraps the very same queue via dnnl::sycl_interop::make_stream, and that queue is always in-order.
• oneDNN checks queue.is_in_order() and executes in in-order mode.

So the order of cont_to_f16 → execute → permute was already guaranteed.

Note: the ordering between cp.execute(strm, ...) and permute is only guaranteed with an in-order queue. If you use an out-of-order queue, you need to pass event dependencies explicitly (sycl_interop::execute).