Extended SYCL oneDNN SDPA to non-FP16 KV caches (Q4_0-Q8_0 and FP32) - #1

Open

johnkarlhill wants to merge 411 commits into

sycl-onednn-fa-quants

This pull request is big! We're only showing the most recent 250 commits

Commits on Jul 11, 2026

opencl: add int8 dp4 dense and MoE prefill optimization for Adreno GPUs (#25537)

wanghqc
and
lhez
authored
1d1d9a9
Vulkan: route large matmuls to medium tile on Adreno (#24877)
Raman-Raje
authored
76f2798
ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer (#24231)

fairydreaming
and
sszymczy
authored
00f5442
server: refactor server_stream (#25541)

ngxson
and
ServeurpersoCom
authored
ea1f7bb
server: allow stream for exec_shell_command (#25526)
ngxson
authored
c92e806
DeepseekV4: clear cache only for seq rather than full (#25521)
am17an
authored
13f2b28
cuda: Don't crash when querying memory on device with no free memory. (#25157)
cphlipot
authored
3cec3bc
gguf : reject empty metadata keys (#24917)
rohitmahesh1
authored
d72bfa3
Fix conditional to display 'LLAMA_SPLIT_MODE_TENSOR not implemented for architecture' message (#24926)
kdkd
authored
e3546c7

Commits on Jul 12, 2026

Commits on Jul 13, 2026

ui: Remove recommended MCP Servers + improve MCP Servers Settings UI/UX (#25535)
allozaur
authored
38fd5c9
sycl: add Q2_K to DMMV reorder path (#25064)
malsbat
authored
e474bba
sycl: add fused top-k MoE (#25217)

newjordan
and
claude
authored
efb3036
chat : fix reasoning leak with force-opened bare <think> templates (#24674)

newjordan
and
aldehir
authored
91c631b
gguf : add tensor shape accessor (#24405)
QuintinShaw
authored
ad8d821
tests: Harmonize header use (#25616)
ckastner
authored
f4253ef
readme : add link to maintainer PRs (#25621)
ggerganov
authored
4193ea6
spec: add Minimax2 eagle3 support

adrianisk
and
ggerganov
authored
259ae1d
vulkan: Use native e2m1 and e4m3 conversions for mxfp4/nvfp4 (#25338)
jeffbolznv
authored
e920c52
CUDA: refactor MMQ kernel configuration (#24127)
JohannesGaessler
authored
6eddde0
model: add Hy3 (hy_v3) support with MTP speculative decoding (#25395)

authored
2969d6d

Commits on Jul 14, 2026

Commits on Jul 15, 2026

Commits on Jul 16, 2026

Commits on Jul 17, 2026

Commits on Jul 18, 2026

Commits on Jul 20, 2026

Commits on Jul 21, 2026

Commits on Jul 22, 2026

Add support for Laguna XS.2 & M.1 (#25165)
joerowell
authored
1f66c3c
llama-arch: fix DeepSeek4 APE tensor op (#25945)
helanfxz
authored
67b9b0e
cuda: GET_ROWS quants (#25962)
ServeurpersoCom
authored
c5a4a0b
webgpu : add CONV_2D_DW (depthwise conv2d) kernel (#25847)
m1el
authored
7347430
ci : fix SYCL package shared library lookup (#25987)
KyleHagy
authored
b4d6c7d
ggml: enable PowerPC backend variants on AIX (#25983)
shalinib-ibm
authored
3ce7da2
Fix DeepSeek4 crafted template (#25414)
pwilkin
authored
f534da2
common: infer the speculative type from the draft repo sidecars (#25989)
ServeurpersoCom
authored
6d5a910
minor: fix reasoning preserve var for DS4 [no ci] (#25999)
pwilkin
authored
e8e6c7a
feat(ui): add symbolic math support to JS sandbox via nerdamer (#25948)

rankaiyx
and
ServeurpersoCom
authored
61328e6
mtmd: use RAII for setting and resetting non-causal attention (#25723)
nikwen
authored
e0833bf
hexagon: activation ops update (#25974)

tboinovski1
and
max-krasnyansky
authored
0278d83
CUDA: Improve NVFP4 W4A4 activation quantization (#25730)
ORippler
authored
1a064ab
ui: Add a "Default" option for the reasoning selector (#25846)
ServeurpersoCom
authored
cf51256
contrib: allow all AI-generated code in general (#26012)
ngxson
authored
4310aa4

Commits on Jul 23, 2026

Commits on Jul 24, 2026

hexagon: further improved pipeline of the core bits (L2, DMA, MM, FA) (#26049)
max-krasnyansky
authored
0a50d99
vendor: update subprocess.h (#26061)
ngxson
authored
0cea362
cohere2 moe template parser: enforce JSON schema for text responses if a response schema is provided (#26018)
boondocklabs
authored
8f5ab83
UI: Fix settings precedence, Factory < Admin (--ui-config-file) < Users (Settings panel) (#26002)
ServeurpersoCom
authored
54ce507
skill: create `add-new-model` and `code-review` (#26042)

ngxson
and
pwilkin
authored
77095ee
opencl: do not treat NULL-mask flash attention as causal (#25771)
kumaal
authored
56a8386
opencl: cache compiled cl_program binaries on disk (#26050)
wanghqc
authored
ed7adbf
HIP: remove rocWMMA FlashAttention (#26046)
JohannesGaessler
authored
fa72aec
llama: various bug fixes (#26051)
ngxson
authored
298219f
server: support "reasoning_effort": "none" in OAI API (#26045)
pnb
authored
27209a5
ui: fix MCP server display name conflicts in tools lists (#26011)
ServeurpersoCom
authored
95a923a
model: add GLM 5.2 Indexer support (#25407)

pcuenca
and
fairydreaming
authored
88bfee1
ui: remove render effects (#26083)
ServeurpersoCom
authored
96013c5
ui: reduce per-token render cost when streaming (#26053)

Zach Winter
and
zachwinter
authored
555881e

Commits on Jul 25, 2026

Commits on Jul 26, 2026

ggml-webgpu: Fix WASM compilation with OpenMP (#25943)
reeselevine
authored
7cdd557
ui: fix context gauge card regressions and land at the conversation end (#26099)
ServeurpersoCom
authored
ff067f7
opencl: fix fused RMS norm mul view offset (#26085)
happyyzy
authored
42fc243
model: Add MiniMax-M3 (MSA: MiniMax Sparse Attention) support (#24908)

authored
b1d4c65
mtmd: add GLM-5.2-Vision (#26126)

ehartford
and
Eric Hartford
authored
af28502
common: add `subproc.h` wrapper, disabled on android/ios (#26102)
ngxson
authored
d2a8182
ui: detect the conversation import format from file contents (#26121)
ServeurpersoCom
authored
55b7d6c
Keep Minimax's indexer tensors at F32 for speed and accuracy (#26144)
bartowski1182
authored
7657a6c
ui: fix system message edit box not expanding to fit content (#26006)
pieroevcc
authored
d4d057b
mtmd: fix android build (#26150)
ngxson
authored
0d47ea7
mtmd: Add Vision Support for Minimax-M3 (#25113)

authored
3d1c3a8

Commits on Jul 27, 2026

Commits on Jul 28, 2026

Commits on Jul 29, 2026

Commits on Jul 30, 2026

mimo2: address MTP review feedback (#26228)
tnhnyzc
authored
64d528b
[UT] enhance UT to show all real unsupported backends (#25234)

arthw
and
ggerganov
authored
a6a77bc
ggml : Fix issue with kleidiai ci and stringop overflow warning (#26277)
JonathanC-ARM
authored
32703b4
ui: IndexedDB and Conversations data fixes (#26278)
allozaur
authored
21a5f5b
metal: fix memory unwire if model is freed without any GPU operations (#26082)

nikwen
and
forforever73
authored
d0bfb19
Remove custom cpu op from the M3 graph, express with stock ops (#26297)
timkhronos
authored
b2f2216
CUDA: add Q2_0 support (#25707)
khosravipasha
authored
9b2a088
ggml : bump version to 0.18.0 (ggml/1576)
ggerganov
committed
6a4c335
sync : ggml
ggerganov
committed
9ebfc3a
readme : refresh (#26280)
ggerganov
authored
6b36c23
ggml-cuda: Allow transpose-free gemmv computation (#26171)
roberteg16
authored
e1a1abb
tests : avoid building get-model.cpp many times (#26317)
ggerganov
authored
47f686f
llama-context : sync pending async copies before clearing embd_seq (#25676)
o7si
authored
432d7ff
Test support for alternative conv layout (#25617)
pmaybank
authored
958d9c0
vulkan: Support quantized concat (#25684)
jeffbolznv
authored
ea63b4d
server: support inp embd to generate next token (#26313)
ngxson
authored
b4ca032
mtmd: add lanczos resize method [no release] (#26341)
ngxson
authored
5f55650

Commits on Jul 31, 2026