Xe2 performance layer: XMX flash-attention recipes, managed SYCL JIT cache, on-device autotune

[offbyonebit]

What this adds

Four optimizations that, as far as I can find, no other Intel Arc runtime (IPEX-LLM, plain llama.cpp launch scripts, Ollama forks) ships today:

1. XMX flash-attention aware recipes

llama.cpp merged an oneDNN-backed SDPA path on 2026-07-15 (PR #25222) that routes long prefills through the XMX systolic arrays — measured up to 4.26× prefill at 80k ctx on Battlemage. The catch: it only engages with an f16 KV cache; quantized KV silently falls back to shader kernels. That inverts arc-llama's old "prefer q8_0 KV" default on roomy cards.

default_recipe now, on Xe2 (Battlemage / Lunar Lake): picks f16 KV + -fa on + -ub 1024 -b 2048 whenever that still leaves ≥16k context, and keeps q8_0 KV only on VRAM-tight setups where context-per-byte wins. New LaunchRecipe fields: flash_attn, batch_size, cache_reuse, no_mmap.

2. Managed persistent SYCL JIT cache (sycl_cache.py)

Battlemage + oneAPI 2026.0 SIGSEGVs in PersistentDeviceCodeCache::getItemFromDisc on stale cache entries, so everyone (including arc-llama until now) sets SYCL_CACHE_PERSISTENT=0 — paying ~20 s of JIT recompilation on every cold start, forever. Instead:

• each (llama-server binary, GPU driver) combo gets its own fingerprinted cache dir under state_dir/sycl-cache/ — a cache is only ever read by the exact stack that wrote it, so the stale-entry crash has nothing to bite on;
• a warm-up marker acts as a crash guard: a run that dies before its first health check wipes + poisons that cache dir and falls back to =0 automatically. Worst case is the old behaviour; common case pays JIT once per llama.cpp upgrade instead of once per model swap.
• opt out via [server] jit_cache = "off".

3. arc-llama tune — on-device autotune

Sweeps the configs that actually move the needle on Arc (q8 KV baseline vs f16+FA at ubatch 512/1024/2048) through the live arc-llama serve instance, scores each by modelled request latency (prompt/pp_rate + gen/gen_rate), prints a table, and persists the winner into the model's recipe. /admin/models/{name}/edit now accepts batch_size, flash_attn, cache_reuse, and null to clear optional fields.

4. Slow-quant registration warnings

Q8_0 weight quants run at ~22% of memory bandwidth on Xe2 vs Q4_K_M's ~55% (llama.cpp#21517) — 4-5× slower generation. add/scan now warn and suggest Q4_K_M/Q4_K_S/Q6_K.

Plus: non-SWA recipes get --cache-reuse 256 (multi-turn chats stop re-prefilling the shared prefix), and MTP models keep their -ub 8 constraint (batch/FA sweep and fields are excluded for them).

Tests

• tests/test_sycl_cache.py (new): fingerprint stability/rotation, crash-guard poisoning, clean-lifecycle re-enable, pruning.
• tests/test_recipes.py: new flag argv emission, Xe2 f16+FA policy, tight-VRAM q8 fallback, SWA cache-reuse exclusion.
• tests/test_launcher.py: managed cache env wiring, jit_cache="off" opt-out, Alchemist untouched, clean-stop disarms guard.
• tests/test_models.py: slow-quant warning matrix, XMX fields persisted to config.

209 passed, 9 skipped locally.

Needs hardware validation (B60)

• -fa on + f16 KV vs old q8 KV defaults on a real Battlemage build with GGML_SYCL_DNN=ON
• cold-start time with the managed JIT cache (expect ~20 s first launch, ~instant after)
• arc-llama tune end-to-end against a running server

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ezf1mE6Cd15wgCFMCTBZrM

---

Generated by Claude Code