[Track B] track ggml-org/llama.cpp#22105 (DFlash drafter) for castle adaptation

[Leechael]

Tracking: #3 (Phase 2)

Purpose

Mirror of ggml-org/llama.cpp#22105 (DFlash drafter by ruixiang63), evaluated on castle hardware as an upstream-tracking alternative to the castle's own DFlash + DDTree implementation.

Current state

Branch starts identical to upstream-pr/22105 head (67cb0d507):

• 67cb0d507 dflash: enable llama-cli & llama-server with np=1
• e344c4a71 dflash: remove rebundant logic & correct bias naming
• 85a0089e6 dflash: add support for qwen3.5/3.6 moe models
• (plus the rest of [Speculative decoding] feat: add DFlash support ggml-org/llama.cpp#22105's stack on EAGLE3 [Speculative decoding] feat: add EAGLE3 speculative decoding support ggml-org/llama.cpp#18039)

No castle-side adjustments applied. Stock ggml-org#22105 was sufficient to run end-to-end on castle hardware.

Castle smoke test (single prompt)

build  : build-track-b/ (cmake + CUDA, sm_89, RTX 4090)
target : Qwen3.5-27B-Q4_K_M.gguf (16 GB)
draft  : dflash-22105.gguf (3.46 GB, bf16, converted from DFlash-draft-hf via #22105's convert_hf_to_gguf.py with --target-model-dir)
prompt : "Write a quicksort algorithm in Python. Write code only."

decode_tps   = 99.1
accept_pct   = 67.8% (61/90)
n_predict    = 68
graphs reused = 3 (decoder graph rebuild — known #22105 future work)
GPU memory   = 24079 = 3446 free + (16034 model + 213 ctx + 548 compute) + 4598 unaccounted

30-prompt benchmark (HumanEval / GSM8K / Math500, 10 each, seed=42)

Same dataset and sampling as the castle self-implementation benchmark in docs/ddtree-dataset-eval-plan.md. KV q8_0, n-batch=2048, n-ubatch=512 (ggml-org#22105 has no --prompt-chunk flag — large batch chosen so prompt fits in a single ubatch).

dataset	n	avg decode tok/s	avg accept %
HumanEval	10	91.11	54.13
GSM8K	10	89.47	56.05
Math500	10	93.42	55.55
avg	30	91.3	55.2

Comparison vs castle self-implementation (numbers from existing castle docs)

stack	avg tok/s	bit-equal	speedup vs AR
AR baseline	46.4	10/10 (by definition)	1.0x
castle self-impl exact-gated	~40	10/10	0.87x
upstream ggml-org#22105 stock	91	bit-eq via ggml-org#19493 checkpoint	2.0x
castle self-impl + TARGET_TOP1	123	4.7/10	2.7x
castle self-impl + unsafe-trust-batched	137	4.7/10	3.0x

Interpretation

Upstream ggml-org#22105 stock sits between the two ends of the castle self-implementation:

• 33% slower than castle's unsafe-trust-batched (which sacrifices bit-equal correctness)
• 2.3× faster than castle's correctness-gated path (castle's exact-gated is validate_tree_with_chain — runs N single-token chain decodes per step, effectively degenerates spec into AR plus overhead)

Why ggml-org#22105 is fast under correctness gate: it leans on upstream #19493 speculative checkpointing (merged 2026-04-19). Rejected suffixes are rolled back via KV checkpoints, so the accepted path runs forward exactly once. Castle's self-implementation predates ggml-org#19493 and replaced it with a per-token chain validation walk.

Trade-off

• Accept bit-equal loss → castle's unsafe path still wins (137 tok/s).
• Require correctness gate → use [Speculative decoding] feat: add DFlash support ggml-org/llama.cpp#22105 (91 tok/s, 2.3× faster than castle's correctness path).

Status

This branch will be kept in sync with upstream-pr/22105 as ruixiang63 evolves it (waits on ggml-org#18039 EAGLE3 merge + ggerganov's unified spec API refactor).

No active development planned on top of this branch. If we later decide to port castle DDTree on top of ggml-org#22105 (to combine bit-equal + faster speedup), it will become a new branch (track-b/ddtree-on-22105 or similar).

Castle artifacts produced during this evaluation

• ~/workshop/lucebox-hub/dflash/deps/llama.cpp/build-track-b/ — CUDA build
• ~/workshop/lucebox-hub/dflash/models/draft/dflash-22105.gguf — DFlash GGUF re-converted with [Speculative decoding] feat: add DFlash support ggml-org/llama.cpp#22105's converter (architecture string is dflash, not the castle-specific dflash-draft)
• ~/workshop/lucebox-hub/dflash/deps/llama.cpp/.venv-track-b/ — Python venv for the convert script
• ~/workshop/lucebox-hub/dflash/deps/llama.cpp/scripts/bench_track_b.py — bench harness adapted for [Speculative decoding] feat: add DFlash support ggml-org/llama.cpp#22105 (parses llama-speculative-simple --dflash stdout)
• /tmp/bench-track-b-v2/ on castle — bench logs and results.json