feat(qwen36): Qwen3.6-35B-A3B engine (CPU): hybrid Gated Attention + Gated DeltaNet + streaming MoE

[kreuzzelg]

Summary

The idea of this and next PR is to optimize loading experters (for MoE model) from RAM to VRAM when the VRAM is limited, so that for example the "Qwen3.6:36b" model can run at a single GPU with 8 GB VRAM with acceptable performance.

Measured (Threadripper 3945WX 12C, RTX 3070 8 GB + Quadro RTX 4000 8 GB, 200-token decode)

configuration	decode tok/s	VRAM hit rate	peak RSS
CPU only	~2.5	–	~13 GB
1× 8 GB GPU	9.9	95 %	40 GB
2× 8 GB GPUs	11.3	100 %	29 GB
Ollama 2x 8GB GPUs	10.5	–	~40 GB

Relationship to #602: the engine core (hybrid forward, container format, converter, tiny/oracle validation) originates from @minne100's PR; this PR retargets it to dev, adds substantial CPU performance work and converter fixes, and drops the Vulkan/serve parts per review guidance.

Self-contained engine for Qwen3.6-35B-A3B (35B total / 3B active, 256 experts/layer, Apache-2.0). This is the "engine first" slice requested in the #602 review (thanks for the guidance!): no shared GLM/Inkling files are touched, no GPU backend is included — those come as separate follow-ups.

What's included

• c/qwen36.c — 40-layer hybrid forward: 10x Gated Attention (GQA, partial RoPE rope_dim=64, per-head q/k RMSNorm, output gate) + 30x Gated DeltaNet (causal depthwise conv ring + recurrent gated delta rule), streaming MoE with per-layer LRU expert cache + shared expert, PILOT router-lookahead prefetch, COLIBRI_RESIDENT pinning, per-phase timers (COLI_TIMERS=1).
• CPU performance: optional per-row int8 quantization of the large dense matrices (COLI_DENSE_I8=1 default; f32 originals freed; =0 restores the bit-exact f32 reference path), hand-written AVX2/FMA int8 GEMV, DeltaNet recurrence parallelized per value head. ~7x vs the scalar baseline.
• c/tools/convert_qwen36.py — HF→container converter (true int4, per-row scales, --selftest). Containers now run out of the box: flat config.json (VL checkpoints nest dims under text_config) and bundled tokenizer.json.
• c/tools/make_qwen36_tiny.py, make_qwen36_oracle.py — weight-free validation without the 70 GB checkpoint.
• docs/qwen36-phase01.md, qwen36-phase02.md.

Validation

• converter --selftest PASS (int4 pack/unpack exact)
• tiny-model logits bit-identical to the validated reference (cosine 1.0)
• real 35B int4 container: coherent generation; dense-int8 logits cosine
  0.9992 vs f32 reference (COLI_DENSE_I8=0 → bit-exact)

Try it

A pre-converted, self-contained int4 container (tokenizer + flat config bundled, produced with this PR's converter) is available: https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4

Notes

• The earlier published i4/i8 HF containers predate the converter fixes: they lack tokenizer.json and ship the nested VL config (workaround: TOK= and a flattened config.json; new conversions are self-contained).

Thanks on Claude Code.