MiMo-V2.6: 9B works today; Flash (mimo2, 310B MoE) feasibility, gap list and plan #935

Tracking issue for MiMo-V2.6 (Xiaomi, MIT). Full write-up: local .docs/MIMO_V26_FEASIBILITY_2026-10-12.md (git-ignored). Evidence tags: [M] measured here, [F] read from HF config / GGUF header / llama.cpp source, [E] estimate. Nothing for Flash is implemented or downloaded.

1. 9B (MiMo-V2.6-Distill-Qwen-9B): works today, no code needed [M]

GGUF arch qwen35 -> Qwen3HybridDense (Qwen3.5-9B base). bartowski Q4_K_M, CPU + Vulkan, serve chat API, MiMo's own chat template (renders; generation tags already tolerated), no BOS, eos <|im_end|>.

• 14-prompt suite (arithmetic, JSON, code, reasoning, single/parallel tool calls, tool-result round trip, multi-turn, thinking on/off, 1872-token needle, streaming): CPU == Vulkan on everything except one near-tie first-token flip; prompt token counts identical to llama.cpp on all 14. llama-server's chat endpoint returns only the first of two parallel tool calls (its raw completion emits both); dotLLM returns both.
• PPL (1020 scored tokens, shared ids): dotLLM CPU 5.5463 / Vulkan 5.5553 / llama.cpp CPU 5.5888 / Vulkan 5.5718. dotLLM CPU-vs-Vulkan mean KL 0.0099 (median 0.0031), top-1 96.6 %. OmniCoder-9B control shows the same noise vs llama.cpp (PPL -0.25 % vs MiMo -0.76 %, top-1 97.5 % vs 95.8 %).
• Same-session Vulkan vs llama.cpp b8683 (Edge was open; llama.cpp ranges): pp512 986-1011 vs 661-725, pp2048 898-936 vs 713-722, tg 28.7-31.4 vs 29.2-30.6 (parity).
• Gap filed: Chat template override missing (--chat-template-file): cannot use ggml-org's patched MiMo template / prefilled <think> #934 (no chat-template override).

2. Flash (mimo2, 309B / 15B active) feasibility

Plain attention + MoE, not a hybrid: 48 layers = 9 global + 39 SWA (window 128, per-head attention sinks on SWA layers only); K head_dim 192 but V head_dim 128; KV heads 4 (global) / 8 (SWA); fused attn_qkv; rotary on the first 64 of 192 dims (theta 1e7 global / 1e4 SWA); V scaled by 0.707; layer 0 dense FFN 16384, layers 1-47: 256 experts top-8, expert ff 2048, sigmoid + selection bias, renormalised, no shared expert; vocab 152576, no BOS; 3 MTP layers in a separate GGUF (SWA + dense FFN + eh_proj; ~1.1 GB reusing trunk emb/lm_head); vision/audio in mmproj (ignore); DFlash drafter is a different artifact (ignore).

What fits the ~77 GiB trunk budget: patrickbdevaney REAP50 Q2_K 61.6 GiB (128/256 experts, gate/up Q2_K, down MXFP4, attn Q8_0; ~10 GiB spare with MTP + KV; quality UNVERIFIED - the card has only calibration saliency retention, no PPL/benchmarks, MTP trained on unpruned experts) and AesSedai BPW2.0 72.07 GiB (full experts IQ1_M/IQ2_XXS, author's PPL 6.545 vs 5.147 = +27 %, KLD 0.37; MTP + scratch marginal). Everything else is 76-163 GiB. KV is cheap: 22.5 KiB/token F16 for the 9 global layers + a ~250 MB SWA ring (a sliding-window ring is required: without it SWA costs 400 KiB/token F32). Bandwidth ceiling REAP50 Q2_K ~19-23 tok/s [E]; attention at Q8_0 is >50 % of bytes/token (requant at load is the first lever).

Reuses: gpt-oss (sink + SWA attention, MXFP4 banks/kernels), GLM-4.7-Flash (sigmoid+bias MoE, leading dense layer, nextn), Gemma-4 (per-layer KV heads / theta / window), qwen4exp + Qwen3-hybrid MoE (packed Q2_K/IQ expert kernels, fused MoE chain, MTP attach/gate/cost model, residency guard).

3. Plan / gap list (checklist)

• P0 (free, no big download) G1: mimo2 arch id + Architecture.MiMoV2 + config extractor + tensor mapping + synthetic-GGUF builder (like SyntheticQwen4ExpGguf); repo checklist: docs/MODEL_CONFIG.md section, docs/SUPPORTED_MODELS.md row, docs/QUANT_FIXTURES.md env var + scripts/parity_sweep_models.json entry. Tiny-random MiMoV2 (HF remote code, 4-6 layers) -> llama.cpp conversion/mimo.py -> GGUF; three-way oracle HF torch / llama.cpp CPU / dotLLM CPU. (needs a torch+transformers env: unverified on this box)
• P1 CPU G2(CPU) GQA with V dim != QK dim + KV layout; G4 per-layer KV heads / theta / partial NEOX rope / V scale; G5 sigmoid-bias top-8 no shared expert + dense layer 0; G7 fused qkv split. Validate REAP50 Q2_K vs llama-perplexity -dev none (b8683 contains the mimo2 arch string; load not verified).
• P2 Vulkan G2 shaders (hd192 QK / hd128 V: coopmat flash is gated at hd<=128 today; split-KV decode; sink variant), G3 SWA ring KV ([perf-survey] Dense Vulkan KV cache is F32, eager and full-length: Llama-8B decode 32 -> 21.5 tok/s at depth 8192 (llama.cpp 29-31), Gemma-4 26B 34.6 -> 25.7 at 6K #918), G6 expert-bank quant coverage in the generic MoE path (Q2_K / IQ1_M / IQ2_* banks currently widen to F32; the qwen3-hybrid loader already keeps them packed), MMVQ arm for Q2_K / IQ1_M (IQ1_M decode MMVQ UNVERIFIED). Depends on [perf-survey] Vulkan generic MoE decode has no MMVQ arm: GLM-4.7-Flash tg 6.8 vs llama.cpp 38-45, gpt-oss-120b tg 12 vs 40 #912/[perf-survey] gpt-oss sink attention scans every KV tile and has no split-KV/flash path: tg 40 -> 22 at depth 1024, 26% of pp512 #915 landing (MXFP4 MMVQ + split-KV sinks are on branch issue/912-generic-moe-decode-mmvq).
• P3 serving chat/tool/reasoning on the real model (same template family as the 9B), long-context story (F16 KV + ring: 128K ~ 2.8 GiB).
• P4 MTP G8: 3 chained heads, separate file, shared emb/lm_head; acceptance on REAP50 unknown.
• P5 perf attention requant Q8_0 -> Q5_K/Q4_K at load, host-side embedding gather ([perf-survey] Vulkan keeps a full vocab x hidden F32 device embedding table for Q4_K/Q6_K embeds (Llama-8B 2.0 GiB = 43% of the file) #919), profiling.

Effort [E]: ~2-3 agent-weeks to validated CPU + Vulkan decode, ~4-5 with MTP and tuning. Highest risk G2 (new attention geometry across three shader families).

4. Download plan (needs user OK first; nothing > 5 GB fetched for Flash so far)

1. P0 first: no big download.
2. Then patrickbdevaney/MiMo-V2.6-Flash-REAP50-GGUF ...-Q2_K.gguf (66.2 GB) + ggml-org/MiMo-V2.6-Flash-RL-GGUF mtp-...-Q8_0.gguf (2.4 GB) via hf download into the hub cache. C: has ~102 GB free (95 % used) -> ~33 GB left; droppable blobs per the handoff if more is needed.
3. If REAP50 quality disappoints on our PPL/task check, swap for AesSedai BPW2.0 (77.4 GB; cannot coexist with 2 on C:). Skip files >= 98 GiB.
4. GPU runs only under scripts/gpu-lock.sh; at 62-67 GiB resident no other large GPU job fits alongside.

Related: .docs/MODEL_COVERAGE_PLAN.md row 12 had rated this P2/skip ("Vulkan MXFP4 missing") - that premise is addressed by the #912 branch.