feat(models): dense Qwen3.5 hybrid architecture (GDN + attention, no MoE) for Bonsai-27B #157

Context

Depends on #151 (PQ2_0 quant type, merging via PR #153). PrismML's Bonsai-27B (general.architecture = qwen35, dense — no MoE experts) currently fails to load: GgufModelConfigExtractor.ParseArchitecture only recognizes "qwen35moe", and the existing Qwen3MoeHybrid architecture is MoE-hardcoded (required MoeLayerWeights field, unconditional expert-tensor lookups) with no dense fallback.

Confirmed via real GGUF metadata + existing code: Bonsai's GDN config (state_size=128, conv_kernel=4, group_count=16, inner_size=6144, time_step_rank=48, full_attention_interval=4) maps exactly onto the existing GatedDeltaNetConfig/TryExtractGdnConfig extraction already used by Qwen3MoeHybrid — the token-mixing recurrence is identical, only the FFN body (dense SwiGLU vs MoE routing) differs.

Scope (this issue)

• CPU: config extraction (Architecture.Qwen3HybridDense, ParseArchitecture arm for "qwen35"), new model class family (Qwen3HybridDenseTransformerModel/LayerWeights/ForwardState) adapted from Qwen3MoeHybridTransformerModel — GDN/attention/RoPE lift near-verbatim, MoE FFN body replaced with dense SwiGLU (gate/up/down), PQ2_0 dispatch arm added alongside I2_S.
• CUDA: forward-pass implementation mirroring the CPU model, for real end-to-end benchmarking on the RTX 3060 (Bonsai's PQ2_0 weights are ~7.2GB, fits in 12GB VRAM).
• Vulkan: explicitly out of scope for this issue — follow-up once CPU+CUDA are validated.

Acceptance criteria

1. Config-extraction unit tests: general.architecture=qwen35 routes to Architecture.Qwen3HybridDense, GDN config values match Bonsai's real metadata exactly.
2. CPU construction test: model loads from the real Ternary-Bonsai-27B-Q2_0.gguf without throwing (weights allocate, layer count/dims match).
3. CUDA real-weight forward-pass smoke test: short deterministic prompt, no NaN/Inf through all layers.
4. Baseline benchmark: tok/s (prefill + decode) on the RTX 3060, real Bonsai weights.

Verify pre-FFN norm tensor name (ffn_norm.weight vs post_attention_norm.weight) against the actual GGUF before assuming either convention.

🤖 Generated with Claude Code