Repository navigation
[Vulkan] Bias-add kernel (bias_add_f32) #7
Description
Context
VulkanTransformerModel.AddBiasRows currently host-maps and adds on CPU. SmolLM has no biases; Phi-3 does and triggers the host fallback — trivially slow.
What
Implement a Vulkan pointwise bias_add_f32 kernel and replace the host fallback.
Why this matters
Avoids device→host→device round-trip per decode step for any model that has projection biases (Phi-3 family and others).
Where to start
src/DotLLM.Vulkan/VulkanTransformerModel.cs— findAddBiasRows.- CUDA reference
native/kernels/bias_add_f32.cu.
Acceptance criteria
- Pointwise kernel with parity test.
- Host fallback replaced.
- Phi-3 argmax-parity integration test added.
Activity
Vulkan: move host-only activation scratch buffers to device-local memory
docs: handoff update — all 4 MLA infra pieces + MoE step 1 + GEMV fix…
Vulkan: add bias_add_f32 kernel + wire it in (issue #7)
jamesburton commentedon Jul 5, 2026
Implemented and merged to dev — native/vulkan/shaders/bias_add_f32.comp exists and is wired (issue #7 referenced directly in its commit history: 416363c "Vulkan: add bias_add_f32 kernel + wire it in (issue #7)").