Step 31: CUDA backend — native C/CUDA library, cuBLAS GEMM, custom kernels, CudaBackend #70

Summary

Implement the CUDA GPU backend for dotLLM — the first GPU acceleration step (Phase 4). This introduces the native C/CUDA shared library (dotllm_native), CUDA kernel implementations for all transformer operations, the CudaBackend implementing IBackend, and P/Invoke interop in DotLLM.Cuda.

Roadmap reference: Phase 4, Step 31
Dependencies: Phase 1–2 (all completed)

Scope

1. Documentation — docs/GPU.md

• Comprehensive GPU backend design document
• Prerequisites (CUDA toolkit, driver versions, supported GPUs)
• Architecture: native library ↔ P/Invoke ↔ C# backend
• Memory management model (device allocator, memory pools, pinned host memory)
• Kernel catalogue and dispatch strategy
• Build system (CMake) and packaging (runtimes/{rid}/native/)
• Integration with existing IBackend/IKernelRunner interfaces
• Performance expectations and measurement methodology

2. Native C/CUDA library (native/)

• Build system: CMakeLists.txt with CUDA toolkit detection, compute capability targets (sm_70+)
• Header: native/include/dotllm_native.h — flat C API, no C++ across boundary
• cuBLAS wrappers (native/src/cublas/):
  • FP32/FP16 GEMM via cuBLAS
  • cuBLAS handle lifecycle (create/destroy per device)
• Custom CUDA kernels (native/src/cuda/):
  • Fused RMSNorm + SiLU kernel
  • RoPE (rotary position embedding) kernel
  • Embedding lookup kernel
  • Element-wise add, multiply kernels
  • Softmax kernel
  • Quantized dequantization kernels (Q8_0, Q4_K_M → FP16)
  • Quantized GEMV for decode (Q8_0, Q4_K)
  • Flash attention kernel (tiled, fused softmax)
  • KV-cache update kernel
• Device management (native/src/cuda/):
  • Device query (count, compute capability, memory)
  • Memory allocation/free (device + pinned host)
  • Stream management (create/destroy/sync)
  • Error handling with descriptive messages

3. C# interop layer (DotLLM.Cuda/)

• P/Invoke declarations: [LibraryImport("dotllm_native")] for all native functions
• CudaBackend: Implements IBackend
  • Device enumeration and selection
  • GPU tensor allocation (cudaMalloc via native)
  • Host↔device copies (cudaMemcpy via native)
  • Stream management wrapper
  • Deterministic disposal (free device memory, destroy handles)
• CudaDevice: Device info (name, compute capability, total/free memory)
• CudaStream: RAII wrapper around CUDA stream handle
• CudaTensor: ITensor implementation for GPU-resident data (opaque nint device pointer)
• CudaKernelRunner: Implements IKernelRunner — dispatches to native kernels

4. GPU-aware model forward pass

• Adapt TransformerModel or create GPU variant that dispatches via IKernelRunner
• Weight upload: load from mmap'd GGUF → cudaMemcpy to device at model load time
• GPU KV-cache: allocate and manage on device
• All intermediate tensors on GPU — no device↔host transfers during forward pass
• Logits transfer back to host only for sampling

5. CLI integration

• --device gpu / --device cpu flag
• --gpu-layers N preparation (for Step 32)
• Device info display on startup

Acceptance Criteria

• docs/GPU.md comprehensive design document
• Native library builds via CMake on Linux (Windows optional initially)
• dotllm_native.h defines complete flat C API
• cuBLAS GEMM works for FP32 and FP16
• Custom kernels: RMSNorm, SiLU, RoPE, softmax, add, multiply, embedding
• Flash attention kernel (at minimum, naive GPU attention)
• Q8_0 and Q4_K dequantization on GPU
• CudaBackend implements IBackend with device memory management
• CudaKernelRunner implements IKernelRunner
• End-to-end: load GGUF model → run forward pass on GPU → generate text
• Numerical accuracy within ε of CPU output for same prompt
• GPU decode throughput within 2× of llama.cpp for equivalent model/quantization
• Unit tests for P/Invoke layer and GPU tensor operations
• Integration test: GPU vs CPU output comparison

Key Design Decisions to Make

1. FP16 vs FP32 compute: Use FP16 (half) for GPU compute? cuBLAS FP16 GEMM is ~2× faster on Tensor Cores.
2. Memory pool: Custom CUDA memory pool vs cudaMalloc/cudaFree per tensor?
3. Stream strategy: Single stream per device vs multi-stream overlap?
4. Quantized matmul: Dequantize-then-cuBLAS vs custom quantized GEMV kernels?
5. Flash attention: Custom kernel vs cuDNN flash attention?

Reference

• llama.cpp ggml-cuda/ for proven kernel implementations
• docs/ARCHITECTURE.md — system architecture
• docs/MULTI_GPU.md — GPU interop patterns (NCCL API, P/Invoke conventions)
• docs/QUANTIZATION.md — quantization block layouts
• CLAUDE.md — GPU interop rules, native API conventions