perf(cuda): incremental KV F16->F32 staging for full-attention layers (depth-dependent decode cost) #182

Background

Continuation of the depth/attention-cost investigation (issues #157-#181, all merged to dev).
See .docs/handoff.md's "Depth-dependent attention finding, precisely localized" section
(2026-07-25) for the full characterization.

Profiling DOTLLM_HYBRID_PROFILE=1 at --depth 0 vs --depth 256 (real Bonsai-27B bench)
found two attention categories grow with context depth. attn-6c-core (+151%, the naive
non-flash attention kernel itself) is architecturally inherent and already tracked as a big
"Future Work" item in docs/CUDA.md (flash attention rewrite) -- out of scope here.

attn-6b-kvdequant (+11%, 0.065ms -> 0.073ms/call) is a real, fixable inefficiency: every
single decode step in ForwardFullAttnBody
(src/DotLLM.Cuda/Architectures/CudaQwen3HybridDenseTransformerModel.cs) calls
LaunchConvertF16ToF32 over the ENTIRE cached KV range (seqKv * kvElems, position 0 to
current depth) even though only ONE new row was appended since the previous call.
EnsureF32KvReadStaging only handles buffer growth (doubling) -- it never tracks how much of
the buffer is already validly converted, so full reconversion happens unconditionally every
decode step: an O(depth) cost paid every step for an O(1)-sized actual update.

Task

Make the F16->F32 KV staging conversion incremental: convert only the newly-appended row(s)
each call, keeping previously-converted F32 values in place, instead of reconverting the whole
[0, seqKv) range from scratch every step. Turns an O(depth) per-decode-step cost into
effectively O(1) for the single-token decode case.

Key design considerations (full detail in .docs/handoff.md's dispatch writeup):

• The current _f32KvReadStagingK/_f32KvReadStagingV are a SINGLE shared buffer pair reused
  across all 16 full-attention layers within one decode step -- this needs to become per-slot
  (per _kvSlotForLayer value) so each layer's incrementally-converted history doesn't get
  clobbered by the next layer's slot.
• Must safely detect and fall back to full reconversion whenever the "simple contiguous append"
  invariant doesn't hold (buffer growth/reallocation, non-contiguous positions, a KV-cache
  handle reused for an unrelated sequence without a length-triggered reset, etc.) --
  correctness over cleverness.
• Prefill (seqLen>1) and single-token decode (seqLen==1) go through the same call site; the
  fix must be correct and non-wasteful for both, not decode-only.

Acceptance criteria

• Real dotllm bench A/B at meaningful depth (e.g. --depth 1024) shows a measurable decode
  win, not just a profiler-modeled one.
• New/extended correctness test proves the incremental path is bit-exact vs. the old
  full-reconversion behavior across many consecutive decode steps at real depth (not just
  depth 0), including at least one buffer-growth event mid-sequence.
• Full DotLLM.Tests.Unit.Cuda suite passes.
• If the real-bench win doesn't materialize or a correctness issue can't be resolved cleanly,
  revert and document why (matches this investigation's established discipline -- several prior
  issues in this thread concluded negative and were reverted with full documentation).