Add KV cache state extraction/restoration to quantized_qwen2 and quantized_qwen3_moe

[alytaphoenix]

Adds explicit KV cache state extraction/restoration to quantized_qwen2::ModelWeights and quantized_qwen3_moe::GGUFQWenMoE, plus a clear_kv_cache on the latter (mirroring the method already present on quantized_qwen2/quantized_qwen3/quantized_llama).

Why

Both model types previously exposed no way to get attention state out of a live model or put a previously-saved state back in — only an implicit reset via index_pos == 0 on the next forward. That's enough for the common single-conversation server loop, but not for a caller that needs to persist a session's KV state across process restarts, or swap which conversation a resident model is serving without reprocessing every token from the start of the conversation.

What

Both new methods share the same shape and pairing convention so callers don't need per-architecture branching:

pub fn kv_cache_state(&self) -> Vec<Option<(Tensor, Tensor)>>;
pub fn set_kv_cache_state(&mut self, state: Vec<Option<(Tensor, Tensor)>>) -> Result<()>;

One entry per layer, in layer order. None means that layer hasn't been forwarded through yet. set_kv_cache_state errors (rather than silently partial-restoring) on a layer-count mismatch, since mismatched state from a different model/architecture should be rejected loudly.

GGUFQWenMoE::clear_kv_cache follows the existing quantized_qwen2/quantized_qwen3 pattern (iterate layers, clear the K/V pair) — it was simply missing for the MoE variant.

No behavior change for existing callers; all three methods are additive.

Testing

cargo build -p candle-transformers --features metal passes clean. This has also been exercised live in downstream code: extracting state after a multi-turn conversation, restoring into a freshly-loaded model, and confirming the restored KV tensors are bit-identical to the originals before continuing generation.

[ivarflakstad]

Code looks good.

Please remove the comments. Not only are they redundant, which already means they shouldn't be there - they are also incorrect.

For example

/// Resets every layer's KV cache -- for reusing one loaded model across
/// independent requests without reloading weights. Added for ratatoskr,
/// which serializes model access behind a mutex (see ratatoskr's
/// src/model/mod.rs); ratatoskr now also tracks warm sessions and only
/// calls this when actually starting a new/different conversation, not
/// unconditionally on every request.

We're not adding this for ratatoskr. We're adding it because we want to be able to clear the kv cache, regardless of model or system using candle.

This happens because the LLM you've used includes the implicit bias of the prompt / project in it's work.

Something to consider for future contributions as well ☺️

[alytaphoenix]

You're right, and sorry for the noise -- I should have caught that before pushing. Removed the ratatoskr-specific framing from both doc comments; they now just describe what the code does for any caller, matching the style of quantized_qwen2's equivalent functions. Will be more careful about this in future contributions. Thanks for catching it.

[alytaphoenix]

Thanks, and sorry for the delay — the earlier commit here only stripped the project-specific framing, which was narrower than what you asked for. The comments themselves were still in place, including the clear_kv_cache one you quoted.

All 30 doc comments this PR added are now removed, across both quantized_qwen2.rs and quantized_qwen3_moe.rs. Since they're gone, the accuracy problem you flagged goes with them — but if it pointed at something wrong in the code rather than just the prose, I'd like to know, since that would be worth fixing separately.

One deliberate exception: /// Clear the KV cache across all layers. in quantized_qwen2.rs is left alone. It's upstream's own comment rather than one this PR introduced, so it didn't seem mine to delete.

[alytaphoenix]

Removed all the doc comments this PR introduced (a63ba35a) — including the clear_kv_cache one you quoted. The one remaining KV-cache comment on quantized_qwen2.rs predates this PR (already on main), so left that alone. Ready for re-review.

[alytaphoenix]

Gentle ping — the doc comments you flagged were all removed in a63ba35a back in August, so the CHANGES_REQUESTED here should be clearable whenever you have a moment. Happy to rebase if it's drifted.