feat(generation): min-p sampling and OpenAI-style frequency/presence penalties

[RajaBabu15]

Summary

Implements #3654: min_p sampling and OpenAI-style frequency/presence penalties in candle_transformers::generation — both expected by anyone exposing an OpenAI-compatible API on top of candle.

Sampling::MinP { p, temperature }

Min-p sampling (arXiv:2407.01082) keeps tokens whose probability is at least p * max_prob, then samples from the renormalized survivors. Wired through LogitsProcessor::sample like the existing TopK/TopP variants; p <= 0 degrades gracefully to plain multinomial sampling.

let mut lp = LogitsProcessor::from_sampling(seed, Sampling::MinP { p: 0.05, temperature: 0.8 });
let next = lp.sample(&logits)?;

apply_frequency_presence_penalty

OpenAI semantics, applied to logits before sampling:

logit[t] -= frequency_penalty * count(t) + presence_penalty   // for every t in context

Distinct from utils::apply_repeat_penalty (multiplicative, count-insensitive). Out-of-vocab context tokens are ignored; zero penalties are a no-op returning the input tensor. Input dtype is preserved.

let logits = apply_frequency_presence_penalty(&logits, &generated, 0.5, 0.3)?;

Tests

cargo test -p candle-transformers --test generation_tests

• sample_with_min_p_dominant_token — deterministic across seeds when one token dominates
• sample_with_min_p_filters_and_renormalizes — sub-threshold token never sampled; survivor odds match the renormalized distribution over 10k draws
• sample_with_min_p_zero_keeps_all_tokens — p = 0 keeps the full distribution
• frequency_presence_penalty_matches_openai_semantics — exact penalty arithmetic
• frequency_presence_penalty_noop_and_out_of_range — zero-penalty no-op, out-of-vocab ids ignored

All existing tests pass; cargo clippy -p candle-transformers --tests -- -D warnings and cargo fmt --check are clean.

Fixes #3654

[astorise]

Thanks for this — min-p sampling and OpenAI-compatible frequency/presence penalties are both useful additions for servers built on candle, and the implementation is solid: sample_minp correctly reuses the existing "zero out below threshold, let WeightedIndex renormalize" pattern already established by sample_topp, and the penalty helper correctly deduplicates context tokens via a HashMap so presence penalty isn't accidentally scaled by occurrence count.

One precision issue found while integrating this into a fork: apply_frequency_presence_penalty computes the adjustment in f32 but then casts the result back to the caller's original dtype before returning:

Tensor::from_vec(logits_v, logits_len, device)?.to_dtype(dtype)
For f16/bf16 model logits, this round-trip can round a small penalty away entirely (or noticeably distort it) — e.g. a f16 logit of 1000.0 has an ULP of ~0.98, so subtracting a penalty of 0.5 and casting back to f16 rounds it away to nothing. This is wasted precision loss with no benefit: LogitsProcessor::sample_f does logits.to_dtype(DType::F32)? as its very first step regardless, so the value is immediately converted back to f32 anyway. utils::apply_repeat_penalty (the existing, established helper for the multiplicative repeat penalty) already avoids this exact problem — it always returns f32 without casting back to the input dtype.

Suggested fix: drop the trailing .to_dtype(dtype) (and the now-unused dtype variable), including in the zero-penalty no-op branch, so the function's return dtype is uniformly f32 regardless of whether a penalty was applied — matching apply_repeat_penalty's convention.

Pushed this fix on top of the PR in my fork, with a regression test reproducing the f16 rounding scenario above: astorise/candle@fcd9d03. Happy to open it as a PR against this branch or main if useful.