Repository navigation
fix: Unparseable MLXCEL_PAGED_SLAB_BLOCKS warns about a derived size it never uses #1137
Description
Problem / Background
When MLXCEL_PAGED_SLAB_BLOCKS is set to something that does not parse as a usize, the server logs that it is falling back to the derived slab size, but it actually keeps the fixed 32-block pool default. The log line describes a behavior the code does not perform.
src/execution/memory_estimate.rs, resolve_paged_slab_blocks (around line 994):
if let Ok(raw) = std::env::var(PAGED_SLAB_BLOCKS_ENV) {
return match raw.trim().parse::<usize>() {
Ok(0) => None,
Ok(n) => Some(n),
Err(_) => {
tracing::warn!(
"{PAGED_SLAB_BLOCKS_ENV}={raw:?} is not a non-negative integer; \
using the derived slab size"
);
None
}
};
}
// derivation logic follows here (byte clamp, geometry probe, etc.)The chain of facts:
- The
Err(_)arm returnsNonefrom an early return, before the derivation logic below it ever runs. The derived slab size is never computed, let alone used. Err(_)is therefore byte-for-byte identical in effect to theOk(0)arm above it, which is documented as pinning the historical 32-block pool default.- The single production call site,
src/server/model_worker.rs:550-559, consumes the result asif let Some(blocks) = paged_slab_blocks { ... }, soNoneskips the pool override entirely. - The pool then falls back to
POOL_SLAB_BLOCKS, which ispub const POOL_SLAB_BLOCKS: usize = 32;atsrc/lib/mlxcel-core/src/cache/paged.rs:509.
So the operator is told the derived size is in use while the run actually gets the fixed 32-block default. Since a layer whose rows outgrow its first slab silently returns to the gather path, this can quietly cost the fused v2 decode path on a long-context deployment, and the log line points the operator away from the cause.
Proposed Solution
Two options, and they are not equivalent. The maintainer should pick one.
(a) Fix the message to describe what happens. Change the warning text to say that the 32-block pool default is being kept, for example "... is not a non-negative integer; keeping the 32-block pool default". One-line change, no runtime behavior change, and it leaves the existing documented contract intact.
(b) Fix the behavior to match the message. Fall through to the derivation on a parse error instead of returning early, so a malformed value really does land on the derived slab size. This is what the text promises today and is arguably the more useful behavior: a typo in the env var would then degrade to the tuned default rather than silently disabling the #899 sizing. This does change runtime behavior for a malformed env value.
If (b) is chosen, the documentation row from #1119 must also be updated: docs/environment-variables.md:244 currently documents the current behavior with "A value that is not a non-negative integer warns and leaves that same 32-block default in place". Under (b) that sentence becomes wrong. Option (a) leaves that row correct as written.
Acceptance Criteria
- A decision is recorded on this issue for whether the fix is (a) message-only or (b) behavior change.
- The warning text and the code path agree: whatever the log line claims is what
resolve_paged_slab_blocksactually returns for an unparseable value. - A unit test pins the chosen contract by asserting the return value of
resolve_paged_slab_blocksfor an unparseableMLXCEL_PAGED_SLAB_BLOCKSvalue (under (a):None, distinct from the derived value; under (b): equal to the value returned when the variable is unset). No existing test covers this today, which is why the log-versus-behavior drift went unnoticed. - The test also distinguishes the parse-error case from the
Ok(0)case, so the two arms cannot silently converge again. - If (b) is chosen, the
MLXCEL_PAGED_SLAB_BLOCKSrow indocs/environment-variables.mdis updated to match, and the0pin remains documented as unchanged.
Technical Considerations
- Scope is small and local: one function in
src/execution/memory_estimate.rs, plus a docs row only under option (b). - The
Ok(0)arm must keep returningNoneeither way. That is the documented escape hatch for pinning pre-Route production batched paged decode through the fused v2 kernel, retiring the gather-then-SDPA hot path #899 allocation behavior and is relied on by benchmark sweeps. - Related context: Route production batched paged decode through the fused v2 kernel, retiring the gather-then-SDPA hot path #899 introduced the derived slab sizing, docs:
environment-variables.mdomits six paged-decode v2 knobs and itsMLXCEL_PAGED_ATTENTION_NATIVErow wrongly says "not a server knob" #1104 and docs: add the six paged-decode v2 variables and correct the NATIVE row #1119 covered the environment-variable documentation for the paged decode v2 knobs.
Activity
chore(execution): parse memory sizes with one shared grammar (#1573)
fix(server): let a malformed MLXCEL_PAGED_SLAB_BLOCKS fall back to th…
fix(server): let a malformed MLXCEL_PAGED_SLAB_BLOCKS fall back to th…
rapsealk commentedon Sep 3, 2026
Decision for acceptance criterion 1: (b), behavior change, in PR #1602.
Rationale: the two options cost the same one-arm edit, and (a) keeps the failure mode this issue names. A typo in
MLXCEL_PAGED_SLAB_BLOCKSwould still silently pin the 32-block slab, so the first long prompt outgrows it and the fused paged decode path (#899) becomes unreachable, while the operator was told the derived size is in use. Under (b) a malformed value warns and is ignored, so the slab is derived exactly as if the variable were unset. TheOk(0)arm is untouched, so the pre-#899 pin and the benchmark sweeps that rely on it are unchanged.Two refinements beyond the issue text, both in the PR: the warning no longer promises "the derived slab size", since the geometry probe below it can still decline a non-pool-eligible model, and instead says the value is ignored and the slab is derived as if unset; and the unit test pins all arms against each other (unset == malformed,
0isNone, an explicit8stays8verbatim below the floor).If (a) is preferred, the diff reduces to the message text plus flipping one assertion to
None.fix(server): let a malformed MLXCEL_PAGED_SLAB_BLOCKS fall back to th…
fix(server): let a malformed MLXCEL_PAGED_SLAB_BLOCKS fall back to th…
fix(server): let a malformed `MLXCEL_PAGED_SLAB_BLOCKS` fall back to …
fix(server): let a malformed `MLXCEL_PAGED_SLAB_BLOCKS` fall back to …
perf(cuda): convert bf16 weights to f16 at load on pre-Ampere GPUs (#…
perf(cuda/quant): add a Volta tensor-core arm to qmm_naive (#1678)