Skip to content

fix: Unparseable MLXCEL_PAGED_SLAB_BLOCKS warns about a derived size it never uses #1137

Closed

Description

@inureyes

Problem / Background

When MLXCEL_PAGED_SLAB_BLOCKS is set to something that does not parse as a usize, the server logs that it is falling back to the derived slab size, but it actually keeps the fixed 32-block pool default. The log line describes a behavior the code does not perform.

src/execution/memory_estimate.rs, resolve_paged_slab_blocks (around line 994):

if let Ok(raw) = std::env::var(PAGED_SLAB_BLOCKS_ENV) {
    return match raw.trim().parse::<usize>() {
        Ok(0) => None,
        Ok(n) => Some(n),
        Err(_) => {
            tracing::warn!(
                "{PAGED_SLAB_BLOCKS_ENV}={raw:?} is not a non-negative integer; \
                 using the derived slab size"
            );
            None
        }
    };
}
// derivation logic follows here (byte clamp, geometry probe, etc.)

The chain of facts:

  1. The Err(_) arm returns None from an early return, before the derivation logic below it ever runs. The derived slab size is never computed, let alone used.
  2. Err(_) is therefore byte-for-byte identical in effect to the Ok(0) arm above it, which is documented as pinning the historical 32-block pool default.
  3. The single production call site, src/server/model_worker.rs:550-559, consumes the result as if let Some(blocks) = paged_slab_blocks { ... }, so None skips the pool override entirely.
  4. The pool then falls back to POOL_SLAB_BLOCKS, which is pub const POOL_SLAB_BLOCKS: usize = 32; at src/lib/mlxcel-core/src/cache/paged.rs:509.

So the operator is told the derived size is in use while the run actually gets the fixed 32-block default. Since a layer whose rows outgrow its first slab silently returns to the gather path, this can quietly cost the fused v2 decode path on a long-context deployment, and the log line points the operator away from the cause.

Proposed Solution

Two options, and they are not equivalent. The maintainer should pick one.

(a) Fix the message to describe what happens. Change the warning text to say that the 32-block pool default is being kept, for example "... is not a non-negative integer; keeping the 32-block pool default". One-line change, no runtime behavior change, and it leaves the existing documented contract intact.

(b) Fix the behavior to match the message. Fall through to the derivation on a parse error instead of returning early, so a malformed value really does land on the derived slab size. This is what the text promises today and is arguably the more useful behavior: a typo in the env var would then degrade to the tuned default rather than silently disabling the #899 sizing. This does change runtime behavior for a malformed env value.

If (b) is chosen, the documentation row from #1119 must also be updated: docs/environment-variables.md:244 currently documents the current behavior with "A value that is not a non-negative integer warns and leaves that same 32-block default in place". Under (b) that sentence becomes wrong. Option (a) leaves that row correct as written.

Acceptance Criteria

  • A decision is recorded on this issue for whether the fix is (a) message-only or (b) behavior change.
  • The warning text and the code path agree: whatever the log line claims is what resolve_paged_slab_blocks actually returns for an unparseable value.
  • A unit test pins the chosen contract by asserting the return value of resolve_paged_slab_blocks for an unparseable MLXCEL_PAGED_SLAB_BLOCKS value (under (a): None, distinct from the derived value; under (b): equal to the value returned when the variable is unset). No existing test covers this today, which is why the log-versus-behavior drift went unnoticed.
  • The test also distinguishes the parse-error case from the Ok(0) case, so the two arms cannot silently converge again.
  • If (b) is chosen, the MLXCEL_PAGED_SLAB_BLOCKS row in docs/environment-variables.md is updated to match, and the 0 pin remains documented as unchanged.

Technical Considerations

Activity

  1. added
    type:bugBug fixes, error corrections, or issue resolutions
    on Aug 13, 2026
  2. rapsealk commented on Sep 3, 2026

    @rapsealk
    Member

    Decision for acceptance criterion 1: (b), behavior change, in PR #1602.

    Rationale: the two options cost the same one-arm edit, and (a) keeps the failure mode this issue names. A typo in MLXCEL_PAGED_SLAB_BLOCKS would still silently pin the 32-block slab, so the first long prompt outgrows it and the fused paged decode path (#899) becomes unreachable, while the operator was told the derived size is in use. Under (b) a malformed value warns and is ignored, so the slab is derived exactly as if the variable were unset. The Ok(0) arm is untouched, so the pre-#899 pin and the benchmark sweeps that rely on it are unchanged.

    Two refinements beyond the issue text, both in the PR: the warning no longer promises "the derived slab size", since the geometry probe below it can still decline a non-pool-eligible model, and instead says the value is ignored and the slab is derived as if unset; and the unit test pins all arms against each other (unset == malformed, 0 is None, an explicit 8 stays 8 verbatim below the floor).

    If (a) is preferred, the diff reduces to the message text plus flipping one assertion to None.

  3. added and removed on Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:lowLow prioritystatus:doneCompletedtype:bugBug fixes, error corrections, or issue resolutions

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions