feat(bench): monotonic OOM backstop and persistent OOM record for decode benchmark sweeps

[inureyes]

Summary

On the GB10 (DGX Spark, CUDA unified memory) machine, benchmark cells that OOM are worse than wasted time: near-OOM CUDA runs have repeatedly left the driver in a degraded state (NVRM NV_ERR_NO_MEMORY accumulating for hours, eventually wedging the kernel and requiring a reboot), so cells that are certain to OOM should never be launched at all. This PR adds two purely empirical, reactive-only OOM-avoidance layers to scripts/bench_longprompt.sh, plus a small hardening item in scripts/bench_decode.sh, plus three review-driven hardening fixes to the OOM-avoidance layers themselves.

What changed

• scripts/bench_longprompt.sh: monotonic within-sweep backstop, keyed per model, that skips every remaining cell at length >= an observed SKIP:oom length for the rest of the run, emitting an explicit SKIP:oom_backstop CSV row plus a stderr message.
• scripts/bench_longprompt.sh: persistent cross-run OOM record at gitignored benchmarks/.oom-record (model,prompt_target_len,date,commit,peak_bytes,source_csv, peak_bytes always blank today). On later runs, any cell whose model has a recorded OOM at a length <= the cell's length is pre-skipped and logged as SKIP:oom_record, with the matching record line echoed to stderr. MLXCEL_BENCH_IGNORE_OOM_RECORD=1 disables the pre-skip for one run (new SKIP:oom cells are still appended). Missing/empty record files and malformed lines are tolerated with a stderr warning, not a hard failure.
• scripts/bench_longprompt.sh: MLXCEL_BENCH_FAKE_OOM="model:len[,model:len...]" testing hook that skips invoking the runner for a listed cell and synthesizes a SKIP:oom row, so the backstop and record logic can be exercised deterministically without a real OOM.
• scripts/bench_longprompt.sh: only the exact SKIP:oom classification feeds either layer; FAIL:bench, timeout (exit 124), SKIP:oom_estimate, a missing ./models symlink, and a missing model directory never do, since the gate is a strict string-equality check on the trailing classification token.
• scripts/bench_longprompt.sh: the record-file format, pre-skip semantics, the override env var, and staleness/pruning guidance (the PR fix(cuda/attention): bound long-prompt prefill memory for flash-ineligible models #676 chunked-prefill fix is used as the concrete precedent: a record taken before that merge is stale and should be pruned with grep -v) are documented in both the script's header comment block and its usage() text.
• scripts/bench_decode.sh: extended the is_oom_failure pattern list with three CUDA-specific allocator markers (CUDA_ERROR_OUT_OF_MEMORY, cudaErrorMemoryAllocation, cuMemAlloc failed), kept alongside the existing generic and Metal patterns under the same "specific and grounded" rule.
• .gitignore: added /benchmarks/.oom-record since it is a locally accumulated, host-specific record and not sweep output meant to be committed.

Review-driven hardening (3 follow-up fixes)

• scripts/bench_longprompt.sh: bash-version guard. The OOM_MIN_LEN / FAKE_OOM_SET associative arrays (declare -A) are a bash 4+ feature, but the script explicitly supports Darwin/Metal and stock macOS ships bash 3.2 as /bin/bash, where declare -A aborts the whole script at startup (including --help). The script now re-execs into a bash >= 4 found on PATH (Homebrew /opt/homebrew/bin or /usr/local/bin) if one exists, and otherwise exits with an actionable "install a newer bash" message instead of the cryptic declare: -A: invalid option failure; the guard cannot loop since it only re-execs into a version that passes the check.
• scripts/bench_longprompt.sh: load_oom_record field-count validation. It previously validated only the first two fields (non-empty model, numeric length), so a truncated or hand-edited entry like model,32768 was accepted and pre-skipped indefinitely, contradicting the header comment's promise that wrong-field-count lines are skipped with a warning. It now validates the full field count (>= 6, the documented model,prompt_target_len,date,commit,peak_bytes,source_csv format) so truncated records are rejected while a source_csv path that itself contains a comma stays accepted.
• scripts/bench_longprompt.sh: --lengths ladder validation. Both the monotonic backstop and the persistent-record pre-skip gate on [[ "$len" -ge N ]], an arithmetic comparison; a non-numeric --lengths entry (a typo such as 2o48 or 32k) made that comparison error out and silently evaluate false, so the cell the backstop was supposed to skip launched anyway and the OOM was never recorded, i.e. the safety net could fail open on a config typo. Every ladder length is now rejected up front, before the output CSV is truncated, if it is not a plain non-negative integer, naming the offending token.

Successful-row CSV output is unchanged; the trailing classification token remains an unnamed field after prompt_target_len, consistent with the existing SKIP:oom_estimate / SKIP:oom / FAIL:bench convention in bench_decode.sh.

Test plan

• bash -n scripts/bench_longprompt.sh and bash -n scripts/bench_decode.sh (syntax check)
• shellcheck scripts/bench_longprompt.sh scripts/bench_decode.sh (no new warnings; one pre-existing unrelated SC2002 note in bench_decode.sh)
• Fake-OOM sweep: MLXCEL_BENCH_FAKE_OOM="qwen2.5-0.5b-4bit:512" with --lengths "512 2048 8192" produced SKIP:oom at 512 and SKIP:oom_backstop at 2048/8192, with no real launch for the backstopped cells; benchmarks/.oom-record gained exactly one correctly-formatted line
• Cross-model isolation: a real launch of a second model (llama-3.2-1b-4bit) in a separate sweep produced a normal successful row and left the first model's .oom-record entry untouched, confirming the backstop and record are keyed per model
• Hand-written record: a manually written (qwen2.5-0.5b-4bit, 512) record line pre-skipped all three ladder lengths as SKIP:oom_record with no launch and left the record file unchanged; re-running with MLXCEL_BENCH_IGNORE_OOM_RECORD=1 executed the cell for real instead of pre-skipping it
• Non-OOM isolation: pointing --models at a nonexistent model directory produced the existing [miss] skip path with zero CSV rows and no benchmarks/.oom-record file created
• Malformed-record rejection: a truncated two-field line (model,32768) appended to .oom-record is skipped with an explicit stderr warning and does not count toward loaded entries, while the valid six-field line alongside it still loads and pre-skips
• Ladder validation: --lengths "512 2o48" exits 1 with an explicit "must be space-separated non-negative integers" message before any cell launches or the output CSV is truncated, while a valid ladder still runs and the backstop still fires
• Working tree verified clean of test artifacts afterward; the ~20 pre-existing untracked benchmarks/*.csv files were left untouched throughout, and the test-only benchmarks/.oom-record file was removed

Closes #807

[inureyes]

Implementation Review Summary

Intent

Add two purely empirical, reactive-only OOM-avoidance layers to scripts/bench_longprompt.sh (within-sweep monotonic backstop emitting SKIP:oom_backstop; persistent gitignored benchmarks/.oom-record pre-skip emitting SKIP:oom_record, with MLXCEL_BENCH_IGNORE_OOM_RECORD=1 override and a MLXCEL_BENCH_FAKE_OOM test hook), a .gitignore entry, and CUDA OOM patterns in bench_decode.sh is_oom_failure.

Findings Addressed

None. No CRITICAL/HIGH/MEDIUM findings; no code changes were required.

Verification (all empirically exercised on this branch, shell-only, no cargo)

• All stated requirements implemented (all 8 acceptance criteria met)
• Backstop: a SKIP:oom cell skips every same-model cell at len >= the OOM length as SKIP:oom_backstop before launch, with a stderr message (verified via MLXCEL_BENCH_FAKE_OOM)
• Record: true OOM appended to gitignored benchmarks/.oom-record as model,len,date,short-commit,,(blank peak),source_csv; git check-ignore confirms the ignore rule
• Later runs pre-skip recorded model at len >= recorded length as SKIP:oom_record (matching line echoed to stderr)
• MLXCEL_BENCH_IGNORE_OOM_RECORD=1 bypasses the pre-skip yet still appends new SKIP:oom cells; documented in header + usage()
• Only exact SKIP:oom feeds either layer (strict == equality); FAIL:bench, exit-124 timeout, SKIP:oom_estimate, missing ./models symlink, and missing model dir ([miss] path precedes the gate) never do
• Successful-row CSV schema unchanged: CSV_HEADER is byte-identical to bench_decode.sh; skip rows use the same 16-field layout with a trailing classification token
• cell_classification parses the last comma-separated field, so a quoted prompt containing commas (e.g. "Hello, how are you today?") is classified correctly; short-prompt successful rows (empty trailing prompt_target_len) and long-prompt rows (numeric trailing field) are correctly treated as non-classified
• Bash correctness under set -euo pipefail: associative-array access guarded with :- defaults, check_oom_record return-1 consumed inside an if condition, empty-array expansion safe on bash 5.2.21; malformed record lines are warned and skipped (not fatal); bash -n and shellcheck clean (only a pre-existing unrelated SC2002 in bench_decode.sh)
• Documentation complete: header comment and usage() cover the record format, pre-skip semantics, override var, and staleness/pruning guidance (PR fix(cuda/attention): bound long-prompt prefill memory for flash-ineligible models #676 precedent)
• Integrated into the actual model-major/length-minor sweep loop, not orphaned helpers
• No unintended structural changes; existing runner invocation preserved

Remaining Items

None.

[inureyes]

Follow-up: supplementary review findings

A supplementary automated review pass surfaced three additional items. Disposition below; two are fixed in 4bfd870, one is reported for maintainer judgment.

Fixed (pushed in 4bfd870)

• HIGH: bash 4 requirement broke stock macOS. declare -A OOM_MIN_LEN / FAKE_OOM_SET are bash 4+ only, but the script explicitly supports Darwin/Metal (it detects M1 Ultra and M5 Max) and stock macOS ships bash 3.2 as /bin/bash, where declare -A aborts the whole script at startup, including --help. This is the first bash-4 dependency in scripts/ (no other bench script uses declare -A). Added an early, bash-3-compatible guard that re-execs under a newer bash if one is on PATH (Homebrew /opt/homebrew/bin or /usr/local/bin) and otherwise exits with an actionable message instead of the cryptic declare: -A: invalid option. The guard only re-execs into a bash whose major version is >= 4, so it cannot loop.
• MEDIUM: truncated OOM-record lines were accepted. load_oom_record validated only the first two fields, so a truncated/hand-edited line like model,32768 was accepted and pre-skipped indefinitely, contradicting the header comment that promises wrong-field-count lines are skipped. Now validates the full field count (>= 6, the documented model,prompt_target_len,date,commit,peak_bytes,source_csv format); a source path containing a comma only inflates the count and stays accepted.

Reported, not changed

• MEDIUM: cuMemAlloc failed may match non-OOM CUDA errors. cuMemAlloc can also fail with CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_DEINITIALIZED, etc., so matching the call name alone could classify a non-OOM defect as SKIP:oom and then persist it. Left as-is deliberately: MLX emits these strings from its vendored C++ runtime (not in this tree), so the exact OOM-path wording cannot be confirmed here, and tightening the pattern risks a false negative, which for this gate is the more dangerous direction. The other two CUDA markers (CUDA_ERROR_OUT_OF_MEMORY, cudaErrorMemoryAllocation) are unambiguous. Flagging for a maintainer who can check MLX's actual CUDA OOM output; if it always includes an OOM reason string, narrowing cuMemAlloc failed to co-require that reason would remove the false-positive surface.

Re-verified after the fixes with bash -n and shellcheck (clean), plus the fake-OOM backstop, persistent-record pre-skip, and malformed/truncated-line paths. Working tree left clean; the pre-existing untracked benchmarks/*.csv files were untouched.