Repository navigation
chore: adopt MLX core 0.32.2 across the inference stack - #790
Merged
Merged
Conversation
Bump libs/mlx-swift to 6b0505c (fork main, PR #16: vendored MLX/MLX-C at 0.32.2 + regenerated JIT sources) and libs/mlx to 734241bb (fork main, PR #10: the 0.32.2 core squash carrying the darkbloom patch set). Content of the new core pin vs the previous darkbloom-base line, verified against the upstream v0.32.2 tag: identical to v0.32.2 plus the 16-file darkbloom set (E=256 expert-tile route, Gemma 4 expert-QMM tile kernel, Metal buffer-count bounds, gpu::eval UAF fix, resource-count telemetry, thread-safe CompileCacheRegistry). The hand-ported qmv_wide on darkbloom-base is superseded by upstream's real kernel inside 0.32.2. What the serving stack gains at B=1-5 (geometry verified from served checkpoint configs): read-each-K/V-byte-once GQA-8 decode attention (Qwen 3.5/3.6 35B are 16:2 heads, GPT-OSS 20B is 64:8), a fused full-attention path for head_dim 256 on NAX (both Qwen 35Bs), qmv_wide/ gemv_wide few-row matmuls, quantized-MoE NAX matmul improvements, and the raised qmv batch limit on M5-class GPUs, plus the dequantize-in-fp32 and qmm floor correctness fixes. libs/mlx .gitmodules branch moves darkbloom-base -> main: PRs #10/#16 made main the integration line; darkbloom-base remains as the legacy pre-0.32.2 base.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Gajesh2007
had a problem deploying
to
benchmarks
August 31, 2026 05:40 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Member
Author
|
Local B=1/B=4 before/after bench: prepared but deferred — the host was running other workloads during the window (BenchCBv2 itself flagged A clean local A/B is staged and ready on a quiet machine — both sides are built with source-matched metallibs, same mlx-swift-lm commit (
cd <side>/libs/mlx-swift-lm
./.build/release/BenchCBv2 \
--model ~/.cache/huggingface/hub/models--EigenLabs--Qwen3.6-35B-A3B-MLX-VL-4bit-g64-router8-mtp/snapshots/baseline-work \
--mode perf --batches 1,4 --steps 64 --prompt-lengths 512 --label <old-core|new-core>This same run doubles as the E=256-expert-tile-route re-check flagged in the PR description. Separately: the Threat Model Review check fails in 9s with |
Member
Author
|
Caveat on the bench repro above: the one BenchCBv2 execution under contention printed the |
Gajesh2007
added a commit
that referenced
this pull request
Aug 31, 2026
…60822 Brings the test-suite refactor up to date with the 22 commits master gained since the last sync (#700-#790: Qwen3-VL and Qwen3.8 serving, first-content deadline admission, MTP-in-the-weights, serial-number privacy, MLX core 0.32.2, usage-checkpoint fixes). Every master-side test added or modified inside files this branch deletes, splits, or rewrites was re-homed into the refactor's successor suites and lanes rather than dropped, including: - PrefillDeadlineProductionConfigTests and the Qwen required-tool-choice tests into the slimmed EngineV2ProductionWiringTests - bridgeDeadlineRefusalReleasesRealStaging into SSDPrefixCacheReservationTests - excludesInferenceControls into WatchdogRecoveryTests - the rewritten #777 MTP config/beta suites into ConfigTests and BetaFeaturesTests (MTPConfigTests stays deleted) - vision/DeepStack/media additions into the split EngineV2Vision* and MediaIngest suites - the #704 writeDeferred provider-writer tests, first-content-budget wire test, #730 serial-privacy subtests, and #752 benchmark-control tests into their branch successors Master-added test files were audited against the lane architecture: the Qwen3.8 production canary trio moved to Live/ (with a #filePath depth fix), SchedulerPrefillDecision's real-weights suite split out to Probes/ behind its per-probe gates, and hermetic additions stay in the default lane. docs/developer/test.md updated to match. Production-code unions of note: registry.go keeps both the branch's PrivateOnly/catalog public-listing gates and master's per-model capability gating; provider_writer.go carries both determinism-seam families; LaunchAgent forwards the union of both env allowlists; postgres_model_registry.go extends nil-canonicalization to master's new required_provider_capabilities column. The branch-side prefill-keepalive tests were dropped with the feature master removed in #704. Two master-branch interaction defects fixed so the merged tree is green: the store contract harness now re-seeds the usage_totals counter row its truncate wipes (the #665 migration guard treats the bare marker as corruption), and the #730 log-report migration test restores the pooled connection's search_path before release. Submodules: mlx-swift and mlx-swift-lm test branches merged with master's 0.32.2-era pins; relocated suites verified present. Verified: go vet + full coordinator/testbed tests green; store contracts green twice consecutively against real Postgres 16; swift build --build-tests clean; default Swift lane 2,264 tests in 248 suites all passing.
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adopts the already-merged MLX 0.32.2 upgrade into the serving stack:
libs/mlx-swift→6b0505c(forkmain, PR chore: upgrade MLX core to 0.32.2 mlx-swift#16: vendored MLX/MLX-C at 0.32.2 + regenerated JIT sources)libs/mlx→734241bb(forkmain, PR chore: upgrade core to MLX 0.32.2 mlx#10: the 0.32.2 core squash carrying the darkbloom patch set).gitmodulesforlibs/mlxmovesdarkbloom-base→main(PRs Add invite code system for coordinator-funded credits #10/Add build provenance attestation verification #16 mademainthe integration line;darkbloom-baseremains as the legacy pre-0.32.2 base)Why
Our previous core pin was 33 commits behind v0.32.0, plus the 279 commits to v0.32.2. That range is dense with kernels aimed at exactly our B=1–5 decode workload (geometry verified from served checkpoint configs):
perf/head256-fused-attentionbranchesqmv_wide/gemv_widefew-row matmul kernels + raised qmv batch limit on M5-class GPUs (mlx#3764, #3888, #3791) — supersedes the hand-ported qmv_wide ondarkbloom-baseqmm_n_naxre-disable + group_size<64 fix (#4202; we ship g64), thread-safe compile-cache erase (#4248) plus the fork'sCompileCacheRegistryperf/vlm-vision-batch)Content of the new core pin was verified against the upstream
v0.32.2tag:git diff v0.32.2 <pin>shows exactly the 16-file darkbloom set (E=256 expert-tile route, Gemma 4 expert-QMM tile kernel, Metal buffer-count bounds,gpu::evalUAF fix, resource-count telemetry, compile-cache registry) and nothing else.Verification
swift build -c releaseof provider-swift against the new pins: clean.make provider-test(2,294 tests in 236 suites, source-matched metallib): all pass. A first run under heavy parallel build load had 2 flaky failures; the identical suite is clean on rerun.Package.swifttracksLayr-Labs/mlx-swiftmain(=6b0505csince 8/25), and this branch's provider release build, the full provider test suite, and an lm SPM test build all compiled lm against it. Note the lm fork's "Build and Test" workflow currently hitsstartup_failureon every recent branch, so no fork CI has actually exercised the combination — the local runs above are the evidence.qwen3.6-35b-a3b-vl-mtp-mxfp8(darkbloom benchmark --sweep) to follow as a PR comment; the benchmarks workflow also runs on this PR.Follow-ups (not this PR)
gather_qmmafter the bump — it may be partially superseded.perf/head256-fused-attention(mlx Redesign frontend as EigenInference with Privy auth and three-section architecture #8, mlx-swift Bump the npm_and_yarn group across 1 directory with 3 updates #13) and mlx-swift Dynamic capacity-based request routing #17 (duplicate of merged Add build provenance attestation verification #16) look superseded.🤖 Generated with Claude Code
https://claude.ai/code/session_01Bk1FuSDRgDnyQUrgJj9VHF