Repository navigation
perf(qwen36): bump mlx-swift-lm to GDN capture-verify + target-prefix MTP (mlx-swift-lm#106) - #616
Merged
Merged
Conversation
…x MTP Submodule branch perf/qwen36-mtp-capture-verify @ 3b66355: batched [B,1+k] MTP verification for the 30 GatedDeltaNet layers (per-position state capture, device-side commit-select, snapshot rollback) plus target_prefix acceptance lifting the temperature==0 MTP gate. No provider-swift source changes required; no Metal source touched (gated-delta kernels are runtime-JIT), so no metallib rebuild needed.
Gajesh2007
had a problem deploying
to
benchmarks
August 13, 2026 22:11 — with 
GitHub Actions
Failure
GitHub Actions
Failure
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Gajesh2007
had a problem deploying
to
benchmarks
August 13, 2026 22:28 — with
GitHub Actions
Failure
GitHub Actions
Failure
…106 squash d06da67)
Gajesh2007
marked this pull request as ready for review
August 13, 2026 22:33
Gajesh2007
had a problem deploying
to
benchmarks
August 13, 2026 22:33 — with
GitHub Actions
Failure
GitHub Actions
Failure
This was referenced Aug 13, 2026
Member
Author
|
Stacked on top of this PR: #617 (base = this branch |
This was referenced Aug 13, 2026
Gajesh2007
added a commit
that referenced
this pull request
Aug 13, 2026
…rics endpoint (#619) * feat(provider): MTP posture and acceptance counters on the local /metrics endpoint Bug 2 from mlx-swift-lm#106 'Discovered bugs': MTP acceptance existed only in-engine (CBv2MTPMetrics via EngineV2Bridge.mtpStatusSnapshot) and in the engine_v2_slot_posture telemetry event — i.e. Datadog Logs. A headless operator (no os_log store in sandboxes) could not observe acceptance, or that MTP silently never activated (inline_artifact_invalid). - LocalMetricsResponder serves GET /metrics: the upstream MLXLMServer ServerMetrics body (same public prometheusMetrics() entry point) plus provider-owned per-model lines in the same Prometheus text style: mtp_enabled, mtp_active, mtp_rounds_total, mtp_tokens_proposed_total, mtp_tokens_accepted_total, and mtp_inactive_reason{reason=...} carrying the MTPFallbackReason enum (inline_artifact_invalid, inert_kv_unsupported, ...) whenever MTP is not productively running. - Wired through both local endpoints via a mtpSlots closure: StandaloneServer slot cache and the unified ProviderLoop modelSlots — the same bridge snapshot the capacity tick feeds DaemonSlotPostureBuilder. - Coordinator telemetry wire types untouched (3-language sync out of scope); daemon-state SlotPosture already carries the mtp posture trio. - Pure provider-swift change; no submodule field added by #106 is required (counters predate it), so it is pin-independent of the #616 bump. * fix(provider): guarantee a newline at the /metrics upstream/MTP seam The MTP block was appended to MLXLMServer's prometheusMetrics() body by bare string concatenation, silently relying on the upstream body's trailing newline — an implementation detail of ServerMetrics .prometheusText()'s multiline literal. A submodule bump dropping that newline would fuse the final upstream sample and the first MTP header into one invalid line (mlx_server_uptime_seconds 12# TYPE mtp_enabled gauge), making Prometheus reject the whole scrape. MTPPrometheusRenderer.joinedBody now makes the seam explicit: exactly one newline between a non-empty upstream body and the MTP block, empty upstream yields the MTP block alone, empty MTP block leaves the upstream body byte-identical, and the joined body keeps the single trailing newline the text format expects. Tests: the /metrics route tests now assert line-level exposition validity (every non-empty line is a # TYPE/# HELP header or a complete name{labels} value sample) instead of substring checks alone, plus a regression case for an unterminated upstream body + resident slot. Verified the regression test fails against the pre-fix bare concat.
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Bumps
libs/mlx-swift-lmto the Qwen3.6 GDN capture-verify + target-prefix MTP branch — companion to Layr-Labs/mlx-swift-lm#106 (full problem statement, measurements, diagrams, and discovered-bug inventory live there).Merge order: land #106 in the submodule first, then update this pin to the merged commit (same flow as #610/#85).
Summary of the underlying change
[1,1+k]target forward with per-position GatedDeltaNet state capture → 113.4 tok/s = 1.08x vs MTP-off, 1.53x vs shipped serial, acceptance 0.79 (identical accept pattern).target_prefixacceptance lifts thetemperature==0eligibility gate — sampled traffic (prod default temp=1) can speculate, exact for the output distribution.Behavior / code delta
flowchart LR subgraph Before A1["submodule @ c04afb4"] --> B1["MTP verify: 2 serial [1,1] forwards -> 0.70x; temp>0 never speculates"] end subgraph After A2["submodule @ 3b66355 (PR #106)"] --> B2["MTP verify: one [1,1+k] capture window -> 1.08x; any-temp speculation via target_prefix"] endNo coordinator/provider Swift source changes in this PR — pointer bump only.
Verification
Submodule: build green (debug+release), 4 new capture-verify tests + extended Qwen integration suite, 87/13 CBv2MTP + 44/7 adjacent suites pass. Measurement method: release-only, paired same-session A/Bs, APFS-cloned snapshot with MTP activation verified via SSE delta pattern (method rules committed in
notes/qwen36-baseline-2026-08-13.mdonperf/qwen36-baseline).Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.