Repository navigation
feat(provider): integrate Qwen3.8-27B full VLM - #700
Conversation
|
@glg2672 is attempting to deploy a commit to the EigenLabs Team on Vercel. A member of the Team first needs to authorize it. |
|
Ran Qwen3.8-27B on an M5 Max in case fresh Apple Silicon numbers are useful for the review. Important caveat first: I did not use Setup
Results
Decode is stable within 0.3% across runs: 18.06 tok/s average. Sanity check: at 614 GB/s and 27.5 GB of weights read per token, the single-stream ceiling Peak memory 28.9 GB leaves a lot of headroom on a 128 GB box for KV cache and concurrency. One observation for the economics, not the codeBeing dense, this model is roughly an order of magnitude slower per stream than the Not an argument against merging — 18 tok/s is perfectly usable, and #733 shows the demand. Happy to build and run this branch on this machine if a fresh-silicon validation datapoint |
|
@codex review this |
|
Hey @glg2672 !! Can you rebase this PR to the latest |
|
And possibly run it with MTP and attached the numbers? Here's our model artifact btw: https://huggingface.co/EigenLabs/Qwen3.8-27B-4bit |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: baa058b313
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
baa058b to
9d75fac
Compare
Update: long-context Qwen3.8 TTFTAddressed the provider-side long-context prefill bottleneck in signed commit Changes
Benchmark evidencePrevious release measurements on Apple M3 Max, 64 GB:
Exploratory CBv2 comparison at 4,096 tokens:
The A/B runs had different host load, so this is directional evidence rather than a final KPI claim. A controlled repeated benchmark with the Qwen3.8 artifact remains the release gate. Validation
flowchart LR
subgraph Before
A1[Dense Qwen request] --> B1[Generic 2,048-token solo stripe]
B1 --> C1[Long prefill]
C1 --> D1[First token]
end
subgraph After
A2[Dense Qwen request] --> B2[Model-aware 4,096-token solo stripe]
B2 --> C2[Amortized long prefill]
C2 --> D2[First token]
end
flowchart TD
subgraph Before
E1[prepareProductionBackend] --> E2[productionSchedulerConfig]
E2 --> E3[Generic 2,048 stripe]
end
subgraph After
F1[prepareProductionBackend] --> F2[productionSchedulerConfig model]
F2 --> F3{Dense Qwen35?}
F3 -->|Yes| F4[4,096 stripe]
F3 -->|No / MoE| F5[Existing policy]
end
Remaining release gate: run the controlled release benchmark and a real streaming HTTP E2E request with the Qwen3.8 artifact before claiming the OpenRouter TTFT target. |
|
Some Benchmark numbers as well
|
|
@codex review this Final signed head is |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5376f62d23
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review this |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review this |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9bc95812e6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review this |
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review this |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: da118c6114
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review this |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d3ebc50ce
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review this |
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…ecipe Verified mlx-community/Qwen3.5-9B-MLX-4bit end-to-end on master (zero code changes — dense qwen3_5 shipped with #700/Qwen3.8-27B). Records the production-engine benchmark matrix, a 1.51x MTP speedup from a locally built combined artifact, and the two non-obvious artifact conventions: +1 RMSNorm conversion from HF and bfloat16 scales/biases (fp16 silently disables the assistant).
…dded Qwen heads only (#777) * fix(provider): restore and extend automatic MTP to the Qwen 3.5 family #752 (v0.8.14) made MTPMode.auto architecture-based: any qwen3_5_moe checkpoint drafted. #700 narrowed it to the single pinned EigenLabs/Qwen3.8-27B-4bit, silently dropping auto-MTP for the Qwen3.5/3.6 35B MoE fleet on upgrade — and dense qwen3_5 checkpoints (the 9B class) never had it. .auto now enables MTP for the pinned Qwen3.8 target plus, by config.json model_type, the qwen3_5 / qwen3_5_moe family — the same set SpecDecArtifactFunnel.isQwen35Target already resolves. The funnel stays the authority: a family checkpoint without an MTP artifact still falls back target-only with a recorded reason, and the kill switch is unchanged. enablesMTP now takes the model type; callers that only know an id keep the pinned-id decision. Verified live on an M4 Max, default config (no mtp_mode key): - Qwen3.5-9B inline MTP: 80.7 tok/s vs 47.6 target-only (1.7x) - Qwen3.6-35B inline MTP: 44-52 tok/s vs 26.5 kill-switch (1.9x) - Qwen3.8-27B pinned path: unchanged (canary + resolver tests) * refactor(provider): embedded MTP only — delete the external Qwen3.8 head path Qwen 3.8-27B moves from the separately published, revision-pinned external assistant (EigenLabs/Qwen3.8-27B-MTP-4bit) to the embedded inline artifact (mtplx_mtp in the target checkpoint), the same shape the 3.6-35B and the 9B ship. The pairing now travels inside the weights, so the machinery that existed only to pin it outside is scrap: - delete StandaloneQwen38MTPResolver (exact-revision cache staging, samePayload verification, recordingSourceRevision binding) and its branch in StandaloneServer.specDecPreparation — the funnel is the one mechanism - delete MTPMode.automaticTargetModelID and collapse enablesMTP to the model-type-only family rule (the pin gated the external pair and nothing else) - delete the 'Standalone exact Qwen3.8 MTP cache' test suite with the resolver; MTPConfigTests now pin the family rule directly Fleet (coordinator) mode is untouched: catalog spec-dec resolution keeps serving the external head until the registry switches the 27B entry to the embedded artifact. Verified live on an M4 Max, default config (no mtp_mode key), on the final binary: embedded 27B artifact drafts at 28-31 tok/s vs 20.3-20.6 with the kill switch; 9B 80.7 vs 47.9; 3.6-35B 44-52 vs 26.5. * feat(provider): auto MTP activates only embedded (mtplx_mtp) Qwen heads .auto now requires BOTH the hardcoded Qwen family gate (qwen3_5, qwen3_5_moe) AND an embedded-head declaration (mtplx_mtp.included=true in the checkpoint's config.json). A Qwen checkpoint without an embedded head is not asked about at all — no catalog lookup, no prefetch, plain config-disabled fallback. Separately published assistants and mtp_drafter_path overrides now require an explicit mtp_mode="on", and catalog prewarm runs only under "on" (auto never consults the catalog). The weights ship the policy: publishing a checkpoint with an embedded head is what turns MTP on for it, with no per-model code or registry spec-dec metadata. Verified live under a default config on the gated binary: embedded 9B drafts at 79-83 tok/s; the bare mlx-community 9B serves target-only. 74/74 MTP suite tests. * docs(provider): beta mtp feature text describes the embedded-head policy Codex review: 'darkbloom beta list' still said automatic mode enables only the exact Qwen3.8 target. It now describes the embedded-gated family rule, and notes that catalog assistants and mtp_drafter_path overrides require the explicit on override. * test(provider): extraction suite colocates the metallib like every other MLX suite QwenVLMTargetExtractionTests evaluates MLX arrays through tiny synthetic modules but was the one MLX-touching suite without LiveInferenceFixtures.ensureMetallibColocated(); a fresh worktree needed the metallib manually copied beside the test runner. The suite init now self-heals: verified by deleting the manual copy and re-running 17/17. * fix(provider): tri-state inline probe preserves invalid-declaration diagnostics Codex review: the boolean declaresInlineArtifact applied the inspection's strict 1 MiB config cap, so a valid-but-oversized config.json that DECLARES an embedded head read as 'absent' — the auto pre-gate then reported config_disabled and the funnel never surfaced inline_artifact_invalid. SpecDecStore.inlineDeclarationProbe now distinguishes declared / absent / undeterminable with a deliberately looser probe bound (64 MiB): declared and undeterminable configs reach inspectInlineArtifact, whose strict cap rejects loudly with the named reason; only a parsed config without a declaration falls through to the catalog resolver silently, as designed. All three callers (funnel inline branch + both auto pre-gates) migrated; boolean discriminator deleted. New InlineDeclarationProbeTests cover the tri-state, the oversized-declared end-to-end inline_artifact_invalid, and the oversized-undeclared catalog fallthrough. * docs(provider): operator MTP guides describe the embedded-head auto policy Codex review: beta-features.md and cli-reference.md still described MTP as a default-off Gemma-only feature that installing a release never enables. Both now document auto's embedded-Qwen activation, the explicit-off rollback, and that catalog spec_dec assistants and mtp_drafter_path overrides require the explicit on override. * docs(reports): carry the Qwen3.5-9B validation + MTP report in this PR Consolidated from #774 (closed in its favor): the validation record behind this PR's policy change, including the embedded-head build conventions, the measured MTP results, and the OPEN greedy-token-parity validation item that gates registry rollout of the embedded artifacts. * fix(provider): close three review gaps in the embedded-MTP path 1. Single-probe inline decision (TOCTOU): the funnel re-probed config.json after the auto pre-gate; a config change between the two reads could flip an embedded-only decision into a catalog resolution auto forbids. SpecDecArtifactFunnel.Request now carries the caller's one authoritative InlineDeclarationProbe and the funnel never re-probes. Pinned by SingleProbeInlineDecisionTests. 2. Inline assistants no longer charge twice: ModelScanner already counts the embedded mtp shard in the target's sizeBytes/estimatedMemoryGb, yet admission and the pending-load reservation added residentBytes on top — near the cap that misreported assistant_memory_unavailable for a fitting target+assistant. SpecDecArtifact.additionalWeightBytes is 0 for .inline and residentBytes otherwise; all three charge sites (ProviderLoop admission, pending-load reservation, standalone) use it. 3. EigenLabs/Qwen3.8-27B-4bit-mtp inherits the target's hardware invariant: ModelRuntimeRequirements now gates every concrete Qwen3.8 build id (M5 + MLX NAX), so the embedded publication cannot dodge eligibility by renaming. Pinned by EmbeddedQwen38RuntimeRequirementsTests. 107/107 across the MTP, funnel, and runtime-requirements suites.
…60822 Brings the test-suite refactor up to date with the 22 commits master gained since the last sync (#700-#790: Qwen3-VL and Qwen3.8 serving, first-content deadline admission, MTP-in-the-weights, serial-number privacy, MLX core 0.32.2, usage-checkpoint fixes). Every master-side test added or modified inside files this branch deletes, splits, or rewrites was re-homed into the refactor's successor suites and lanes rather than dropped, including: - PrefillDeadlineProductionConfigTests and the Qwen required-tool-choice tests into the slimmed EngineV2ProductionWiringTests - bridgeDeadlineRefusalReleasesRealStaging into SSDPrefixCacheReservationTests - excludesInferenceControls into WatchdogRecoveryTests - the rewritten #777 MTP config/beta suites into ConfigTests and BetaFeaturesTests (MTPConfigTests stays deleted) - vision/DeepStack/media additions into the split EngineV2Vision* and MediaIngest suites - the #704 writeDeferred provider-writer tests, first-content-budget wire test, #730 serial-privacy subtests, and #752 benchmark-control tests into their branch successors Master-added test files were audited against the lane architecture: the Qwen3.8 production canary trio moved to Live/ (with a #filePath depth fix), SchedulerPrefillDecision's real-weights suite split out to Probes/ behind its per-probe gates, and hermetic additions stay in the default lane. docs/developer/test.md updated to match. Production-code unions of note: registry.go keeps both the branch's PrivateOnly/catalog public-listing gates and master's per-model capability gating; provider_writer.go carries both determinism-seam families; LaunchAgent forwards the union of both env allowlists; postgres_model_registry.go extends nil-canonicalization to master's new required_provider_capabilities column. The branch-side prefill-keepalive tests were dropped with the feature master removed in #704. Two master-branch interaction defects fixed so the merged tree is green: the store contract harness now re-seeds the usage_totals counter row its truncate wipes (the #665 migration guard treats the bare marker as corruption), and the #730 log-report migration test restores the pooled connection's search_path before release. Submodules: mlx-swift and mlx-swift-lm test branches merged with master's 0.32.2-era pins; relocated suites verified present. Verified: go vet + full coordinator/testbed tests green; store contracts green twice consecutively against real Postgres 16; swift build --build-tests clean; default Swift lane 2,264 tests in 248 suites all passing.
The anchored ^TestBenchmark_ selector in the approval-gated benchmark lane no longer matches the four TestBenchmark* policy/configuration tests, and the root e2e package is excluded from the coordinator race gate — so no CI lane executed them (Codex P2). TestQwen38GatePolicy and TestQwen38ExpectedBuiltKVBackend had the same gap since #700 (they match neither the benchmark selector nor any TestIntegration_ lane). All six are pure logic and race-clean, so they now run by exact name with executed-count tripwires in the always-on CI job, which beats parking them behind benchmark approval.
4 participants
Integrate Qwen3.8-27B full VLM into Darkbloom
PR stack and contributor lineage
Layr-Labs/mlx-swift-lm#118— runtime dependency, now merged and pinned here at upstream81dd564fef05fa9e8449ed8a6695aedad92a7ff0(tree-identical to reviewed PR head07c08af4cd41b109aeac6e7f1d0aad5de38631cb).5376f62d23f9d5e7ced128e06e2afbde284754d4adds the final security, rollout, MTP-default, media, E2E, and release-safety work. Signed review-fix commit75d2717227f1fecb468ce466bca633f71e1b73fahardens runtime-policy recovery, the resume-specific challenge deadline, and alias-fallback runtime defaults. Signed startup-fix commit9bc95812e6e7146f03defff2fc7b1bd02e32a6beexplicitly binds the canary/runtime metallib before NAX detection. Signed compatibility/publication/E2E-fix commitfebe3cc011a2cb53b901a46fc93da0264593dc4cpreserves legacy reconnects, carries protected capability requirements through the canonical registration workflow, and prewarms the exact Qwen slot before live inference. Signed pin commitda118c6114629a6e91d158f66b4cd281baf00ce9advances the gitlink to upstream's merged runtime commit81dd564fef05fa9e8449ed8a6695aedad92a7ff0. Signed final review-fix commit6d3ebc50cefba35a6138805f04d222cae46a7a93fixes forced Qwen thinking, resolved-model parser binding, and pending-load cleanup. Signed prompt-parity fix48d3e4abc9ec3c49b3dd9ff6c2a9a7416a78c4f4mirrors forced-tool thinking in the sidecar; preceding code headfa2facbde93faaca8eb80bc2bfbb086df82bab63records the canonical color-bar fixture and completed post-change encrypted E2E. Performance provenance is exact: decode medians at9bc95812e6e7146f03defff2fc7b1bd02e32a6be, TTFT/RSS atfebe3cc011a2cb53b901a46fc93da0264593dc4c, with provider Swift/runtime trees byte-identical throughfa2facb.This PR is ready for review. The first exact M5+NAX live canary reached the real hardware gate and exposed the startup-binding defect documented below; the corrected full live canary, benchmark, and non-publication gates are complete at tested code head
fa2facbde93faaca8eb80bc2bfbb086df82bab63. Current root head48d3e4abc9ec3c49b3dd9ff6c2a9a7416a78c4f4adds prompt-sidecar parity on top of final review-fix6d3ebc50cefba35a6138805f04d222cae46a7a93and merged-runtime pinda118c6114629a6e91d158f66b4cd281baf00ce9; publication and rollout remain gated.Summary
Integrates Qwen3.8-27B as a dense
qwen3_5VLM across provider inference, request semantics, registry metadata, admission, encrypted coordinator/provider E2E coverage, and provider integrity controls.Exact PR head:
48d3e4abc9ec3c49b3dd9ff6c2a9a7416a78c4f4; signed integration base:5376f62d23f9d5e7ced128e06e2afbde284754d4.Runtime dependency: merged
Layr-Labs/mlx-swift-lm#118, pinned at upstream81dd564fef05fa9e8449ed8a6695aedad92a7ff0(tree-identical to reviewed source head07c08af4cd41b109aeac6e7f1d0aad5de38631cb).Before
After
Final model and artifact contract
EigenLabs/Qwen3.8-27B-4bit301e9e2767fd0efcfab7883004720ba3c9a552a1EigenLabs/Qwen3.8-27B-MTP-4bit329261c5e0b3f9c233485e682cb3b67b88c20a55qwen3.8-27bqwen/qwen3.8-27b>= 0.8.15libs/mlx-swift-lmat merged upstream81dd564fef05fa9e8449ed8a6695aedad92a7ff0The production speculative-decoding manifest/config digests, sizes, and R2 prefix remain approval-gated. Values exercised by the registry fixture are explicitly test-only and are not publication identifiers.
Changes
Provider and runtime
requireMetalor the NAX diagnostic. The live canary passes its explicit staged metallib URL; conflicting source or digest rebinding fails closed without changing the established MLX loader path.T×H×Wtower invocation for the whole video, then ordered frame-output splitting; it does not silently switch to frame-by-frame tower evaluation. Console/UI video upload is not enabled.preserve_thinkingis forwarded.Exact-model MTP default and rollback
EigenLabs/Qwen3.8-27B-4bittarget. The target model remains authoritative for every accepted token and final output.mtp_mode = "off"(includingdarkbloom beta disable mtp) restores target-only decoding.DARKBLOOM_CBV2_MTP=0is an independent process-level kill switch that restores target-only decoding.onremains available for other supported targets. Missing, malformed, unavailable, incompatible, or memory-inadmissible assistant state fails safely to target-only decoding with a stable reason.Coordinator, API, and security model
reasoning_parser,tool_call_parser) from registry runtime parameters; explicit caller values win and catalog-only fields cannot leak into provider requests.apple_m5andmlx_naxcapabilities for this concrete build; malformed or unknown requirements fail closed and unknown reported extras are inert.apple_m5/mlx_naxclaims are stripped and periodic nonce challenges remain the liveness proof.required_provider_capabilitiesthrough the canonical publish script and registration workflow. Values are trimmed, validated, and de-duplicated; existing models remain compatible with an empty array, while the exact Qwen target defaults toapple_m5,mlx_nax.load_modelcommand, waits for usable idle capacity from the exact provider/model/backend slot, and surfaces terminal load failures before inference.Validation evidence
Current root head is
48d3e4abc9ec3c49b3dd9ff6c2a9a7416a78c4f4; final review-fix head6d3ebc50cefba35a6138805f04d222cae46a7a93precedes it, and merged-pin headda118c6114629a6e91d158f66b4cd281baf00ce9pins upstream runtime81dd564fef05fa9e8449ed8a6695aedad92a7ff0, that merge tree is byte-identical to reviewed runtime source head07c08af4cd41b109aeac6e7f1d0aad5de38631cb. Decode three-run medians were captured at root9bc95812e6e7146f03defff2fc7b1bd02e32a6be; TTFT and/usr/bin/timemaximum RSS were captured atfebe3cc011a2cb53b901a46fc93da0264593dc4c; the canonical color-bar encrypted E2E 3/3 result was captured at preceding code headfa2facbde93faaca8eb80bc2bfbb086df82bab63. Provider Swift/runtime trees are byte-identical from9bc95812throughfa2facb;da118c6114629a6e91d158f66b4cd281baf00ce9changes only the gitlink from reviewed head07c08af4cd41b109aeac6e7f1d0aad5de38631cbto tree-identical upstream merge81dd564fef05fa9e8449ed8a6695aedad92a7ff0. Earlier integration and review-fix evidence remains attributable to heads5376f62and75d271722as recorded below.Final exact-M5 benchmark and canary
Method. Apple M5 Max, 128 GB; NAX reported
true; canonical metallib SHA-25611893e5b08ad4b0f2e917d052514f02f62ac9af6689a38c63fbc1321ac26d577. Every run used only exact immutable targetEigenLabs/Qwen3.8-27B-4bit@301e9e2767fd0efcfab7883004720ba3c9a552a1, standalone assistantEigenLabs/Qwen3.8-27B-MTP-4bit@329261c5e0b3f9c233485e682cb3b67b88c20a55, and runtime source07c08af4cd41b109aeac6e7f1d0aad5de38631cb(now pinned through tree-identical upstream merge81dd564fef05fa9e8449ed8a6695aedad92a7ff0). Decode values are three-run medians at root9bc95812e6e7146f03defff2fc7b1bd02e32a6be; TTFT and/usr/bin/timeRSS are from rootfebe3cc011a2cb53b901a46fc93da0264593dc4c; encrypted color-bar E2E is 3/3 at rootfa2facbde93faaca8eb80bc2bfbb086df82bab63. Provider Swift/runtime trees are byte-identical across those roots; the root diff from9bc95812throughfa2facbtouches only coordinator, workflow, and E2E files, merged-pin headda118c6114629a6e91d158f66b4cd281baf00ce9advances the gitlink to tree-identical runtime81dd564fef05fa9e8449ed8a6695aedad92a7ff0; review-fix head6d3ebc50cefba35a6138805f04d222cae46a7a93adds the final tool/revocation fixes, and current head48d3e4abc9ec3c49b3dd9ff6c2a9a7416a78c4f4restores prompt-sidecar parity. The older generic benchmark decode result was invalid and is explicitly excluded.MTP is proposal-only: the target model remains authoritative for every accepted token and the final output. Absent MTP configuration enables the assistant only for this exact target; explicit
mtp_mode = "off"(includingdarkbloom beta disable mtp) orDARKBLOOM_CBV2_MTP=0restores target-only decoding.Maximum RSS was
17,161,175,040bytes (15.98 GiB). The full exact-model canary surfaces passed. The encrypted real-process tool + canonical color-bar video E2E passed 3/3 with elapsed times55.77 s,56.21 s, and56.42 s; logs place tool generation at approximately1.3–1.4 sand video generation at approximately1.84–1.95 s.Receipt SHA-256:
4625e377de5cdcadeef3b90dc72601df9b7920918f3cda769ac8ebcf62f06d35.No artifact/model publication or upload, catalog or alias mutation, provider release/deploy, database mutation, or production traffic change occurred.
Initial live canary on Apple M5 Max/macOS 26.5.1 failed before model load after 0.470 seconds with
Qwen3.8 canary requires shared provider capabilities: mlx_nax(Qwen38ProductionCanaryTests.swift:21). The canary had staged the canonical metallib beside the XCTest bundle, but the production locator used the process executable directory while MLX C++ resolvescurrent_binary_dirthroughdladdr; binding therefore returned no hash and capability detection correctly withheldmlx_nax.Startup-fix head
9bc95812e: the canary now passes the explicitensureMetallibColocated()URL into the retained anonymous metallib binding beforerequireMetaland the NAX diagnostic.swift build --build-testspassed; the focusedMetallibHashTests|ProviderRuntimeCapabilityTests|StartCommandTests|Qwen38ProductionCanaryTestsselection ran 28 tests with 27 passes, 0 failures, and 1 expected live-opt-in skip. The canonical metallib remained stable at SHA-25611893e5b08ad4b0f2e917d052514f02f62ac9af6689a38c63fbc1321ac26d577.Compatibility/publication/E2E-fix head
febe3cc011: freshness coverage passed normally and under-race(3 top-level tests + 8 subcases each); prewarm coverage passed normally and under-race(3 tests each); Qwen live opt-out coverage passed 2 tests with 1 expected skip; the publication mock passed using fakeswift/gcloud/aws, so no upload occurred.gofmt -d,git diff --check,bash -n, and workflow YAML parsing were clean.Review-fix head
75d271722:gofmt -dproduced no output;git diff --checkwas clean; focused default-vet selection passed 5 top-level tests plus 12 subtests (17/17);go vet ./coordinator/apiwas clean; the same focused selection under-racepassed 17/17 with no race reports; fullgo test -count=1 ./coordinator/apipassed.Runtime PR fix: allow Swift providers without Python privacy caps #118 at
07c08af: 24 focused tests passed; full runtime suite passed with 1,504 tests, 1 skip, 0 failures.Prompt-parity head
48d3e4abc9e: sidecar required/named Qwen normalization now forces the same tool-only template context as ProviderCore.cargo fmt --check, focused regression, full 82-test sidecar suite, and pre-push coordinator tests passed.Final review-fix head
6d3ebc50cef: forced Qwen tool choices render tool-only prompts; effective parsers are validated after model resolution/default injection and every fallback; capability revocation clears newly ineligible pending loads. Focused normal/race tests passed; fullcoordinator/apiandcoordinator/registrypassed; full providerswift testpassed 82 XCTest plus 2,290 Swift Testing cases with 0 failures.Merged-pin head
da118c61146: upstream squash commit81dd564fef05fa9e8449ed8a6695aedad92a7ff0is tree-identical to reviewed head07c08af4cd41b109aeac6e7f1d0aad5de38631cb. Re-run validation passed: runtimeswift testexecuted 605 XCTest cases plus 899 Swift Testing cases (1 expected skip, 0 failures); providerswift testexecuted 82 XCTest cases plus 2,289 Swift Testing cases with 0 failures.Root provider suite: 2,367 expanded; 2,326 passed, 41 intentionally live-gated skips, 0 failures. Canonical root MLX pin was
0a725e3; staged metallib SHA-256 was11893e5b…d577.Focused provider selection: 329 selected; 328 passed and the one exact live canary was skipped because live opt-in was not supplied.
make coordinator-test: 24 packages passed plus 3 packages with no test files.Committed-head
go test -race ./...incoordinator: 24 packages passed plus 3 with no test files; no race failures.golangci-lintat CI-pinned v2.1.6: 0 issues.Coordinator build passed.
Prompt parity: 42 immutable vectors passed; 375/375 load requests succeeded at 25 QPS with 0 errors, mismatches, or restarts.
Qwen E2E gate-policy coverage passed. The real process-boundary test compiles and skips without explicit live opt-in rather than manufacturing a live result.
Focused provider suites cover the full-grid image/video path and ordered splitting.
Final independent code and security audits reported no findings; the added-line secret scan found no credential, private-key, or token material.
Pre-existing non-fatal Darwin linker warnings (
LC_DYSYMTAB) appeared during committed-head race testing; the tests passed.Remaining pre-merge and launch gates
true; all canary surfaces passed.Rollback and operational posture
Code rollback is a root-commit/gitlink revert. MTP rollback is immediate and exact-model scoped: set
mtp_mode = "off"orDARKBLOOM_CBV2_MTP=0to restore target-only decoding. Operational rollback keeps the alias/catalog inactive or moves the alias away from the concrete build after separate human approval.This PR performed no R2 upload or mutation, no catalog or alias mutation, no provider release, no GitHub release, no database mutation, and no production traffic change. The target/MTP artifacts and production metadata remain unshipped and approval-gated.