AXQuant is a precision allocator for Apple Silicon. It inspects a supported Safetensors checkpoint, assigns 4-bit, 6-bit, 8-bit, or BF16 per tensor, keeps sensitive layers (norms, heads, routers, vision/audio, MTP) at hard floors, and writes the manifests AX Engine and MLX-LM need. Convert still goes through MLX. AX Engine is speed; AXQuant chooses the precision mix.
It does not train the source model or add new capabilities. The goal is a smaller, cheaper checkpoint that still behaves well.
Current PyPI / certified pin: v1.8.1
(axquant==1.8.1). The 1.8 inspect → plan → affine U32 convert → certify path
stays.
1.9.0 (this branch, Apple / MLX): smarter allocation under one memory
budget via diagnose-joint and plan-joint. It does not replace 1.8 convert.
CUDA / NVFP4 is 2.x. See
docs/guides/experimental-joint-interaction.md
and docs/releases/1.9.0.md.
Install from PyPI, then convert. You do not need to clone this repository.
Apple Silicon Mac (M1–M5) and Python 3.11+. Conversion needs the MLX extra; that stack
only runs on arm64 macOS. Use Homebrew brew install python@3.13 or
python.org.
Always install into a virtual environment. Homebrew Python is
PEP 668 externally managed: a bare
python -m pip install axquant against system Python fails with
externally-managed-environment. That is expected; do not pass
--break-system-packages.
Copy the block as a whole (plain ASCII quotes; single-quote the extra so zsh does not treat
[mlx] as a glob):
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
python -m pip install 'axquant[mlx]==1.8.1'
axquant --helpWith the venv active, python and axquant are both from .venv. Without activate, call
.venv/bin/axquant directly.
| Goal | Command |
|---|---|
| Convert / analyze / evaluate (typical) | python -m pip install 'axquant[mlx]==1.8.1' inside a venv |
| Inspect / plan / report only (no Metal) | python -m pip install 'axquant==1.8.1' inside a venv |
| Global CLI via Homebrew tooling | brew install pipx && pipx install 'axquant[mlx]==1.8.1' |
If zsh prints missing end of string, a curly/smart quote usually got pasted. Re-type the
line or paste only from the fenced block above.
Package index: pypi.org/project/axquant. Wheels and checksums also ship on GitHub Releases (not the GitHub Packages tab; that UI is for npm/containers, not pip).
Point quantize at a local BF16 Safetensors directory. --target-bpw defaults to 4.8.
The result is a development checkpoint — good for trying the model locally, not a public
quality or speed claim.
axquant quantize /path/to/model-bf16Useful options:
# Choose the bit budget and output folder
axquant quantize /path/to/model-bf16 --target-bpw 4.8 --output ./AXQuant-output
# Hugging Face id (downloads only when you pass --allow-download)
axquant quantize Qwen/Qwen3.6-27B --allow-download --revision COMMIT_SHA
# Confirm the family is convertible before spending the convert
axquant inspect --model /path/to/model-bf16 --output inventory.jsonLoad the output with MLX-LM:
python -m pip install -U mlx-lm
mlx_lm.generate --model ./AXQuant-output --prompt "Hello" --max-tokens 64 --temp 0.0Skip convert and use a ready-made pack from
AutomatosX on Hugging Face
(certified AXQ).
axquant quantize --help lists every flag. Family-specific notes (ASR normalization, VL
image smoke, recipes) are under
Simple development conversion.
Conversion needs enough unified memory for the source model. Public certification of a pack is a separate, evidence-gated path — see Current status.
Start here
Product status and packs
- Current status (tiers, Hub catalog, known gaps, 72 h endurance)
Operators
- From-source install (Mac)
- Simple development conversion
- Staged development conversion
- CLI workflow
- Measured planning and validation
- Evidence and safety boundaries
- Model naming
Contributors
BF16 Safetensors checkpoint supported by its promoted MLX backend
(pin a revision for measured/release evidence)
│
▼
inspect → plan → convert → runtime-check / validate
│ │
│ └── AXQuant-optimized MLX checkpoint
│ + AX Engine runtime metadata
│ + plan, manifest, and provenance
└── model inventory and protection boundaries
Most users only need Install and Convert:
python -m pip install 'axquant[mlx]==1.8.1'
axquant quantize /path/to/model-bf16The staged journey (inspect → plan → convert → validate → publish) is for measured releases and public claims. See Simple development conversion and Measured planning and validation.
AXQuant converts unquantized Safetensors checkpoints of families at the convertible tier or
above through the public runtime backend promoted for that architecture. Text families use
MLX-LM. Qwen3-VL uses MLX-VLM with its vision tower protected at BF16. Qwen3-ASR uses MLX-Audio
with its audio tower protected at BF16; the pinned upstream thinker.* checkpoint must first be
normalized with scripts/hf_to_mlx_bf16.py, which records axquant_source.json. Planning directly
from the unnormalized ASR export is rejected because it contains a duplicated tied LM head and
runtime-specific tensor layouts.
A pinned source revision is mandatory for measured sensitivity and release evidence; an unpinned local source is permitted only for development workflows. MoE expert stacks quantize as fused switch modules with a uniform per-group precision, and routers keep an 8-bit floor. The checkpoint must use the expected configuration and indexed Safetensors layout. Remaining recognized families (for example Nemotron Super/Ultra) stay inspect-only until promotion evidence exists.
A successful conversion produces a new portable MLX model directory containing:
- mixed-precision model weights and standard MLX configuration files;
- the exact quantization plan used for the conversion;
- an AXQuant artifact manifest with checksums and provenance;
- architecture-specific runtime metadata for AX Engine, MLX-LM, MLX-Audio, or MLX-VLM;
- an AX Engine native manifest when the runtime tool is available;
- a byte-preserved external MTP sidecar by default, or an explicitly prepared development sidecar with transform-level provenance;
- a raw, checksummed BF16 sidecar for protected vision tensors when MLX-LM excludes them, or protected modality tensors in the main MLX-Audio/MLX-VLM checkpoint.
The artifact manifest records authoritative main-model and total logical parameters, physical Safetensors bytes, and measured BPW. The language-model output remains usable as a standard MLX checkpoint. AX Engine consumes the additional AXQuant metadata for runtime-specific behavior; MLX-LM may ignore that metadata and use ordinary decode.
Uniform quantization gives every eligible tensor the same precision; rule-based per-module overrides assign precision by name pattern. AXQuant instead allocates precision per tensor from a budget-constrained solve over measured sensitivity, so the model spends more bits where the measurement shows it matters and fewer where it does not.
Its design centers on:
- mixed precision: 4-bit, 6-bit, 8-bit, and BF16 assignments, with an experimental
2/3-bit range for robust trunk tensors (AX Engine gates them behind
AX_ENGINE_2BIT_EXPERIMENTAL/AX_ENGINE_3BIT_EXPERIMENTAL); - quality protection: hard precision floors for sensitive model components;
- MTP awareness: explicit MTP detection, protection, validation, and runtime metadata;
- workload awareness: separate objectives for general and agent/coding workloads;
- real deployment cost: actual artifact bytes, unified memory, latency, and throughput;
- reproducibility: revision-pinned release inputs, deterministic artifacts, checksums, and manifests;
- fail-closed conversion: incomplete plans or unmatched modules stop conversion;
- independent implementation: public APIs and research without reused quantizer internals.
The latest tagged toolkit version is 1.8.1 (GitHub tag v1.8.1, PyPI axquant==1.8.1;
packaging classifier: Beta). Its inspection, planning, conversion, runtime-check,
validation, and publication-gating commands are implemented and covered by the test suite.
Certification is checkpoint- and evidence-specific; a working command does not by itself
certify an output.
Ready-made packs: AutomatosX on Hugging Face
(collections,
certified AXQ).
Certification host (from now on): conversion, all Tier 1, and all Tier 2
certifications must run on Mac Studio M2 Ultra, 192 GB (host id df-macstudio-m2)
with Ext12T. Do not convert or certify on df-macbookpro-m5 or df-macbookpro-m3.
Existing certificates stay bound to the host recorded in each JSON record
(df-macstudio-m2, df-macbookpro-m5, or df-macbookpro-m3) until recertified on
df-macstudio-m2. The frozen M0–M8 flagship campaign schema still names
df-macbookpro-m5 until that contract is versioned separately.
AX Engine 6.15.0 passed a 72-hour endurance soak on df-macmini-03
(report).
Headline matrix below lists public catalog packs only (dual Tier 1+2 first).
Full list of every AXQ certificate record (including unlisted no-MTP siblings and
evaluation archives): docs/certifications/full-list.md.
Listed-pack index with Hub commits: docs/certifications/.
The tables are generated from certificate JSON
(python scripts/render_certification_docs.py --write); do not edit the table cells by hand.
| Pack family | Tier 1 (Quality) | Tier 2 (MTP -- Scoped) |
|---|---|---|
| Qwen3.8-27B MLX AXQ 4-bit MTP | Certified | Certified |
| Qwen3.8-27B MLX AXQ 6-bit MTP | Certified | Certified |
| Qwen 3.6 27B MLX AXQ 4-bit MTP | Certified | Certified |
| Qwen 3.6 27B MLX AXQ 6-bit MTP | Certified | Certified |
| Qwen 3.6 35B-A3B MLX AXQ 4-bit MTP | Certified | Certified |
| Qwen 3.6 35B-A3B MLX AXQ 6-bit MTP | Certified | Certified |
| Qwen3-VL 30B-A3B Instruct MLX AXQ 4-bit | Certified | N/A (no MTP) |
| Qwen3-VL 30B-A3B Instruct MLX AXQ 6-bit | Certified | N/A (no MTP) |
| Holo3-35B-A3B MLX AXQ 4-bit | Certified | N/A (no MTP) |
| Holo3-35B-A3B MLX AXQ 6-bit | Certified | N/A (no MTP) |
| Ornith-1.0-35B MLX AXQ 4-bit | Certified | N/A (no MTP) |
| Ornith-1.0-35B MLX AXQ 6-bit | Certified | N/A (no MTP) |
| Holo-3.1-35B-A3B MLX AXQ MXFP4 | Certified | N/A (no MTP) |
| GPT-OSS 20B MLX AXQ 4-bit | Certified | N/A (no MTP) |
| GPT-OSS 20B MLX AXQ 6-bit | Certified | N/A (no MTP) |
| GPT-OSS 120B MLX AXQ 6-bit | Certified | N/A (no MTP) |
| Qwen3.8-27B MLX AXQ MXFP4 MTP | Certified | Not Certified |
| Qwen3.8-27B MLX AXQ 8-bit MTP | Certified | Not Certified |
| DeepSeek V4 Flash MLX AXQ 2-bit MTP (exp.) | Certified | Not Certified |
| Gemma 4 12B MLX AXQ 4-bit | Certified | Not Certified |
| Gemma 4 12B MLX AXQ 6-bit | Certified | Not Certified |
| Gemma 4 26B-A4B MLX AXQ 4-bit | Certified | Not Certified |
| Gemma 4 26B-A4B MLX AXQ 6-bit | Certified | Not Certified |
| Gemma 4 31B MLX AXQ 4-bit | Certified | Not Certified |
| Gemma 4 31B MLX AXQ 6-bit | Certified | Not Certified |
| DeepSeek V4 Flash-0731 MLX AXQ 2-bit MTP (exp.) | Not Certified | Not Certified |
| DeepSeek V4 Flash-0731 MLX AXQ 4-bit MTP | Not Certified | Not Certified |
| DeepSeek V4 Flash-0731 MLX AXQ MXFP4 | Not Certified | Not Certified |
| DeepSeek V4 Flash-0731 MLX AXQ 6-bit | Not Certified | Not Certified |
The sparse-expert (35B-A3B) Tier 2 path is closed on AX Engine 6.14.1 with the MoE exact
profile (async draft, verify-submit interval 8, pipeline granularity layer) on
df-macbookpro-m5.
Qwen3.8-27B AXQ 4-bit and 6-bit MTP packs are checkpoint Tier 1 + scoped Tier 2 MTP
certified on df-macbookpro-m3 (AX Engine 6.16.1, QWEN38_EXACT_MTP_PROFILE_ENV / async draft).
Non-MTP siblings remain Tier 1 only (Tier 2 N/A). Product default remains direct fallback;
acceleration is opt-in under the formal exact profile. Vision weights are BF16-protected, but
end-to-end image/video quality is not certified; see the
Qwen3.8-27B AXQ VL retention assessment.
Gemma 4 (12B / 26B-A4B / 31B) AXQ 4-bit and 6-bit fused assistant-MTP Hub packs are
checkpoint Tier 1 certified on df-macbookpro-m5 (size, matched quality, load). Tier 2
is not certified on any Gemma pack: formal assistant-MTP A/B pilots can clear speed gates
while greedy exactness fails when drafts are accepted. The 12B packs were rebuilt from
google/gemma-4-12b-it after the earlier non-IT base failed quality. Product default remains
direct fallback until Tier 2 gates pass on a released engine.
Qwen3-Coder-Next AXQ MXFP4 is checkpoint Tier 1 certified on df-macstudio-m2
(non-MTP direct-decode; Tier 2 N/A). The AXQ 4/6-bit siblings remain certified on
df-macbookpro-m5. GPT-OSS 20B AXQ 4-bit and 6-bit, and GPT-OSS 120B
AXQ 6-bit, are checkpoint Tier 1 certified on the same host (non-MTP; 120B via manual no-4-bit
agent-coding recipe). GPT-OSS 120B AXQ 4-bit is not certified (agent-coding retention 0.952
< 0.98) — see
evaluation record. DeepSeek V4 Flash AXQ 2-bit experimental (older DeepSeek-V4-Flash source)
is checkpoint Tier 1 certified on df-macstudio-m2 (generation viability;
MTP Tier 2 not claimed). 3-bit Flash SKUs are withdrawn (no unique quality
or size slot vs 2-bit / 4-bit). Flash-0731 ship SKUs are 2-bit, 4-bit g128,
MXFP4, and 6-bit g128 (deepseek-ai/DeepSeek-V4-Flash-0731@7872f01b); 0731
certificates are in progress
(2-bit eval). Other
catalog entries remain development artifacts unless their own exact revision
has a certificate.
Qwen3.8-2.4T-A95B experimental AXQ 2-bit is on the Hub at
AX-Qwen3.8-2.4T-A95B-MLX-AXQ-2bit-MTP
(layer-stack expert stream, ~1.13 TiB, measured 4.074 BPW, native MTP sidecar
packaged; acceleration not claimed). This revision will not be certified
— SSD paging is too slow for practical serving; it stays as a hobby /
curiosity pack. No AXQ 4-bit pack will be released for this base. Technical report:
docs/reports/qwen38-axq-2bit.md. Separate OptiQ 2/4-bit
repos are not AX Engine artifacts
(docs/reports/qwen38-optiq-experimental.md).
DeepSeek-V4-Pro-0813 experimental AXQ 2-bit MTP is the same Super-class
hobby path at
AX-DeepSeek-V4-Pro-0813-MLX-AXQ-2bit-MTP
(layer-stack expert stream of the official mixed FP4+FP8 snapshot; DSpark
sidecar packaged; acceleration not claimed). This revision will not be
certified — SSD paging is too slow for practical serving. No AXQ 4-bit
pack will be released for this base. Technical report:
docs/reports/deepseek-v4-pro-0813-axq-2bit.md.
MiniMax-M3 experimental AXQ 2-bit is at
AX-MiniMax-M3-MLX-AXQ-2bit
(BF16 source, layer-stack expert stream required, vision BF16 in-shard).
Config num_mtp_modules is not packaged MTP, so the leaf has no -MTP.
This revision will not be certified. MXFP4 sibling:
AX-MiniMax-M3-MLX-AXQ-MXFP4.
Kimi-K3 experimental AXQ 2-bit is at
AX-Kimi-K3-MLX-AXQ-2bit
(native MXFP4 source dequantized to affine 2-bit, stream required, MoonViT-V2 BF16 sidecar).
No packaged MTP. This revision will not be certified. No AXQ MXFP4 sibling.
- Published Certification Spec v1.0 and
axquant verify-certso a third party can re-check a local certificate bundle without network access. - Added
axquant optimizeto spend one explicit memory budget on weights plus optional KV cache. Infeasible requests fail before conversion. - Froze the affine U32 interchange that AX Engine and
MLX-LM already load. Public Hub names stay
4bit/6bitSKUs; measured BPW is the claim (migration). CUDA and other physical formats remain out of scope.
- Coding evaluation now uses a deny-default macOS Seatbelt policy with sealed inputs, explicit runtime/toolchain read scopes, network denial, and parent-owned output pipes. The pipe boundary closes a confused-deputy path where generated code could replace scorer log files with symbolic links. See SECURITY.md for the threat model and operating guidance.
- Qwen3.8-27B AXQ 4/6-bit language-path packs are checkpoint Tier 1 certified; the MTP variants also have scoped Tier 2 certificates on AX Engine 6.16.1 under the exact Qwen3.8 profile.
- Qwen3-VL 30B-A3B Instruct AXQ 4/6-bit and Holo3-35B-A3B AXQ 4/6-bit are checkpoint Tier 1 certified. Holo3 MTP alignment tooling now covers measure, decide, teacher-force, and staged adaptation, but the public Holo3 products remain direct-decode because Tier 2 did not pass.
- The generated full certification list separates the public catalog from unlisted evaluation records and keeps Tier 1 quality distinct from scoped Tier 2 acceleration.
- AX Engine 6.15.0 completed a 72-hour Qwen 3.6 AXQ 6-bit endurance run with 3,643/3,643 successful requests, zero errors, and no observed RSS leak or swap. This is runtime evidence, not a new checkpoint certificate (report).
- DeepSeek V4 Flash is convertible (mixed FP4+FP8 source → dequant/affine re-pack) with
experimental 2-bit recipes. The older-source AXQ 2-bit experimental pack is checkpoint
Tier 1 certified on
df-macstudio-m2(2bit). 3-bit Flash is withdrawn. Requires anmlx-lmbuild that includesdeepseek_v4(v1.6.0+). - Convert/inventory hardening for DeepSeek sanitizer renames, FP4 expert logical params,
MultiLinear
wo_adequant, and byte-preserved MTP sidecars (v1.6.0). - Patch: correct fused-gate shapes for even Qwen expert counts; MTP module fusion skip;
HC learnable scale aliases no longer invent
.scales(v1.6.1). - GPT-OSS 20B/120B MXFP4 sources are convertible through a fail-closed affine re-pack path; the 20B 4/6-bit and 120B 6-bit packs are checkpoint Tier 1 certified (v1.6.2). GPT-OSS 120B 4-bit failed agent-coding quality and is not published (Hub pack removed).
- Public certificate JSON is now validated through strict schemas and drives generated certification matrices; every versioned artifact schema is frozen behind canonical snapshots and a digest manifest (v1.6.2).
- Checkpoint Tier 1 and scoped MTP Tier 2 are separate claims. Exact Qwen 3.6 27B/35B-A3B revisions have scoped certificates, while Gemma-4, Qwen3-Coder-Next, and experimental DeepSeek V4 Flash packs publish their narrower checkpoint verdicts (v1.6.2).
- Coding-suite and general-holdout overlap share
campaign-overlap's CJK-awareaxquant-token-5gram-v2tokenizer; regenerate coding-suite manifests built under v1. GPTQ column codes use the jointround(w/s + z)form shared with AWQ (v1.5.1). - The flagship formal-host identifier is
df-macbookpro-m5(wasmbp-m5): the machine is a MacBook Pro M5 with 128 GB unified memory and an 18-core CPU. The id is the machine's canonical DNS name and a schema literal on the host contract, preflight, and certified-claim hardware scope. No campaign or claim ever bound the old id (v1.5.0).
campaign-overlapnormalization is Unicode-aware (axquant-token-5gram-v2): CJK and other non-ASCII scripts now produce real shingles instead of failing closed, unblocking flagship campaign freezes over multilingual datasets; ASCII-only reports are byte-identical to v1.campaign-overlap --id-fieldis repeatable with ordered fallback (defaultid, thentask_id), so one overlap run spans calibration corpora and strictQualityTasksuites (v1.4.1).- Quantized MTP sidecars can emit AX Engine's MLX-packed layout (
mlx-affine-packed-u32) with round-trip verification and--runtime-jsonmtp_sidecar_bitsstamping. Capability and contract gates still fail closed until an AX Engine build reports that layout as executable, so shipped public packs keep byte-preserved sidecars (see still-incomplete list below). benchmark-kernels --from-ax-engineingests the engine's raw kernel-latency documents into host-scoped tables that plug directly intoplan --latency-table.
See the v1.7.0 release notes for the complete change list and download verification instructions. Past tags keep their notes on GitHub Releases; the next tag's curated body is prepared under docs/releases/.
| Scope | Use today | Public certification status |
|---|---|---|
Qwen3.8-27B dense (qwen38-dense-v1) |
convertible; certified dense VLM language-path track |
AXQ 4/6-bit ± MTP are checkpoint Tier 1; MTP packs also have scoped Tier 2 on AX Engine 6.16.1. Vision stays BF16; end-to-end VL quality is not certified. Super-class 2.4T is a separate experimental stream. |
| Qwen 3.6 language paths | convertible; formal campaign / primary investment track |
27B dense + 35B-A3B MoE AXQ 4/6-bit are Tier 1 + scoped Tier 2 MTP certified; other revisions remain development evidence |
Qwen 3.5 dense; Qwen3.5-class 35B-A3B MoE + fine-tunes (Ornith-1.0-35B, Holo3-35B-A3B, Holo-3.1-35B-A3B) via qwen35-moe-v1; Qwen3 dense/Embedding/Next; MiniCPM5; Mistral/Devstral/Ministral |
convertible through their promoted MLX text paths |
Ornith 4/6-bit checkpoint Tier 1 (4bit, 6bit); Holo3 4/6-bit checkpoint Tier 1 (4bit, 6bit); Holo-3.1 MXFP4 checkpoint Tier 1 (MXFP4) |
DeepSeek-OCR-2 (deepseek-ocr2-v1, MLX-VLM) |
convertible thin — language MoE quantized, vision BF16-protected |
Development evidence only (runbook) |
Muse-Glimmer-30B (muse-glimmer-v1, MLX-VLM) |
convertible thin — dense language quantized, vision BF16-protected |
Development evidence only (runbook) |
| Gemma-4 12B / 26B-A4B / 31B AXQ 4/6-bit | convertible + fused assistant-MTP Hub packs |
Checkpoint Tier 1 certified; Tier 2 not certified; 12B from google/gemma-4-12b-it |
| DeepSeek V4 Flash | convertible thin path (FP4+FP8 re-pack; needs mlx-lm with deepseek_v4) |
2-bit experimental Tier 1 on df-macstudio-m2 (3-bit SKUs withdrawn; Hub repos removed); other packs development evidence |
| GPT-OSS 20B / 120B | convertible thin path (MXFP4 re-pack; needs mlx-lm with gpt_oss) |
20B 4/6-bit + 120B 6-bit Tier 1 on df-macbookpro-m5; 120B 4-bit not certified (agent-coding) and Hub pack removed |
| Qwen3-ASR 1.7B; Qwen3-VL 8B Instruct; Qwen3-VL 30B-A3B Instruct (MoE) | convertible with protected modality towers (MLX-Audio / MLX-VLM convert); 30B MoE: AX Engine primary + MLX-VLM compatible |
30B Instruct 4/6-bit Tier 1 on df-macbookpro-m5 / AX Engine 6.15.0; 8B / ASR still development |
| Nemotron 3 Nano | convertible thin path |
Development evidence only |
| Other or unmatched checkpoints | inspect-only |
Not eligible for conversion or certification |
The detailed registry-derived matrix below is authoritative. Run axquant support-matrix for the
exact tier of a checkpoint before beginning work.
Further reading: AXQ model fleet v2 migration and audit, migration guide (v1.1.x → v1.2.0), migration guide (v1.0.x → v1.1.x), environment compatibility matrix, and known issues.
Release artifacts are built and signed (keyless Sigstore attestation) by the release workflow;
verify a downloaded dist with gh attestation verify <file> --repo defai-digital/axquant and
shasum -a 256 -c SHA256SUMS.txt.
AXQuant separates checkpoint and acceleration claims. Dual Tier 1 + scoped Tier 2 certificates
now exist for both Qwen3.8-27B AXQ 4/6-bit MTP (AX Engine 6.16.1 on df-macbookpro-m3)
and Qwen 3.6 27B / 35B-A3B AXQ 4/6-bit MTP (AX Engine 6.14.x on df-macbookpro-m5).
The exact Qwen 3.6 27B AXQ 6-bit revision, for example, has passed:
- Tier 1 (checkpoint): measured plan and size, matched general/agent-coding quality, zero-fallback conversion, and the safe default runtime route (certificate).
- Tier 2 (MTP acceleration, scoped): greedy exactness plus ≥1.20× token-weighted and ≥1.10× prompt-median decode speedup on authorizing decode-heavy profiles on MacBook Pro M5 (128 GB, 18-core) with AX Engine 6.14.0 under the formal exact MTP contract (certificate).
Product default remains Qwen linear MTP direct fallback (safe Tier 1 default). The certified acceleration route is the formal opt-in exact / certification-candidate contract—not a promise that every short-answer prompt is faster. Full M0–M8 flagship publication remains a separate campaign track. Other packs and revisions remain development artifacts until separately certified; a certificate never promotes a family by association.
AXQuant records an evidence-backed support tier for every recognized model family
(certified / convertible / inspect-only). Conversion requires at least the convertible
tier; tier promotion requires recorded promotion evidence, and certification requires the full
release audit. The current tier matrix:
| Family | Adapter | Tier |
|---|---|---|
| Qwen3.8-27B dense VLM | qwen38-dense-v1 |
convertible; AXQ 4/6-bit ± MTP checkpoint Tier 1; MTP packs also scoped Tier 2 on AX Engine 6.16.1; vision BF16; not the Qwen 3.6 campaign track |
| Qwen 3.6 (27B dense + 35B-A3B MoE language paths) | qwen36-v1 |
convertible; formal campaign / primary investment track |
| Qwen 3.8 2.4T-A95B text MoE | qwen38-moe-v1 |
convertible thin development path; native layer-stack expert stream required; no cert track |
| Qwen 3.5 dense | qwen35-dense-v1 |
convertible; development claims only |
| Qwen3.5-class 35B-A3B MoE + fine-tunes (Ornith-1.0-35B, Holo3-35B-A3B, Holo-3.1-35B-A3B) | qwen35-moe-v1 |
convertible; Ornith 4/6-bit Tier 1 (4bit, 6bit); Holo3 4/6-bit Tier 1 (4bit, 6bit); Holo-3.1 MXFP4 Tier 1 (MXFP4); not Qwen 3.6 cert track |
| DeepSeek-OCR-2 | deepseek-ocr2-v1 |
convertible thin via MLX-VLM; development only (runbook) |
| Muse-Glimmer-30B | muse-glimmer-v1 |
convertible thin via MLX-VLM; development only (runbook) |
| Qwen3-Next / Coder-Next (hybrid MoE) | qwen3-next-v1 |
convertible; Coder-Next AXQ MXFP4/4/6-bit checkpoint Tier 1 certified (MXFP4, 4bit, 6bit); no MTP / Tier 2 N/A; other Next checkpoints remain development |
Qwen3 dense + Embeddings (model_type=qwen3) |
qwen3-dense-v1 |
convertible; includes Qwen3-Embedding-0.6B/4B/8B |
| Qwen3-ASR 1.7B | qwen3-asr-v1 |
convertible after pinned MLX-Audio BF16 normalization; audio tower protected |
| Qwen3-VL 8B Instruct | qwen3-vl-v1 |
convertible through MLX-VLM; vision tower protected |
| Qwen3-VL 30B-A3B Instruct (MoE) | qwen3-vl-moe-v1 |
convertible thin — exact Instruct only; MLX-VLM convert; AX Engine primary + MLX-VLM compatible; vision BF16; no MTP; 4/6-bit Tier 1 certified on df-macbookpro-m5 |
| MiniCPM5 dense | minicpm5-dense-v1 |
convertible; development claims only |
| Gemma-4 dense / unified | gemma4-dense-v1 |
convertible — gemma4_unified prepared at convert time to gemma4 text path; multimodal sidecars preserved |
| Nemotron 3 (thin) | nemotron3-v1 |
convertible only for Nano-30B-A3B hybrid MoE; Super/Ultra inspect-only until a Nemotron-specific stream convert exists |
| Mistral / Devstral dense | mistral-devstral-dense-v1 |
convertible — model_type=mistral (MLX remaps to llama) or llama exports named Mistral/Devstral/Ministral |
| Mistral 3 / Ministral-3 shell | mistral3-dense-v1 |
convertible — language path via nested text_config; vision stripped by MLX sanitize |
| GPT-OSS (MoE) | gpt-oss-v1 |
convertible thin — MXFP4 re-pack via --allow-quantized; 20B 4/6-bit + 120B 6-bit Tier 1 (20B-4, 20B-6, 120B-6); 120B 4-bit not certified (record); no MTP |
New families start at inspect-only until promotion evidence exists. Run
axquant support-matrix and axquant support-policy for the registry-derived source of truth.
| Area | Current support |
|---|---|
| Platform | macOS on Apple Silicon (M-series) with MLX |
| Conversion input | Unquantized Safetensors checkpoint supported by the promoted MLX backend; revision pin required for measured/release evidence |
| Conversion targets | Qwen3.8-27B; Qwen 3.6 27B/35B-A3B; Qwen 3.5 dense; Qwen3.5-class 35B-A3B MoE / Ornith / Holo3; Qwen3 dense + Embeddings; Qwen3-Next/Coder-Next MoE; Qwen3-ASR 1.7B; Qwen3-VL 8B Instruct; Qwen3-VL 30B-A3B Instruct MoE; DeepSeek-OCR-2; Muse-Glimmer-30B; MiniCPM5; Gemma-4; Nemotron Nano only (thin); Mistral/Devstral/Ministral and Mistral3 shells; GPT-OSS MoE (thin MXFP4 re-pack) |
| Family support tiers | certified / convertible / inspect-only, recorded in every inventory and plan |
| Precision choices | 4-bit, 6-bit, 8-bit, and BF16 (plus experimental 2-bit and 3-bit behind AX Engine's documented gates); measured affine, DWQ-clipped affine, portable AWQ, and GPTQ |
| Planning | Manual recipes and a planner that consumes measured sensitivity artifacts |
| MTP | Detection, byte-preserved sidecars, and an opt-in Qwen 3.6 AX Engine layout backend |
| Primary runtime | AX Engine for text tracks and Qwen3-VL MoE (30B-A3B Instruct); MLX-Audio for Qwen3-ASR; MLX-VLM for dense Qwen3-VL 8B |
| Compatibility runtime | Architecture-specific standard inference; generic text artifacts use MLX-LM; VL MoE also supports MLX-VLM (vision smoke / Hub consumers) |
| Output integrity | Atomic conversion, exact parameter coverage, measured BPW, checksums, manifests, and runtime metadata |
Conversion and generation smokes establish artifact compatibility, not model quality or release certification. The public model cards and manifests record each checkpoint's exact source revision, plan, achieved BPW, sidecars, and evidence limits. Keep hardware names, network addresses, and local artifact paths in local operational records rather than public docs.
The default 4.8 BPW budget can be infeasible when protection floors raise the policy minimum
(for example, Gemma-4, Devstral, and Mistral3). The simple quantize path raises the requested
budget once to the computed minimum and records that decision. Use an explicit --target-bpw at
or above the floor when the budget must be fixed.
Public packs on AutomatosX are development evidence unless an exact immutable revision is linked to a certificate. As of v1.7.0 the public catalog includes multiple certified families — Qwen3.8-27B, Qwen 3.6 27B/35B-A3B, Qwen3-VL 30B-A3B Instruct, Holo3-35B-A3B, Gemma 4, Qwen3-Coder-Next, GPT-OSS, and experimental DeepSeek V4 Flash 2-bit (3-bit SKUs withdrawn; Hub repos removed) — each bound to its own certificate. The headline matrix under Current status lists the listed public packs; the full list includes unlisted no-MTP siblings and evaluation archives.
The stable repository names remain the canonical model identifiers. The certified Qwen 3.6 27B
AXQ 6-bit repository serves v3 on main with the preceding audited artifact pinned at v2.
The original v2 fleet rebuild (see model-fleet-v2.md) still serves
those packs on main tagged v2, with artifacts predating v2 recoverable at legacy-pre-v2.
Later certified families (Qwen3.8, Holo3, Qwen3-VL 30B, GPT-OSS, Gemma 4 IT rebuilds, and
others) are not that v2 migration; pin the Hub commit from the certificate. Temporary
edition-suffixed migration repositories are not part of the public catalog.
Each repo ships a full model card (README.md) plus public AXQuant provenance
(axquant_manifest.json, axquant_plan.json, runtime metadata, sidecars when
present). Cards are multi-family aware and state evidence limits explicitly.
The table below lists the current public packs under their stable names; the
AutomatosX collections
group them by family (Qwen, Gemma, coding, vision, embeddings, and more), with a
certified AXQ
starting list and a
complete index.
The BPW values are rounded from each current public manifest's
measured_main_bpw; the linked model card and manifest remain authoritative.
No distinct AXQ-4bit pack is published when protection floors collapse the low-memory budget
onto the same (or near-identical) artifact as the 6bit budget — publishing both would only
mislead. Affected bases today: Qwen3.5-9B, MiniCPM5-1B, and Ministral-3-8B (use the
6bit pack only).
| Pack | Main-model BPW | Notes |
|---|---|---|
AX-Qwen3.8-27B-MLX-AXQ-MXFP4-MTP |
4.8441 | Tier 1 MXFP4 trunk + packaged MTP (cert); MTP BF16; scoped Tier 2 not certified on Studio (eval) |
AX-Qwen3.8-27B-MLX-AXQ-4bit |
5.0667 | Tier 1 no-MTP (cert); dense hybrid qwen3_5 |
AX-Qwen3.8-27B-MLX-AXQ-4bit-MTP |
5.0667 | Tier 1 + scoped Tier 2 MTP on df-macbookpro-m3 (Tier 1, Tier 2); Studio recert not certified (eval) |
AX-Qwen3.8-27B-MLX-AXQ-6bit |
5.8448 | Tier 1 no-MTP (cert) |
AX-Qwen3.8-27B-MLX-AXQ-6bit-MTP |
5.8448 | Tier 1 + scoped Tier 2 MTP on df-macbookpro-m3 (Tier 1, Tier 2); Studio recert not certified (eval) |
AX-Qwen3.8-27B-MLX-AXQ-8bit-MTP |
8.6271 | Tier 1 8-bit affine + packaged MTP (cert); MTP BF16; scoped Tier 2 not certified (short Studio probe passed; not --full) (eval) |
AX-Qwen3.6-27B-MLX-AXQ-4bit |
5.418315 | Tier 1 no-MTP (cert); language path matches MTP sibling without mtp.safetensors |
AX-Qwen3.6-27B-MLX-AXQ-4bit-MTP |
5.418315 | Tier 1 + scoped Tier 2 MTP (Tier 1, Tier 2); product class 5p6bpw |
AX-Qwen3.6-27B-MLX-AXQ-6bit |
5.805849 | Tier 1 no-MTP (cert); language path matches v3 MTP sibling without mtp.safetensors |
AX-Qwen3.6-27B-MLX-AXQ-6bit-MTP |
5.805849 | v3 Tier 1 + scoped Tier 2 MTP certified (Tier 1, Tier 2); default route still direct fallback |
AX-Qwen3.6-35B-A3B-MLX-AXQ-4bit |
4.878782 | Tier 1 no-MTP (cert); language path matches MTP sibling without mtp.safetensors |
AX-Qwen3.6-35B-A3B-MLX-AXQ-4bit-MTP |
4.878782 | Tier 1 + scoped Tier 2 MTP (Tier 1, Tier 2); product default still direct fallback |
AX-Qwen3.6-35B-A3B-MLX-AXQ-6bit |
5.759473 | Tier 1 no-MTP (cert); language path matches MTP sibling without mtp.safetensors |
AX-Qwen3.6-35B-A3B-MLX-AXQ-6bit-MTP |
5.759473 | Tier 1 + scoped Tier 2 MTP (Tier 1, Tier 2); product default still direct fallback |
AX-Qwen3.5-9B-MLX-AXQ-6bit-MTP |
6.736665 | secondary; only published budget (floor-collapsed; no 4bit sibling) |
AX-gemma-4-12b-MLX-AXQ-4bit-MTP |
~4.90 | Tier 1 certified (cert); IT rebuild; assistant-MTP fused; Tier 2 not certified |
AX-gemma-4-12b-MLX-AXQ-6bit-MTP |
~6.00 | Tier 1 certified (cert); IT rebuild; assistant-MTP fused; Tier 2 not certified |
AX-gemma-4-26b-a4b-MLX-AXQ-4bit-MTP |
~4.90 | Tier 1 certified (cert); assistant-MTP fused; Tier 2 not certified |
AX-gemma-4-26b-a4b-MLX-AXQ-6bit-MTP |
~6.00 | Tier 1 certified (cert); assistant-MTP fused; Tier 2 not certified |
AX-gemma-4-31b-MLX-AXQ-4bit-MTP |
~4.90 | Tier 1 certified (cert); assistant-MTP fused; Tier 2 not certified |
AX-gemma-4-31b-MLX-AXQ-6bit-MTP |
~6.00 | Tier 1 certified (cert); assistant-MTP fused; Tier 2 not certified |
AX-Devstral-Small-2505-MLX-AXQ-4bit |
4.949963 | secondary coding/agent |
AX-Devstral-Small-2505-MLX-AXQ-6bit |
5.999989 | secondary coding/agent |
AX-Mistral-Small-3.1-24B-Instruct-2503-MLX-AXQ-4bit |
5.150021 | secondary; vision sidecar preserved |
AX-Mistral-Small-3.1-24B-Instruct-2503-MLX-AXQ-6bit |
5.999949 | secondary; vision sidecar preserved |
AX-MiniCPM5-1B-MLX-AXQ-6bit |
7.380428 | secondary fixture; only published budget (floor-collapsed; no 4bit sibling) |
AX-Nemotron-3-Nano-30B-A3B-MLX-AXQ-4bit |
4.799310 | thin Nano support |
AX-Nemotron-3-Nano-30B-A3B-MLX-AXQ-6bit |
5.990219 | thin Nano support |
AX-Qwen3-Embedding-0.6B-MLX-AXQ-4bit |
5.550330 | embedding; feature-extraction card |
AX-Qwen3-Embedding-0.6B-MLX-AXQ-8bit |
8.000275 | embedding |
AX-Qwen3-Embedding-4B-MLX-AXQ-4bit |
4.890183 | embedding |
AX-Qwen3-Embedding-4B-MLX-AXQ-8bit |
7.999979 | embedding |
AX-Qwen3-Embedding-8B-MLX-AXQ-4bit |
4.830057 | embedding |
AX-Qwen3-Embedding-8B-MLX-AXQ-8bit |
7.999911 | embedding |
AX-Qwen3-Coder-Next-MLX-AXQ-MXFP4 |
4.315400 | Tier 1 certified (cert); no MTP; factory host df-macstudio-m2 |
AX-Qwen3-Coder-Next-MLX-AXQ-4bit |
4.797752 | Tier 1 certified (cert); no MTP; corrected indexed-expert packing |
AX-Qwen3-Coder-Next-MLX-AXQ-6bit |
5.998996 | Tier 1 certified (cert); no MTP; corrected indexed-expert packing |
AX-gpt-oss-20b-MLX-AXQ-4bit |
5.055321 | Tier 1 certified (cert); no MTP; manual attention-8 / expert-4 recipe |
AX-gpt-oss-20b-MLX-AXQ-6bit |
6.000037 | Tier 1 certified (cert); no MTP; MXFP4 re-pack |
AX-gpt-oss-120b-MLX-AXQ-6bit |
6.576880 | Tier 1 certified (cert); no MTP; manual no-4-bit agent-coding recipe |
| (120B 4-bit not listed) | — | Not certified — agent-coding retention 0.952 < 0.98; further recert skipped; evaluation record |
AX-DeepSeek-V4-Flash-MLX-AXQ-2bit-MTP |
3.132899 | Tier 1 certified (exp.) (cert) on df-macstudio-m2; older DeepSeek-V4-Flash source; product class 2bit-experimental; MTP assets packaged |
AX-DeepSeek-V4-Flash-0731-MLX-AXQ-2bit-MTP |
3.133 | Not certified (eval); uniform 2-bit v0.1 + MTP; factory v-extract combined 0.887 < 0.90 on df-macstudio-m2; MTP assets packaged |
AX-DeepSeek-V4-Flash-0731-MLX-AXQ-4bit-MTP |
— | Reserved / not uploaded. -MTP is kept because the 2-bit sibling ships mtp.safetensors. Hub tree has no weights yet. Eval record is not a shipped-pack claim. |
AX-DeepSeek-V4-Flash-0731-MLX-AXQ-MXFP4 |
— | Stub / not uploaded. No -MTP until a sidecar ships. Eval; convert --q-mode mxfp4; 192 GB may block generate |
AX-DeepSeek-V4-Flash-0731-MLX-AXQ-6bit |
— | Stub / not uploaded. No -MTP until a sidecar ships. Eval; 192 GB factory host cannot run dual-suite generate |
AX-DeepSeek-V4-Pro-0813-MLX-AXQ-2bit-MTP |
4.097 | Not certified (hobby / curiosity Super-class stream pack). Official deepseek-ai/DeepSeek-V4-Pro-0813@72e1d323; ax_expert_stream.json required; DSpark sidecar packaged, acceleration not claimed. Report. No 4-bit sibling. |
AX-MiniMax-M3-MLX-AXQ-2bit |
4.143 | Not certified (hobby / curiosity Super-class stream pack). Official MiniMaxAI/MiniMax-M3@f0e1c1e0; ax_expert_stream.json required; config MTP flags are not packaged MTP (no -MTP). Vision BF16. |
AX-MiniMax-M3-MLX-AXQ-MXFP4 |
4.366 | Not certified (hobby / curiosity Super-class stream pack). Same MiniMax-M3 BF16 pin; --q-mode mxfp4; vision.safetensors sidecar; no packaged MTP. |
AX-Kimi-K3-MLX-AXQ-2bit |
4.018 | Not certified (hobby / curiosity Super-class stream pack). Official moonshotai/Kimi-K3@a590ce09; native MXFP4 experts dequantized to affine 2-bit; ax_expert_stream.json required; vision.safetensors sidecar; no packaged MTP. 2-bit only. |
AX-Ornith-1.0-35B-MLX-AXQ-4bit |
4.880062 | Tier 1 certified (cert); source deepreinforce-ai/Ornith-1.0-35B@5df2ed3f675c7beaa490328cc70bb573b65fb660; adapter qwen35-moe-v1; vision BF16; no MTP / not Qwen 3.6 cert |
AX-Ornith-1.0-35B-MLX-AXQ-6bit |
6.000062 | Tier 1 certified (cert); same source pin; vision BF16; no MTP / not Qwen 3.6 cert track |
AX-Holo3-35B-A3B-MLX-AXQ-4bit |
5.665439 | Tier 1 certified (cert); attention-6 / expert-4 recovery recipe; source Hcompany/Holo3-35B-A3B@208d5ae3a03f99d561f32ab5e606f73397a390ea; adapter qwen35-moe-v1; vision BF16; no MTP / not Qwen 3.6 cert |
AX-Holo3-35B-A3B-MLX-AXQ-6bit |
7.006493 | Tier 1 certified (cert); same source pin; vision BF16; no MTP / not Qwen 3.6 cert track |
AX-Holo-3.1-35B-A3B-MLX-AXQ-MXFP4 |
4.874068 | Tier 1 certified (cert); source Hcompany/Holo-3.1-35B-A3B@2bdb92851a8cd9d72cdd891fdf38cfcc7fefae2c; adapter qwen35-moe-v1; vision BF16; no MTP / not Qwen 3.6 cert |
AX-Holo-3.1-35B-A3B-MLX-AXQ-6bit |
7.507688 | Not certified (eval); same source pin; general retention 0.929 < 0.98 |
AX-Holo-3.1-35B-A3B-MLX-AXQ-8bit |
9.423039 | Not certified (eval); same source pin; general retention 0.929 < 0.98 |
AX-DeepSeek-OCR-2-MLX-AXQ-4bit |
~6.80 total | Development only (runbook); language trunk 4-bit, vision BF16; official deepseek-ai/DeepSeek-OCR-2@aaa02f3811945a91062062994c5c4a3f4c0af2b0 |
AX-DeepSeek-OCR-2-MLX-AXQ-6bit |
~8.40 total | Development only (runbook); language trunk 6-bit (experts), attention 8-bit, vision BF16 |
AX-Muse-Glimmer-30B-MLX-AXQ-4bit |
~5.95 total | Not certified (eval); mlx-vlm generate smoked on Studio; mlx-lm cannot score muse_glimmer; vision BF16; runbook |
AX-Muse-Glimmer-30B-MLX-AXQ-6bit |
~7.69 total | Not certified (eval); same Studio backend gap; language 6-bit, vision BF16 |
AX-Qwen3-ASR-1.7B-MLX-AXQ-4bit |
6.910001 | MLX-Audio; protected BF16 audio tower |
AX-Qwen3-ASR-1.7B-MLX-AXQ-6bit |
8.350084 | MLX-Audio; protected BF16 audio tower |
AX-Qwen3-VL-8B-Instruct-MLX-AXQ-4bit |
6.359976 | MLX-VLM; protected BF16 vision tower |
AX-Qwen3-VL-8B-Instruct-MLX-AXQ-6bit |
7.999975 | MLX-VLM; protected BF16 vision tower |
AX-Qwen3-VL-30B-A3B-Instruct-MLX-AXQ-4bit |
4.860055 | MoE VL; AX Engine primary + MLX-VLM; BF16 vision; no MTP; Tier 1 certified |
AX-Qwen3-VL-30B-A3B-Instruct-MLX-AXQ-6bit |
6.000054 | MoE VL; AX Engine primary + MLX-VLM; BF16 vision; no MTP; Tier 1 certified |
AX-Qwen3-VL-32B-Thinking-MLX-AXQ-6bit |
6.937967 | Not certified (eval); dense qwen3_vl Thinking; language 6-bit, vision BF16; agent-coding 0/0 at 64 tokens (think-trace) |
AX-Qwen3-VL-32B-Thinking-MLX-AXQ-MXFP4 |
4.833093 | Not certified (eval); same pin; language MXFP4, vision BF16 |
AX-Ministral-3-8B-Instruct-2512-MLX-AXQ-6bit |
5.999992 | Mistral3 language path; only published budget (near floor-collapse; no 4bit sibling) |
AX-Ministral-3-14B-Instruct-2512-MLX-AXQ-4bit |
5.610033 | Mistral3 language path |
AX-Ministral-3-14B-Instruct-2512-MLX-AXQ-6bit |
5.999912 | Mistral3 language path |
Development naming: AX-<Base>-MLX-AXQ-<4bit|6bit|8bit>[-MTP] (MLX-style bit labels,
not GGUF q4). Artifact editions are recorded in the model card and immutable Hub tags instead
of changing the repository identifier. The class is a planning budget, not a claim that every
tensor uses that width. Not every base publishes every class — see the floor-collapse note above.
Certified naming: a checkpoint Tier 1 certificate may retain the stable product-class
repository name when it pins the exact Hub tag/commit, artifact edition, hashes, and measured BPW.
An acceleration-bearing flagship uses
AX-<Base>-MLX-AXQ-MP-<measured-main-BPW>bpw[-MTP], rounded to two decimal places with decimal
half-up rules (for example MP-5p30bpw-MTP). target_class remains metadata.
Quick load (MLX-LM):
python -m pip install -U mlx-lm
mlx_lm.generate --model AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-6bit-MTP \
--prompt "Hello" --max-tokens 64 --temp 0.0Investment policy: axquant support-policy (formal campaign / primary investment track =
Qwen 3.6; Qwen3.8-27B is a separate certified dense VLM track; Nemotron = thin Nano only).
Regenerate a public card from a local pack:
python scripts/prepare_development_model_card.py \
--artifact /path/to/AX-...-MLX-AXQ-6bit-MTP \
--repo-id AutomatosX/AX-...-MLX-AXQ-6bit-MTP \
--artifact-edition 2Implemented now:
- indexed Safetensors inspection and logical parameter reconstruction;
- deterministic, provenance-bound tokenized calibration caches;
- resumable per-tensor MLX probes with 4/6/8/BF16 affine candidates and targeted DWQ/AWQ/GPTQ refinement;
- portable AWQ activation-scale search and GPTQ Hessian error compensation with convert-time refinement and affine packing;
- checksum-bound per-module activation capture (
capture-activations) feeding AWQ/GPTQ probes and conversion; - Qwen 3.6 tensor classification, MTP detection, and vision protection;
- Qwen3.8-27B dense language-path conversion (
qwen38-dense-v1) with BF16-protected vision and certified AXQ 4/6-bit ± MTP packs; - Qwen3-ASR and Qwen3-VL text-path quantization through public MLX-Audio/MLX-VLM backends, with BF16 modality-tower protection and real media runtime smokes;
- Qwen3-VL 30B-A3B Instruct MoE convert (AX Engine primary + MLX-VLM compatible) and Holo3 35B-A3B grafted-MTP alignment tooling;
- auditable manual recipes with mandatory precision floors;
- mixed-precision planning from compatible sensitivity reports;
- architecture-specific MLX conversion with plan-to-module coverage checks;
- atomic output staging that prevents partial final checkpoints;
- AX Engine manifest generation and runtime readiness checks;
- identical-checkpoint AX Engine MTP off/on benchmarking with greedy-output equality;
- deterministic quality/benchmark suites and complete-model MLX quality evaluation;
- validation gates for externally measured quality and performance evidence;
- guarded Hugging Face publication;
- tiered family support with declarative adapters (Qwen 3.6 formal campaign primary;
Qwen3.8-27B certified dense VLM; Qwen 3.5, Qwen3-ASR, Qwen3-VL, MiniCPM5, Gemma-4,
Mistral/Devstral, Mistral3, and Nemotron Nano at
convertible; Nemotron Super/Ultra remain inspect-only), including byte-preserving extraction of integrated MTP heads and protected vision into canonical checksummed sidecars; - development Hub model cards (
axquant.model_card/scripts/prepare_development_model_card.py) that sanitize provenance and document evidence limits for public packs; axquant quantize: one-command development conversion with explicit development-evidence labeling;- checksummed recipe bundles (
recipe-export,quantize --recipe) that bind published plans to user conversions without upgrading their evidence kind, resolvable locally or from revision-pinnedhf://references; prepared releases package their bundle automatically; - a registry-derived support matrix (
support-matrix) with investment posture andsupport-policybest practices (Qwen 3.6 campaign primary; Qwen3.8 certified separately; thin Nemotron Nano only); - per-layer KV-cache precision planning and runtime execution: prior-based
(
--kv-cache prior) and measured (analyze-kv+plan --kv-cache measured, digest-bound to the sensitivity report) planning, andruntime-check --runtime mlx-lm-kvexecutes the plan's exact per-layer table at runtime — one cache object per layer through MLX-LM's publicprompt_cache/QuantizedKVCacheAPI, with per-layer mixed precisions (e.g. 8-bit boundary + 4-bit interior layers) verified active on real artifacts. The ordinary generation smoke also applies the advisory global KV values. Families whose attention implementation rejects quantized caches fail closed (the hybrid Qwen 3.6 path awaits AX Engine-native KV, the scoped engine project); - a fail-closed measured-KV release chain: conversion packages the bound
kv_sensitivity.json(convert --kv-sensitivity) and publication re-verifies the digest and reproduces the exact per-layer allocation from the packaged report; - an evidence-bound head-to-head page renderer that loads only checksum-verified evaluation bundles and always lists unavailable mandatory baselines with their reasons;
- a bundled, clean-room-authored reference calibration dataset (160 samples across 7 domains —
coding, json, tool, multilingual, long-context, reasoning, general) with a
validate-calibration-datasetcommand, so a user without their own domain-representative calibration text can still run the full measured pipeline; an integration test proves the complete chain (inspect → tokenize-calibration → analyze → plan) closes end to end on it; - a repository evaluation task suite (
data/eval/) with 60 clean-room-authored tasks across four categories (coding, reasoning, json-tool, instruction) forevaluate-qualityandcompare-quality, covering python-syntax, JSON validity, exact match, regex, and token-F1 scoring.
Still incomplete (external evidence / runtime / deferred scope — not missing toolkit commands):
- Qwen 3.6 full M0–M8 flagship publication campaign is a separate process from the closed Tier 1/Tier 2 metric certificates for the 27B AXQ 6-bit v3 artifact; product default MTP remains fail-closed / direct fallback until an explicit runtime promotion;
- interaction-optimization evidence: the toolkit path exists
(
refine-select --interaction, holdout-safe by construction), but no bound candidate has yet been optimized against real measured development-role evaluations; - validated conversion evidence for any future official dense Qwen 3.6 sizes beyond current smokes;
- certification evidence for remaining secondary / inspect-only families (Nemotron Super/Ultra remain inspect-only; Ornith, DeepSeek-OCR-2, Muse-Glimmer, Qwen3-VL 8B, and Qwen3-ASR are still development evidence);
- quantized external MTP sidecars in production: the toolkit can emit the engine's
mlx-affine-packed-u32layout viaquantize-mtp-sidecar(with a fail-closed capability probe), but no AX Engine build yet reports that layout as executable, so every shipped public pack keeps a byte-preserved sidecar; - measured KV serving-quality evidence: the report-only artifact and
kv-serving-qualitycommand exist, but the dual-profile short/long-context measurements that would fill them have not been run; - vision-tower quantization (deferred scope — Qwen3-VL language paths convert, but vision towers remain BF16 until vision-specific evaluation evidence exists);
- per-expert (unfused) MoE precision (deferred scope — packed expert stacks quantize as fused switch modules with one precision per group; finer splits need MLX-LM-side support).
The validation-index, hardware-registry, compatibility-matrix, and release-audit commands
enforce release gate order, dual-profile completeness, and evidence binding.
Architecture-prior analysis, smoke probes, and manual plans are explicitly marked as non-release development evidence. They cannot support production-quality or performance claims.
Users should install from PyPI under Install. An editable checkout is only
needed to change the toolkit. Conversion needs .[mlx]; AX Engine manifest generation
needs ax-engine-bench on PATH:
git clone https://github.com/defai-digital/axquant.git
cd axquant
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip
python -m pip install -e '.[mlx]' # conversion path
# python -m pip install -e '.[dev,mlx]' # plus tests and lint
axquant --helpAXQuant uses a two-door model:
| Door | When | Command |
|---|---|---|
| Simple (dev) | local trials, fit-check, smoke | axquant quantize MODEL (--target-bpw defaults to 4.8) |
| Release | public quality/speed claims | staged analyze → plan → convert → validate → scoreboard |
Simple convert is always development evidence. It never upgrades to a certified claim.
# Local BF16 checkpoint — one command from source to development artifact
axquant quantize /models/Qwen3.6-27B-bf16 --target-bpw 4.8
# Explicit flags still work
axquant quantize \
--model /models/Qwen3.6-27B-bf16 \
--model-id Qwen/Qwen3.6-27B \
--revision REVISION_SHA \
--target-bpw 4.8 \
--runtime-smoke mlx-lm \
--json quantize-summary.json
# Hub id (download opt-in; pin a revision for reproducibility)
axquant quantize Qwen/Qwen3.6-27B --target-bpw 4.8 --allow-download --revision REVISION_SHAQwen3-ASR requires one pinned BF16 normalization step before inspection or planning:
python scripts/hf_to_mlx_bf16.py \
--hf-id Qwen/Qwen3-ASR-1.7B \
--revision REVISION_SHA \
--mlx-path /models/Qwen3-ASR-1.7B-MLX-BF16 \
--work /models/.axquant-source-work
axquant quantize /models/Qwen3-ASR-1.7B-MLX-BF16 \
--target-bpw 6.91 \
--runtime-smoke mlx-audio \
--audio-input ./sample.wavQwen3-VL converts from its pinned upstream BF16 checkpoint through MLX-VLM:
axquant quantize /models/Qwen3-VL-8B-Instruct \
--model-id Qwen/Qwen3-VL-8B-Instruct \
--revision REVISION_SHA \
--target-bpw 6.36 \
--runtime-smoke mlx-vlm \
--image-input ./sample.pngQwen3-VL 30B-A3B Instruct (MoE) — thin convert; product class 4bit and 6bit packs; primary runtime AX Engine, vision smoke via MLX-VLM (no MTP):
# BF16 source only (not community 3bit / FP8)
SRC=/models/Qwen3-VL-30B-A3B-Instruct
REV=REVISION_SHA
IMG=./sample.png
axquant quantize "$SRC" \
--model-id Qwen/Qwen3-VL-30B-A3B-Instruct --revision "$REV" \
--target-bpw 4.8 \
--output ./AX-Qwen3-VL-30B-A3B-Instruct-MLX-AXQ-4bit \
--runtime-smoke mlx-vlm --image-input "$IMG"
axquant quantize "$SRC" \
--model-id Qwen/Qwen3-VL-30B-A3B-Instruct --revision "$REV" \
--target-bpw 6.0 \
--output ./AX-Qwen3-VL-30B-A3B-Instruct-MLX-AXQ-6bit \
--runtime-smoke mlx-vlm --image-input "$IMG"
# Optional AX Engine readiness smoke after convert:
# axquant runtime-check --model ./AX-...-4bit --runtime ax-engineDefaults on the simple path:
- ladder
priorwith multi-group grid(32, 64); - output directory
./AX-<model>-MLX-AXQ-4bitwhen--outputis omitted; - development-evidence banner in logs and summary notes;
- family tier gates (inspect-only still fails closed).
axquant simple-convert-help # two-door best practices
axquant ladders --markdown-output convert-ladders.md
axquant probe-capacity --inventory architecture_report.json --output probe-capacity.json
axquant scoreboard --plan plan-01.json --output scoreboard.json --markdown-output scoreboard.mdTo reuse published planning evidence, pass a checksummed recipe bundle with --recipe — either a
local path or a revision-pinned Hub reference such as
--recipe hf://AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-4bit@COMMIT_SHA/recipe/axquant_recipe_bundle.json
(the revision pin is mandatory and the payload checksum is always verified). Expert memory-tier
development recipes live under examples/expert-memory-tier-v0.1.yaml (2-bit fused experts, 8-bit
routers; requires AX Engine experimental 2-bit flags). To add prior-based per-layer KV-cache
metadata, pass --kv-cache prior.
The staged path below uses the reviewed manual recipe. This proves the conversion workflow stage by stage, but its output remains unmeasured development evidence.
axquant inspect \
--model /models/Qwen3.6-27B-bf16 \
--model-id Qwen/Qwen3.6-27B \
--revision REVISION_SHA \
--output inventory.jsonInspection verifies the checkpoint layout, identifies the supported architecture, classifies each tensor, detects MTP, and records which components must remain protected.
axquant plan-manual \
--inventory inventory.json \
--recipe examples/qwen36-27b-manual-v0.1.yaml \
--output manual-plan.json \
--markdown-output manual-plan.mdThe included recipe applies 4-bit defaults, keeps attention weights at 6-bit, and preserves protected components at their required precision. The generated plan records every assignment and its reason.
axquant convert \
--model /models/Qwen3.6-27B-bf16 \
--revision REVISION_SHA \
--plan manual-plan.json \
--allow-unmeasured \
--ax-engine-manifest if-available \
--output AX-Qwen3.6-27B-MLX-AXQ-4bitIf the plan preserves MTP as an external bundle, conversion requires:
--mtp-sidecar /models/Qwen3.6-27B-bf16/mtp.safetensorsThe default --mtp-layout byte-preserved path never changes tensor payloads; the copied bundle's
mtplx_runtime.json declares mtp_norm_layout: raw_hf_delta so AX Engine converts every MTP
norm deterministically at load time instead of guessing from tensor statistics.
For the resident-loadable Qwen 3.5, Qwen 3.6, and Qwen 3.8 dense adapters, that runtime file also
declares the canonical qwen3-next-mtp identity required by strict sidecar importers such as oMLX.
Known historical qwen-dense / qwen-moe-packed labels are normalized without replacing any
existing exactness evidence. The Super-class qwen38-moe-v1 stream pack is deliberately excluded:
it cannot resident-load on a Mac, and an oMLX/MTPLX sidecar-import claim would be misleading.
The complete, family-specific Hub inventory is maintained in the
AXQ MTP runtime matrix. It distinguishes resident Qwen sidecars, Gemma
assistant bundles, DeepSeek nextn sidecars, and the Qwen expert-stream artifact; these contracts
are not interchangeable.
The explicit development path:
--mtp-layout ax-engine-qwen36-v1accepts only a checksum-bound raw Qwen 3.6 bundle with the exact 15-tensor BF16 contract. It adds one, with BF16 rounding, to the seven named MTP RMSNorm tensors, proves the eight projection payloads unchanged, and writes a new provenance manifest plus a depth-1 AX Engine runtime contract. This opt-in layout is not a release waiver: identical-checkpoint MTP exactness, acceptance, and throughput must still pass the ordinary validation gates.
The --allow-unmeasured and --ax-engine-manifest if-available options are development-only.
Omit them from a release workflow: release conversion requires measured evidence and a valid AX
Engine manifest. A measured plan must also pass
--calibration-manifest calibration_manifest.json; conversion verifies its checksum and
provenance against the plan and packages it with the artifact.
axquant runtime-check \
--model AX-Qwen3.6-27B-MLX-AXQ-4bit \
--model-id AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-4bit \
--revision candidate-revision \
--runtime ax-engine \
--output runtime-check.jsonUse --runtime mlx-lm to perform the MLX-LM generation smoke. Qwen3-ASR uses
--runtime mlx-audio --audio-input ./sample.wav; Qwen3-VL uses
--runtime mlx-vlm --image-input ./sample.png. --static-only is an MLX-LM development
diagnostic.
Run axquant COMMAND --help for the full options of any command.
| Command | Purpose | Current maturity |
|---|---|---|
feasibility |
Audit source and comparison checkpoints before conversion | Implemented |
source-checkpoint-manifest |
Derive and bind the immutable source revision, tokenizer, architecture, and file digests for exact-checkpoint certification | Implemented |
certification-policy |
Emit the frozen Qwen3-Next non-MTP certification policy and policy digest | Implemented |
prepare-coding-suite |
Build the checksum-bound 128-task Qwen3-Next coding suite, toolchain manifest, and calibration-overlap report (axquant-token-5gram-v2) |
Implemented; formal use requires all pinned toolchains; regenerate manifests for Seatbelt policy v3 |
evaluate-coding-suite |
Run resumable generation and deny-default executable scoring for coding-suite v2 | Implemented; Apple Silicon/Seatbelt execution evidence required |
verify-coding-suite |
Self-test every coding oracle and scorer by requiring the reference to pass and an empty mutant to fail | Implemented; run before suite freeze |
evaluate-general-quality |
Evaluate the disjoint direct-track general holdout and archive every raw model output | Implemented; BF16 and candidate runs must use matched settings |
direct-validation-index |
Recompute policy-bound BF16/candidate quality retention for both direct-track profiles | Implemented; emits a fail-closed index for N4 |
prepare-general-overlap |
Recompute exact/near-duplicate separation between general holdout and calibration (axquant-token-5gram-v2) |
Implemented; any match blocks direct validation |
inspect |
Inventory tensors, architecture, quantization, and MTP | Implemented |
calibrate |
Validate calibration input, record provenance, and build a tokenized cache (--manifest-only skips tokenization) |
Implemented |
validate-calibration-dataset |
Check a calibration JSONL against the toolkit's domain/size/format bar (defaults to the bundled reference dataset) | Implemented |
tokenize-calibration |
Build and verify a deterministic tokenized cache | Implemented |
capture-activations |
Capture per-module Linear input activations from a verified tokenized cache into a checksum-bound artifact | Implemented |
analyze |
Generate architecture priors or measure resumable affine/DWQ/AWQ/GPTQ/BF16 sensitivity from a calibration cache | Implemented |
analyze-kv |
Measure per-layer KV-cache sensitivity over a tokenized calibration cache | Implemented; development evidence |
plan |
Allocate 4/6/8/BF16 from a sensitivity report | Implemented; release use requires measured evidence |
optimize |
Plan weights and optional KV cache under one explicit memory budget and runtime reserve | Implemented; architecture-prior inputs remain estimates |
diagnose-joint |
Measure weight x KV interaction I(W,KV) and memory-budget crossover across context lengths | 1.9.0 development evidence only; never a certification claim; 1.8 convert unchanged |
plan-joint |
I-gated WeightPlan x KVPlan search that writes a convert-ready development plan | 1.9.0; small I keeps 1.8 independent optimize; material I selects a coupled cell |
plan-replay |
Replay a measured plan against its current sensitivity report with exact tensor/signature/metric checks | Implemented; fail-closed migration path |
plan-manual |
Apply an explicit YAML precision recipe | Implemented for development |
plan-experimental-mix |
Measured mixed 2/3/4-bit on the robust trunk; fused switch modules upgrade as one unit | Development only; not plan-joint; not mlx-optiq |
quantize |
Simple development convert: positional MODEL, optional --target-bpw / --output / --allow-download; ladder prior multi-group default |
Implemented; always development evidence (two-door) |
simple-convert-help |
Print simple-convert best practices (two-door model) | Implemented |
ladders |
List convert ladders (prior → measured-lite → measured-full → refine-awq-dwq) with cost/evidence |
Implemented |
probe-capacity |
Recommend sensitivity probe mode under host memory (bf16-full / measured-lite / streaming / prior-only) | Implemented |
scoreboard |
Certification scoreboard from plan + optional size/quality/MTP evidence (MTP speed owned by AX Engine) | Implemented |
bind-sensitivity |
Bind weight (+ optional KV) sensitivity digests into one lineage artifact | Implemented |
recovery-rank |
Rank quantized tensors for opt-in recovery by sensitivity (not implied by convert) | Implemented |
deferred-features |
List fail-closed deferred expansion features (vision-tower quant, per-expert unfused, domain LoRA) | Implemented |
recipe-export |
Export a revision-pinned plan as a checksummed recipe bundle | Implemented |
support-matrix |
List families with tier, investment posture, priority, and policy notes | Implemented |
support-policy |
Print family investment best practices (primary/secondary/thin) | Implemented |
head-to-head |
Render the public comparison page from a bound benchmark evidence index | Implemented |
convert |
Create the mixed-precision MLX checkpoint and metadata | Implemented for checkpoints at the convertible tier or above |
runtime-check |
Run AX Engine readiness or actual MLX-LM, MLX-Audio, or MLX-VLM generation | Implemented |
prepare-suite |
Materialize deterministic disjoint benchmark inputs | Implemented |
evaluate-quality |
Run MLX perplexity and scored generation tasks | Implemented |
compare-quality |
Compare matched quality runs with per-task visibility | Implemented |
benchmark |
Collect AX Engine runtime evidence | Implemented |
benchmark-ab |
Compare one checkpoint with MTP disabled/enabled | Implemented |
compose-gemma4-assistant-mtp |
Compose a Tier 2 candidate: byte-identical AXQ Gemma target + assistant/ drafter + ax_gemma4_assistant_mtp.json (does not mutate the Tier 1 pack) |
Implemented; product Hub packs ship fused under …-MLX-AXQ-*-MTP |
prepare-grafted-mtp |
Extract and bind a Qwen3.5/3.6 MoE MTP donor head for a Holo3-class trunk | Implemented; graft provenance explicitly records that the donor was not co-trained |
compose-grafted-mtp |
Attach a prepared grafted MTP sidecar without mutating the certified trunk tensors | Implemented |
mtp-align-prepare-data |
Build trunk-greedy self-distillation labels and optional cached features for MTP adaptation | Implemented; development workflow |
mtp-align-teacher-force |
Measure offline depth-1 MTP top-1 agreement against trunk-greedy labels | Implemented; development diagnostic |
mtp-align-adapt-fc |
Adapt the grafted MTP FC and normalization tensors while freezing the transformer | Implemented; development workflow |
mtp-align-adapt-full |
Continue adaptation with all packed MTP tensors unfrozen | Implemented; development workflow |
mtp-align-evaluate |
Score MTP probe or engine A/B evidence against the alignment ladder | Implemented; decision support |
benchmark-kernels |
Measure host-scoped decode/prefill kernel latency per (bits, group size) for plan --latency-table |
Implemented |
quantize-mtp-sidecar |
Emit an opt-in quantized MTP sidecar next to the untouched byte-preserved default, gated on a live or recorded AX Engine capability check | Implemented |
kv-serving-quality |
Bind executed per-layer KV precisions to dual-profile quality retention as a report-only artifact | Implemented |
mtp-diagnose |
Run the MTP kill-switch diagnostic matrix | Implemented; diagnostic evidence only |
benchmark-index |
Bind every required baseline or record why it is unavailable | Implemented |
validation-index |
Require disjoint passing agent-coding and general evidence | Implemented |
refine |
Generate proxy-ranked bounded precision swaps | Development only |
recover |
Record optional post-PTQ recovery provenance | Implemented as identity-copy provenance; no weight mutation |
refine-measure |
Build checksum-bound complete-candidate evidence | Implemented |
refine-select |
Select only from checksum-bound, validated complete candidates | Implemented |
refine-export |
Export standalone executable plans from a refinement result | Implemented |
refine-run |
Resume complete conversion, quality, MTP, validation, and selection runs | Implemented |
pareto |
Report non-dominated validated candidates on named hardware | Implemented |
hardware-registry |
Certify checksum-bound kernel, version, power, and shape coverage | Implemented |
campaign-overlap |
Build privacy-preserving exact/5-gram overlap evidence (axquant-token-5gram-v2; repeatable --id-field, default id then task_id) |
Implemented |
campaign-frontier |
Verify every cheapest-failure-first candidate gate and derive the eligible frontier | Implemented |
campaign-freeze |
Freeze one exact qwen36-mtp-v2 source/candidate/evidence graph |
Implemented |
campaign-preflight |
Verify frozen bindings, durable storage, and exact formal host identity (df-macbookpro-m5 = MacBook Pro M5, 128 GB, 18-core) |
Implemented |
campaign-start-formal |
Start one budgeted formal cycle only after matching preflight | Implemented |
campaign-complete-formal |
Derive pass/fail from the bound completion and consume both formal holdouts | Implemented |
campaign-close-no-go |
Close a pre-formal campaign without consuming its blind holdout | Implemented |
campaign-record-publication |
Bind downloaded Hub bytes, revision, audit, claim, lifecycle, and runtime re-verification | Implemented |
artifact-lifecycle |
Append legal development → candidate → frozen → certified/superseded/revoked transitions | Implemented |
claim-render |
Generate measured-BPW public claims and the certified model card from bound evidence | Implemented |
release-audit |
Dispatch historical Qwen 3.6 v4, Qwen3-Next N0–N8, or additive qwen36-mtp-v2 M0–M8 proof |
Implemented |
compatibility-matrix |
Bind family-wide artifact, runtime, and validation evidence | Implemented |
validate |
Apply release thresholds to external benchmark evidence | Implemented |
size-evidence |
Bind authoritative candidate/uniform-4 or uniform-6 artifact sizes | Implemented |
release-exception |
Record an approved, expiring, evidence-bound size exception | Implemented |
report |
Render plan and validation reports | Implemented |
publish-prepare |
Assemble a release only after validation | Implemented |
publish |
Preview or execute a guarded Hugging Face upload | Implemented |
verify-reproduction |
Verify regenerated weight bytes and bound provenance | Implemented |
verify-cert |
Offline-check a public certificate and optional artifact bundle with a machine-readable verdict | Implemented; exits nonzero on any inconsistent binding |
name |
Generate the recommended AXQuant model name | Implemented |
DWQ release evidence uses the same deterministic 0.1/99.9-percentile clipping implementation during sensitivity probing and conversion. A targeted run adds measured DWQ candidates to an existing complete affine report without rewriting any base candidate:
axquant analyze \
--model Qwen/Qwen3.6-27B \
--revision pinned-source-revision \
--calibration calibration-cache \
--base-sensitivity measured-affine-sensitivity.json \
--methods dwq \
--target-tensor model.language_model.layers.4.mlp.up_proj.weight \
--state dwq-probe-progress.json \
--output measured-affine-dwq-sensitivity.jsonThe merged report records the base report's semantic digest, inventory digest, probe backend,
target count, and method set. Release audit requests list every ancestor under
sensitivity_lineage; M3 replays the chain and rejects removed or modified base candidates,
undeclared additions, protocol drift, cycles, missing parents, and unused reports.
Once a measured sensitivity report is available, create a plan without the development override:
axquant plan \
--analysis measured-analysis.json \
--target-bpw 4.8 \
--bits 4,6,8,16 \
--mtp protected \
--output quantization-plans--lm-head-floor 8bit is the governed size-gate path: it lowers the LM-head weight
floor from BF16 to 8-bit for that plan only, records the deviation in
constraints.lm_head_min_bits, and requires a measured 8-bit LM-head sensitivity candidate
before the release audit accepts the plan. The default floor stays BF16.
Validate externally collected benchmark bundles:
axquant size-evidence \
--artifact-manifest candidate/axquant_manifest.json \
--model-id AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-4bit \
--revision candidate-revision \
--output candidate-size-evidence.json
axquant validate \
--reference-evaluation reference-evaluation.json \
--candidate-direct-evaluation candidate-mtp-off.json \
--candidate-evaluation candidate-mtp-on.json \
--mtp-ab candidate-mtp-ab.json \
--size-reference uniform4-size-evidence.json \
--candidate-size candidate-size-evidence.json \
--profile agent-coding \
--output validation.jsonFor an MTP speed claim, --mtp-ab binds the matched AX Engine direct/MTP comparison used for
token-weighted decode speedup, prompt-median speedup, and greedy-output exactness. AXQuant rejects
the bundle when its model identity, workload, software, hardware, controls, or environment do not
match the candidate evidence.
For a 6bit certification, freeze the class explicitly and derive the size reference from the
matching complete uniform-6 baseline. The same max_weight_size_ratio threshold is then applied
to the uniform-6 denominator; a 4-bit candidate cannot switch denominators opportunistically:
axquant size-evidence \
--feasibility-report feasibility.json \
--reference-kind uniform-6bit \
--output uniform6-size-evidence.json
axquant validate \
--reference-evaluation reference-evaluation.json \
--candidate-direct-evaluation candidate-mtp-off.json \
--candidate-evaluation candidate-mtp-on.json \
--mtp-ab candidate-mtp-ab.json \
--size-reference uniform6-size-evidence.json \
--candidate-size candidate-size-evidence.json \
--target-class 6bit \
--profile agent-coding \
--output validation.jsonIf a measured Pareto candidate misses both the BPW target and the uniform-4 size-ratio gate, a release authority can record a time-bounded exception. The command computes the observed values from the two size artifacts; it does not accept caller-authored observed values:
axquant release-exception \
--exception-id AXQ-SIZE-001 \
--plan selected-plan.json \
--candidate-size candidate-size-evidence.json \
--size-reference uniform4-size-evidence.json \
--tradeoff-evidence measured-tradeoff.json \
--measured-tradeoff "Measured quality, speed, and memory tradeoff approved for release." \
--owner "AutomatosX release owner" \
--approved-by "Named release authority" \
--approval-reference "release-decision-001" \
--approved-at 2026-07-30T12:00:00Z \
--expires-at 2027-01-31T00:00:00Z \
--output release-exception.json
axquant validate \
--reference-evaluation reference-evaluation.json \
--candidate-direct-evaluation candidate-mtp-off.json \
--candidate-evaluation candidate-mtp-on.json \
--size-reference uniform4-size-evidence.json \
--candidate-size candidate-size-evidence.json \
--plan selected-plan.json \
--release-exception release-exception.json \
--exception-evidence tradeoff=measured-tradeoff.json \
--profile agent-coding \
--output validation.jsonThe exception can downgrade only artifact.weight_size_ratio; it must also disclose the failed
measured-BPW target. Quality, speed, memory, fallback, integrity, and provenance failures remain
errors. Release audit requests that use an exception must list its file under
release_exceptions and provide the exact plan, candidate_size, size_reference, and
tradeoff paths under release_exception_evidence. M4 reloads and hashes every file, checks both
validation profiles, verifies approval and expiry, and compares the packaged
release_exception.json with the approved record.
For a refinement candidate, derive its selection record from the converted manifest, matched quality comparison, and release validation rather than authoring measurement values:
axquant refine-measure \
--refinement refinement.json \
--candidate-id cand-0000-000 \
--measurement-id cand-0000-000-m3-max \
--artifact-manifest candidate/axquant_manifest.json \
--quality-comparison candidate/quality-comparison.json \
--validation candidate/validation.json \
--output measurements.jsonUse --existing measurements.json with a new output path to accumulate another candidate or a
second named-host result for the same candidate. Measurement IDs must be unique. refine-select
uses the worst measured objective and BPW across every host record for a candidate, so adding
hardware evidence cannot make selection less conservative. The complete objective combines task
retention and perplexity with MTP acceptance, peak memory, and effective speed. Refinement
parentage is a precision-only monotonic chain: a child may upgrade formats but cannot downgrade
or exchange an unrelated tensor.
Prepare the exact complete-candidate run without executing expensive model work:
axquant refine-run \
--request examples/refinement-execution-request.yaml \
--output-dir run/complete-candidatesReview execution-manifest.json, then add --execute. The runner resumes checksum-verified
completed outputs, skips the remainder of a candidate after an execution failure, treats
validation exit 1 as measured failed-gate evidence, merges complete measurements, and runs
refine-select plus pareto.
Every release benchmark must name its power mode and quantizer/version. refine-run reads
benchmark_power_mode from its request, derives the AXQuant identity from each plan, and includes
both raw A/B logs in the resumable output contract. Standalone baseline runs use
--power-mode, --quantizer, and --quantizer-version. benchmark-ab derives adjacent-token
repetition directly from emitted token IDs, records depth-one proposal accuracy, and derives
greedy divergence from the matched A/B outputs. Its release speed gate defaults to
token-weighted-decode-tps: total output tokens divided by total generation wall time, with the
same calculation applied to both arms. The artifact also records the legacy prompt-median TPS
ratio and requires it to remain at or above 1.10x, preventing a long decode from hiding a
typical-prompt regression. Release MTP evidence therefore requires token-weighted decode speedup
>=1.20x, prompt-median speedup >=1.10x, and exact greedy outputs. Use
--speedup-metric prompt-median-tps only when reproducing the
legacy protocol. For a uniform-6 reference A/B, use
--direct-baseline-kind uniform-6bit --mtp-baseline-kind uniform-6bit; the default kinds remain
the AXQuant MTP-off/on release pair. Use --record-failed-speedup for an evidence sweep that must
retain both evaluation bundles when only the speed floor fails: the command writes the complete
evidence, returns status 1, and leaves exactness and matched-control invariants fail-closed.
Build the M7 hardware registry only from the resulting raw
logs, evaluation bundles, validation, plan, converted artifact manifest, sensitivity report,
quality comparison, and quantizer execution manifest:
axquant hardware-registry \
--request examples/hardware-registry-request.yaml \
--output hardware-profile-registry.jsonThe command returns 1 while validation is failing, any runtime or conversion fallback is
present, provenance is inconsistent, the complete objective cannot be rebuilt from the artifact,
quality, and validation files, or the claimed bit/group/role/shape coverage is not measured. The
registry records both the semantic and file digest of its complete-candidate measurement set.
Publication verifies that file, packages it as refinement_measurements.json, packages every
objective input, and rewrites the registry to packaged relative paths. Each registry entry
identifies the exact measurement ID, allowing one candidate and plan to be certified on multiple
named hosts.
The certified Qwen 3.6 path starts from the exact source
Qwen/Qwen3.6-27B@6a9e13bd6fc8f0983b9b99948120bc37f49c13e9. It is separate from the
historical v4 development audit:
axquant campaign-freeze \
--request flagship-campaign-request.json \
--output flagship-campaign.json
# Authorizing preflight must run on the formal host:
# MacBook Pro M5, 128 GB, 18-core (host id df-macbookpro-m5).
axquant campaign-preflight \
--campaign flagship-campaign.json \
--output flagship-campaign-preflight.json
axquant release-audit \
--request flagship-release-audit-request.json \
--output flagship-authorization-audit.jsonAn authorization-ready audit proves the frozen campaign and current M0–M8 evidence but is
deliberately not publication-ready until the independent lifecycle and claim closure is present.
The campaign request and every transition, raw-evidence, review, no-go, and publication record
must remain inside the declared non-symlinked durable root. Formal preflight also requires fresh
doctor, Metal, zero-fallback, storage, power, and thermal results bound to the exact frozen
host contract (MacBook Pro M5, 128 GB, 18-core; host id df-macbookpro-m5).
After the legal frozen → certified event, claim-render creates public-claim.json and the
measured-BPW README.md. An independent final publication review binds those exact files and the
authorization audit under the durable campaign root; the final flagship request must pass M0–M8
again. Preview and executed publication both rerun that exact final request. A v4 audit cannot
authorize a package containing flagship claims or lifecycle metadata.
Prepare the release directory locally, then run the aggregate proof before publishing a certified checkpoint:
axquant publish-prepare \
--model AX-Qwen3.6-27B-MLX-AXQ-4bit \
--repo AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-4bit \
--validation-index release-validation-index.json \
--hardware-registry hardware-profile-registry.json \
--pareto-report pareto-report.jsonaxquant release-audit \
--request examples/release-audit-request.yaml \
--output release-audit.jsonThis revalidates indexed evaluation, complete-refinement, and hardware file checksums; binds the
selected interaction improvement to the packaged measurement set; reruns reproduction
verification; inspects the wheel metadata, contents, and every RECORD member hash/size; and
requires the packaged plan, validation/benchmark evidence, hardware registry/evidence,
refinement measurements, Pareto report, and recipe to match the external evidence graph. The
M0 check recomputes checkpoint completeness, parameter/architecture equivalence, revisions, MTP,
and baseline runtime results rather than trusting the feasibility status label. M1 requires every
artifact Safetensors file to have one safe, size- and checksum-valid manifest record. M2 reloads
the indexed evaluations and rechecks complete trials, matched controls and hardware, provenance,
fallbacks, identical-checkpoint MTP pairing, one cross-profile candidate/reference pair, and
disjoint datasets. M3 reloads the checksum-bound calibration manifest, verifies separation and
provenance, requires finite tensor-scoped measurements, and verifies every targeted-sensitivity
ancestor. M6 reloads the bound artifact, quality comparison, and validation for every
measurement, recomputes the versioned complete objective, and requires a measured, validated,
monotonic parent/child gain. Complete-measurement construction also rejects non-authoritative
profile thresholds, an inconsistent validation pass label, core release metrics below their
active thresholds, nonzero kernel fallbacks, and a passing size overage without its governed
plan-bound exception. M7 rebuilds every Pareto point and frontier member from the bound
measurement set. The
compatibility matrix must bind that same candidate manifest, runtime checks, and validation. The
audit also reloads every checkpoint from the original compatibility request and re-hashes its
manifest, plan, runtime checks, and validation. The wheel must declare Python 3.11+, MIT, and all
runtime dependencies; the artifact, plan, recipe, and wheel must identify the same AXQuant
version. It reports M0 through M8 separately and returns 0 only when all nine milestones pass;
an alpha or pre-1.0 wheel, including one still carrying an Alpha distribution classifier, cannot
pass M8. An executed publication packages that exact authorizing result as release_audit.json
and refuses to overwrite a different existing audit.
Preview publication first. Add --yes only when the release should be uploaded; an executed
upload also requires the matching audit and its original request so the full M0–M8 proof can be
rerun from current evidence:
axquant publish \
--model AX-Qwen3.6-27B-MLX-AXQ-4bit \
--repo AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-4bit \
--validation-index release-validation-index.json \
--hardware-registry hardware-profile-registry.json \
--pareto-report pareto-report.json \
--release-audit release-audit.json \
--release-audit-request examples/release-audit-request.yamlBefore publication, build the complete comparison index. BF16, uniform 4-bit, uniform 6-bit, and the identical AXQuant MTP-off/on pair are mandatory. Mixed-precision, AWQ, and DWQ entries may be unavailable, but they cannot be omitted and must state why:
axquant benchmark-index \
--request examples/benchmark-evidence-request.yaml \
--output benchmark-evidence-index.jsonBuild one benchmark index and validation report for each required profile, using distinct evaluation datasets. Then bind them into the publication gate:
axquant validation-index \
--request examples/release-validation-request.yaml \
--output release-validation-index.jsonPublication rejects a missing profile, a reused dataset, differing candidate/reference identities, a failed validation, or a non-ready benchmark index.
Every prepared release includes reproduction_recipe.yaml with argument-array commands for
downloading the pinned source, converting it, checking both runtimes, and verifying every
regenerated Safetensors file. Prepared MTP layouts additionally checksum-bind the provenance and
runtime companion files required to reuse the transformed sidecar without applying the transform
again. After running those commands, verification can also be invoked directly:
axquant verify-reproduction \
--recipe reproduction_recipe.yaml \
--artifact regenerated-model \
--output reproduction-verification.jsonBuild the M5 family matrix from checksum-bound artifact, AX Engine, MLX-LM, and validation evidence:
axquant compatibility-matrix \
--request examples/qwen36-compatibility-request.yaml \
--output compatibility-matrix.jsonThe request declares the complete official dense catalog as verified at a timezone-qualified
timestamp. The command returns 1 and still writes the matrix when any declared official dense
Qwen 3.6 model is absent, uses inconsistent candidate evidence, or lacks a compatible
agent-coding or general validation profile. The checked-in example lists 27B as the only dense
size in the linked catalog; refresh catalog_verified_at and required_dense_models before every
release. FP8 is a representation of a parameter size, not a second model size.
- Architecture priors are never described as measured sensitivity.
--allow-unmeasuredis restricted to development conversion.- Conversion fails if the plan does not cover every module it claims to quantize.
- External MTP sidecars remain byte-for-byte unchanged unless the explicit, provenance-checked Qwen 3.6 AX Engine layout backend is selected.
- Output is staged and atomically renamed only after conversion succeeds.
- Release claims require complete-model quality and hardware evidence.
- Credentials and Hugging Face tokens are never written to logs or manifests.
Recommended model names use:
OWNER/AX-BASE-MODEL-MLX-AXQ-TARGET
For example:
AutomatosX/AX-Qwen3.6-27B-MLX-AXQ-4bit
The target suffix describes the checkpoint class, not a claim that every tensor uses that bit width. The manifest contains the actual precision distribution and effective bits per weight.
A Mac with MLX installed will not catch Ubuntu CI failures. CI splits surfaces on
purpose: Ubuntu = non-MLX (.[dev] only; MLX cannot run on Linux runners) and
macOS = MLX (.[dev,mlx]). Prefer the local CI mirror:
./scripts/ci-local.shThat runs ruff, format, mypy, then a non-MLX venv with a sanitized PATH (matching
GitHub Actions non-MLX jobs), and the host MLX suite when available. See
docs/guides/ci-root-causes.md and CONTRIBUTING.md.
.venv/bin/pytest
.venv/bin/ruff check .
.venv/bin/ruff format --check .
.venv/bin/mypy srcTests use small synthetic Safetensors fixtures and do not require real model weights.
Contributions are warmly welcome. AXQuant is built to help Mac users get more reliable, efficient local inference and a better experience on Apple Silicon. Fork the repository and send us a pull request for bug fixes, documentation, tests, usability improvements, runtime compatibility, architecture adapters, or reproducible quantization research.
See CONTRIBUTING.md for the fork-and-pull-request workflow, development setup, validation commands, and evidence requirements. For a substantial design change or new model family, open a GitHub issue first so the scope and required promotion evidence are clear.
Index: docs/README.md.
| Doc | Audience |
|---|---|
| AX Engine 72-hour endurance | Users — AX Engine 6.15.0 passed a 72 h soak on Qwen 3.6 27B AXQ 6-bit |
| Qwen3.8-27B AXQ VL retention assessment | Users and evaluators — BF16 vision preservation, current evidence limits, and required VL validation |
| Qwen3.8 AXQ 2-bit report | Users — Super-class 2-bit convert evidence; this revision will not be certified (too slow) |
| Known issues | Operators — documented limitations and fail-closed gates |
| Environment compatibility | Operators — platforms, Python, MLX extras |
| Flagship certification | Certification operators — qwen36-mtp-v2 sequence |
| Certified checkpoints | Users and auditors — exact public verdicts, scopes, and hashes |
| AXQ model fleet v2 | Hub pack maintainers — stable names and editions |
| Migration v1.1 / v1.2 | Upgraders from earlier toolkit releases |
| CI root causes and prevention | Contributors — Ubuntu non-MLX vs macOS MLX, PyPI gate |
| AXQ pack interchange v1 | Operators — affine U32 pack contract |
| Release notes convention | Maintainers — curated GitHub Release body per version |
| GitHub Releases | Everyone — published version history |
| Third-party notices | Legal — research and dependency attribution |
Product requirements, the architecture decision register, technical specifications, and the independent-implementation policy are maintained internally and are not published in this repository.
AXQuant is released under the MIT License. Dependencies, model checkpoints, calibration datasets, and external tools retain their own licenses.