quant: add Bonsai TQ2G128 (qt=40) and BQ1G128 (qt=41) - #597
Open
nwoolmer wants to merge 1 commit into
Open
Conversation
nwoolmer
force-pushed
the
feat/ternary-bonsai-27b
branch
3 times, most recently
from
August 18, 2026 09:36
718d418 to
e55628b
Compare
Kaden-Schutt
added a commit
that referenced
this pull request
Aug 18, 2026
…t ledger A ceiling cannot see substitution. `arch_key_dispatch <= 12` stays green if you delete a justified site and add an unjustified one, and that is the exact shape of f00fa80, where landing qt=43 deleted six match arms belonging to other formats while the build stayed green. check-layering.py already knew this: it emits UNLISTED (in the tree, not in the table) and STALE (in the table, not in the tree) rather than a number. This applies that shape to two more surfaces. docs/quant-formats/qt-register.txt — 36 rows, one per QuantType variant, each carrying a load disposition. Asserted by check-quant-registry.py: qt_unlisted qt_stale qt_unparseable qt_disposition_mismatch all == 0 The first draft of this gate was three counts including `qt_unregistered <= 10`. That ceiling was uninterpretable: ten variants were a mix of deliberate non-passthrough (22 is an index tensor, 28/29 load via paro.rs, 31-37 via arch loaders) and genuine omissions like cf061b7, where qt=40's encoder, GEMV and is_mq arms all landed and the model would not load. A declared disposition per row makes those two cases mechanically distinguishable. The register also adjudicates allocation. PR #597 (BQ1G128) and PR #599 (MQ1G1024GL) both claimed qt=41; ruled for #597, since MQ1G1024GL was never wired or tested and BQ1G128 is substantially the same format. 40 and 41 are held as `reserved`, the one disposition exempt from STALE so a number can be claimed before its variant exists -- a register that can only mirror the present cannot stop two branches claiming the same number, which is the case it exists for. docs/governance/debt-dispatch-bypass.txt — 222 arch-crate call sites reach Gpu::gemv_*/gemm_*/fused_* directly instead of resolving through hipfire_dispatch::KernelRegistry, which has existed since e822b31 (2026-05-30) and which one and a half arch crates use. This is a DEBT table, not a purity check: every row is permitted, growth fails, and paying down is a one-line edit. bypass_unlisted bypass_stale bypass_regressions bypass_slack == 0 bypass_total <= 222 bypass_slack is the one that makes it converge rather than stall. Migrating a crate without lowering its row leaves a stale allowance that the debt can quietly regrow into; asserting slack at 0 forces the ledger down whenever the tree improves. Negative-controlled, not assumed. Substitution -- swap one register row out and a fake one in, row count unchanged at 36 -- fails with qt_unlisted=1 qt_stale=1. Substitution across crates -- cohere2moe +1, qwen35 -1, bypass_total unchanged at 222 -- fails with bypass_regressions=1 bypass_slack=1, where a total-only ceiling passes. Disposition flip, register/enum name drift, a deleted from_u8 arm, and a reserved row demoted to passthrough each fail on their own metric. Baseline and restore green every time. Ratchet goes 12 -> 21 asserted metrics, 0 violations. Workspace 0 errors.
Kaden-Schutt
added a commit
that referenced
this pull request
Aug 18, 2026
Every gate this programme built was a script a human had to remember to run. Nothing in CI invoked leanup-ratchets.sh, check-layering.py, check-arch-dispatch.py or check-crate-maps.py; grepping for them outside scripts/ returns only comments in Cargo.toml. no-gpu-ci.sh does run tools.change_gate.tests.* -- it tests the gate selector without ever using it. Phase 3A's mandate is that every gate either fails on regression or is deleted. A gate nothing runs fails on nothing, which is the same defect as the four self-certifying checks 3A was created to find, arrived at from the other side. The job is GPU-free and asserts, not advises: these thresholds are green on the branch introducing them, so a red is a real regression rather than lint debt. scripts/gates.sh is the separate model/GPU harness and still needs hardware; it is deliberately not wired here. ratchet-diff runs PR-only and needs fetch-depth: 0, because it reads the base revision's thresholds and a shallow clone has no merge base to resolve. Also refreshes crates/hipfire-quantize/map.md, which ae18341 made stale by two lines -- caught by running the job locally before wiring it, which is the whole argument for wiring it. Consequence worth stating plainly: this will red #597 on landing. It adds qt 40/41 to RAW_CODECS without from_u8 arms or register rows, so qt_unlisted goes to 2. That is the gate doing its job on a real defect, not collateral damage, but it lands on someone else's ready PR and they should hear it from a person rather than from a build failure.
Kaden-Schutt
force-pushed
the
feat/ternary-bonsai-27b
branch
from
August 18, 2026 23:32
02af60e to
e1573b9
Compare
Kaden-Schutt
changed the title
Kaden-Schutt
force-pushed
the
feat/ternary-bonsai-27b
branch
from
August 18, 2026 23:36
e1573b9 to
038ebc6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebuilt onto current
master(80a572c8) as one Saddle/Ratchet-aligned commit.This PR exclusively owns the Bonsai low-bit wire formats:
PR #599 is stacked on this branch and does not remove, redefine, or modify qt40/qt41.
What lands
max|w|, and refuse out-of-set ternary code 3.check_ternary_pack_healthand K%128 geometry guards.bonsai:27b-tq2,bonsai:27b-bq1, plus thebonsaialias.Quality evidence
Matched f32-KV KLD against the pinned qwen3.6-27B MQ4 teacher:
86a5f80fd29d545abb1093dead242725ced6d68b8607c6d566d897b1a82442dcb2ea3f7f2160fe961d5ed256f474a4428b0f88e3a923e70fa80d3bf3bde5098e83dc56409efa0503e171d91fffcad25c8a9f000120f61cf9cbba589a09fbbb07The result is the fixed uniform codebook, not the raw bit budget: the project’s free-codebook MQ2-Lloyd control remains materially better than uniform MQ2.
Architecture validation
gfx1201
bonsai:27b-tq2: five coherent serve-harness battery turns, empty=0, attractor=0; avg prefill 204.7 tok/s, avg decode 32.3 tok/s.bonsai:27b-bq1: five coherent turns, empty=0, attractor=0; avg prefill 295.8 tok/s, avg decode 41.2 tok/s.3cec9bea52ff28927377f78b37906a62.gfx1010
Final-source
scripts/compile-kernels.sh gfx1010compiled the relevant scalar family:dequant_{tq2g128,bq1g128}_to_f16gemv_{tq2g128,bq1g128}gemm_{tq2g128,bq1g128}_prefillgemm_qkvza_tq2g128The gfx1010 compiler correctly rejects the wave32-WMMA variants. A direct TQ dequant run passed on the available gfx1010 host; a full 27B BQ serve load was blocked by that host’s 8.6 GB VRAM rather than a kernel or dispatch failure.
Gates
No merge is performed by this update.