Repository navigation
test(rocm): verify the unified engine on gfx1151 after the engine epic - #2250
Merged
Merged
Conversation
…ve engine epic Phase 7 of epic #2166, run on the Radeon 8060S (gfx1151) host after Phases 0 to 6 merged on a CUDA machine. Every ROCm-only test and every test that skips without a kernel port ran here for the first time against the merged engine, and the results page records where each ROCm behavior now lives on the engine path, the ROCm parity measurement, and the engine's throughput against the pre-epic CxxGenerator on the same host and day. - docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md: the gate, the named tests (all ran, none skipped, including the 20 GiB u32 pool-offset test under --ignored), the post-epic location of each ROCm behavior, parity (every CLI-vs-server pair identical on Qwen3-0.6B, Llama-3.1-8B and granite-4.0-h-tiny; dense vs paged identical for greedy over 64 tokens and 0 decided-position mismatches over 128 teacher-forced decode steps with the HIP paged v2 kernel in the loop; the seeded stream diverges at token 6 and 18 with the paged kernel), and 11 interleaved decode cells against 4c44e31 with a null arm: ten within ADR 0007's 1.0 percent threshold, gemma-3-4b-it-4bit at 2048 tokens 1.2 percent below, isolated to the B=1 lookahead pipeline on the rotating cache (filed as #2239). - Found on both builds and filed separately: affine MoE prefill lost the expert-batched kernel when #2137 cleared right_sorted on the sorted path (granite 573 vs 919 tok/s, Mixtral 26 vs 126), cross-backend (#2240). - ADR 0007 and docs/CONTINUOUS_BATCHING.md record the ROCm parity definition applied and the measured result; docs/installation.md points at the page. - The drivers and records are under docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/ (per-run guard logs, one lock hold for the whole session, a teacher-forced dense-vs-paged trace through the server's /completion n_probs, and a rounds summarizer). - src/models/llama3.rs: the fused RoPE-append comment named update_and_fetch as the consumer; since #2171 it is cache.attend on either storage. Verification: the named tests and the full gate on gfx1151 are listed in the page and the PR. Metal and CUDA are not available on this host; nothing on those paths changes (one comment in llama3.rs). Refs #2192, #2166.
inureyes
added
type:test
2 of 3 tasks
inureyes
added a commit
that referenced
this pull request
Oct 10, 2026
## Summary End-of-run measurement for #2229 (PR #2256) and #2255 (PR #2262) on GB10, and the two values #2255 left provisional, now set from it. Report: `docs/benchmark_results/server-prompt-lookup-gb10-2026-10-11.md`; drivers, raw output and analysis script under `docs/benchmark_results/data/server-prompt-lookup-gb10-2026-10-11/`. - `PROMPT_LOOKUP_PAGED_CONTEXT_LIMIT`: `None` to `Some(4096)`. A paged sequence stops running verify rounds past 4096 tokens of context and decodes plainly. - `DEFAULT_PROMPT_LOOKUP_MAX_BATCH`: stays 2; the doc comment and the `--prompt-lookup-max-batch` help now cite the measurement instead of calling it provisional. - `docs/CONTINUOUS_BATCHING.md` and the CHANGELOG entry state the measured gains and the cost on low-copy chat, and point to #2264. ## Results **#2229, penalty requests on the CLI path** (`--repetition-penalty 1.1`, 5 rounds with a null arm): against main before #2229 the engine client now decodes 23 % (Qwen3-1.7B, 256 tokens), 16 % (Qwen3, 8192), 31 % (Llama-3.2-1B, 256) and 23 % (Llama, 8192) faster, every round above every round. Against the pre-epic `CxxGenerator` (`4c44e317`) it is equal on Qwen3 at 256 tokens (inside the null range) and 1.5 to 2.7 % faster in the other three cells. **#2255, server prompt lookup** (Qwen3-1.7B, 256 tokens): | Criterion | Result | |---|---| | (a) plain chat, B=1, flag on vs off, no worse than -1.0 % | **Not met.** -3.49 % (-3.73..-3.22) over 8 chat-template questions x 5 rounds, 25 to 45 % of proposed tokens accepted. Tracked in #2264. | | (b) copy-heavy, B=1 | +53 to +67 %, 4.2 tokens per verify | | (c) concurrency 4 and 8, default gate | on matches off in every workload; a gate of 8 loses 13 % on the chat-template copy workload at concurrency 4, so the default stays 2 | | (d) paged, long prompts | on / off ratio for copy and open question: 2.69x / 1.27x at 1024, 2.79x / 1.64x at 2048, 1.42x / 1.12x at 4096, 1.15x / 0.62x at 8192; dense at 8192: 2.10x / 0.99x | The flag is opt-in, so decode without it is unchanged. ## Test plan - [x] clippy `--profile test-fast --features cuda --lib --tests --bins -D warnings` - [x] `--lib server::batch` (526), `server::speculative_dispatch` (17), `--test cli_help_consistency` (27), `--test dead_doc_pointers`, Apache header check, all under `gpu-lock` with `--test-threads=1` - [ ] `scripts/ci/check_binary_assets.py` fails on main's two undeclared tarballs from #2250 (`docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/`); this PR adds no binary file Refs #2229, #2255, #2264
This was referenced Oct 10, 2026
inureyes
added a commit
that referenced
this pull request
Oct 11, 2026
…#2266) ## Summary The `binary assets` CI job has failed on `main` since PR #2250: it committed `docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz` (15.1 KB) and `rounds/raw-logs.tar.gz` (13.8 KB) without declaring them in `scripts/ci/check_binary_assets.py`, and every PR merged since carried the same failure. Both archives contain only plain-text run logs (212 files), so this unpacks them in place instead of adding a binary exception: - `guard/guard-logs.tar.gz` becomes `guard/logs/` (the guard's per-run logs, `guard/` and `rounds/guard/` inside it as archived). - `rounds/raw-logs.tar.gz` becomes `rounds/raw/`, the path `rocm-unified-engine-gfx1151-2026-10-08.md` already cites for the raw `mlxcel-bench-decode` output. - The report's reference to `guard/guard-logs.tar.gz` now points at `guard/logs/`. Text diffs and delta-compresses in history, and the guard's 256 KB binary budget stays for assets that need to be binary. No log content changes. ## Test plan - [x] `python3 scripts/ci/check_binary_assets.py`: `binary-assets: OK, 9 declared binary files, 63.3 KB of 256 KB budget` - [x] `file` reports every unpacked file as text; none is matched by `.gitignore`
3 tasks done
inureyes
added a commit
that referenced
this pull request
Oct 11, 2026
…rballs (#2267) ## Summary Follow-up to #2266. That PR cleared the `binary assets` failure from #2250 by unpacking the ROCm verification run logs into 212 text files. The logs are only opened to audit a run, so this keeps them archived instead: - Restores `docs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/guard/guard-logs.tar.gz` and `rounds/raw-logs.tar.gz`, byte-identical to #2250's, and removes the unpacked `guard/logs/` and `rounds/raw/`. - Restores the report's reference to `guard/guard-logs.tar.gz`. - Adds a `DECLARED` rule to `scripts/ci/check_binary_assets.py` for `docs/benchmark_results/data/*.tar.gz` (raw per-run logs behind a committed benchmark report), 32 KB per file. ## Test plan - [x] `python3 scripts/ci/check_binary_assets.py`: OK, 11 declared binary files, 92.2 KB of the 256 KB budget - [x] `scripts/ci/check_binary_assets_test.sh`: all cases pass - [x] `scripts/insert_apache_header.py --check`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ROCm verification of the unified engine (Phase 7 of epic #2166) on the gfx1151 host: the gate, every named ROCm test, where each ROCm behavior now lives on the engine path, the ROCm parity measurement and definition, and the engine's throughput against the pre-epic
CxxGeneratoron the same host, same day, interleaved with a null arm. Results page:docs/benchmark_results/rocm-unified-engine-gfx1151-2026-10-08.md, with the drivers and raw records underdocs/benchmark_results/data/rocm-unified-engine-gfx1151-2026-10-08/.paged_pool_past_u32_elements_matches_gatherunder--ignoredin a guarded idle window. The two CUDA-failingssm_update_parity_testscases pass on gfx1151.docs/CONTINUOUS_BATCHING.md.mlxcel run, chat) shows no gap on ROCm.right_sortedon the sorted path (granite 573 vs 919 tok/s, Mixtral 26 vs 126; cross-backend by the same flag on Metal and CUDA): filed as fix(moe): sorted gather_qmm prefill lost right_sorted since #2137, MoE prefill 1.7x to 4.8x slower #2241.src/models/llama3.rs: one stale comment corrected (the fused RoPE-append output is consumed bycache.attendsince refactor: one KV storage abstraction with dense and paged backends #2171).Verification
On gfx1151 (ROCm 10.0.0, HIP 7.15), branch rebased onto
mainat1b4e3657:make verify-rocmwithMLXCEL_ROCM_SMOKE_MODEL=models/mlx/Qwen3-0.6B-4bit: 12194 passed, 0 failed, 403 ignored across 163 test binaries,[verify-rocm] OK(prerequisites: versions, dtype keys, kernel-port dispatch, llama-compat, overlay records, Python tooling, fmt, clippy with-D warnings, the GPU smoke). The named tests, the parity sessions and the benchmark rounds are logged in the results page and its data directory (every GPU run underscripts/rocm_gpu_guard.sh, all benchmark runs CLEAN).Not verified: Metal and CUDA (not available on this host). Nothing on those paths changes; the only code change is a comment in
src/models/llama3.rs.Closes #2192