Repository navigation
fix: stop unprofitable server prompt lookup rounds - #2274
Merged
Merged
Conversation
Server prompt lookup can keep accepting isolated tokens while repeated verify rounds cost more than they emit. Add an opt-in emission governor, stronger shadow confirmation, and earlier server re-priming while preserving CLI defaults and per-row scheduler state. Cover partial-match pauses, actual pipeline recovery, resumed copy verification, concurrent row isolation, and fully accepted narrow blocks. Formatting and static checks pass; CUDA compilation, runtime tests, and performance acceptance remain pending coordinator execution. Refs #2264
Record each running server's executable and arguments, reject wrong arms and unexpected verification counters, and verify the MLX pin through the canonical helper against the binary source. Preserve original workloads in a separate evidence directory with bounded execution and a fail-closed acceptance analyzer. Correct stale configuration comments. Sixteen synthetic analyzer cases and shell/Python syntax checks pass. These changes affect comments and measurement scripts only; the release binary source remains 89581b8dd6cbffd315833ee284d7bb9aacc9e307. CUDA runtime and performance validation are pending. Refs #2264
Initialize the new server-only configuration fields from the unchanged defaults in the existing scripted-miss test fixture so core tests compile without changing the CLI-policy scenarios.
The dispatch resolver now applies CUDA server break-even settings, but its no-drafter regression still expected the unchanged CLI configuration. Assert both configurations explicitly and verify that the dispatch accessor returns the server settings. The coordinator observed 17 passing dispatch tests and this stale expectation failure; governor, scheduler, and CLI tests passed. This test-only correction passes formatting and diff checks; its runtime rerun remains coordinator-owned. Refs #2264
The unchanged 4096-target copy workload tokenizes to 4099–4102 prompt tokens. A same-build diagnostic found zero proposals in every flag-on request because the old 4096-total-token cap excluded them before decoding. Raise the paged cap to 4608, the smallest 512-token boundary covering 4102 prompt plus 256 generated tokens, and keep 8192-token contexts excluded. Add scheduler boundary coverage and document the separate diagnostic without claiming final performance acceptance. Tests and the fresh integrated release build remain pending while another issue's acceptance run is active. Refs #2264
Keep the original off/on diagnostic records, running-process metadata, model and binary provenance, and server logs byte-for-byte, with a SHA-256 manifest. The source-89581b8d binary used the old 4096 cap and proposed no tokens for any actual 4099–4102-token copy prompt. This is pre-acceptance admission evidence only. Final performance acceptance requires a new integrated build with the 4608 cap. Refs #2264
The server break-even governor treated two fully accepted tokens the same as two accepted from a wider proposal. A short proposal therefore counted as below-floor evidence and could pause before the copy run widened, cutting useful verification from copy-heavy requests. Pass the actual proposal length into the governor and pause only when the latest below-floor round left proposed tokens unaccepted. Add a regression for a partial narrow round followed by a fully accepted narrow round, while retaining the pause for the same acceptance count from a wide proposal. Formatting passes. Runtime tests and acceptance measurements are delegated to the final integrated verification because the CUDA native build is intentionally owned by the orchestrator. Refs #2264
Archive the complete first integrated measurement without removing outliers or weakening acceptance thresholds. It passed chat and concurrency but failed short-copy and two paged long-copy gain floors. Add checksums and a separate directory with unchanged drivers for the corrected governor's final verification. Refs #2264
The acceptance trace showed that one 7/0 round after a long profitable copy run could pull the emitted-token EMA below four and pause lookup permanently. The previous sample-count guard counted all historical rounds, so it did not require repeated low-yield evidence. Track consecutive incomplete rounds below the emission floor and pause only after two. Full or floor-meeting rounds reset the streak, and shadow resume clears it. Regression coverage reproduces the traced 2/2, 7/7, 0/7 boundary and confirms that a second consecutive low round still pauses. Formatting passes. CUDA runtime tests and the instrumented trajectory replay are delegated to the orchestrator's warm build. Refs #2264
The short-proposal correction did not recover the measured copy workload, so preserve the second attempt as incomplete failed evidence. Archive reproducible before/after governor traces proving that isolated misses caused the pause and that consecutive-low-yield evidence restores the copy path. Prepare an unchanged full matrix in a separate final-attempt directory. The corrected traced build passed 49 governor/drafter and 20 scheduler tests. Trace timings are diagnostic only; final production acceptance remains pending. Refs #2264
Preserve the third complete measurement and its remaining chat regression instead of marking the issue complete. Copy, concurrency, and long-context gates pass, but chat remains at -1.419% against the -1% floor. Record the retry-limit checkpoint and the next trajectory diagnostic needed before further policy changes. Fix only evidence path discovery for nested attempts; the acceptance formulas and thresholds are unchanged. All 17 synthetic analyzer checks pass, including nested evidence. Runtime source is unchanged from the fully tested release. Refs #2264
The blanket low-yield grace preserved copy runs but added unprofitable chat verifications. Limit grace to one complete miss after an accepted-token EMA at least equal to the emission floor; partial low-yield matches pause once their emitted EMA is below the floor. The unchanged diagnostic workloads preserve copy at 239 proposed, 214 accepted, and 35 rounds while reducing chat by 11 verify rounds. All 54 governor/drafter and 20 scheduler tests pass. Preserve raw traces and prepare the fourth unchanged performance attempt; final performance acceptance remains pending. Refs #2264
Archive the fourth quiet GB10 matrix and its source, binary, arm metadata, checksums, and validation summaries. Every unchanged criterion passes: chat -0.275%, short-copy gains 148.602% to 159.283%, overlapping concurrency-two ranges, and all long-copy retention floors. Document the selective boundary-miss heuristic and its measured scope while preserving the three prior failed attempts. Release build, full CUDA workspace tests, clippy, formatting, and fresh-main contract checks pass. Refs #2264
inureyes
added
status:review
State that grace covers only the first low-yield round and that pausing still requires the emitted-token EMA below the floor. This avoids carrying the earlier blanket two-round description into the final selective policy. Link the acceptance checkpoint to PR #2274. Validation: documentation-only diff; measured production source and dependencies remain unchanged. Refs #2264
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Server prompt lookup now pauses low-yield verify rounds using a four-token emission floor while preserving profitable copy spans. A complete miss receives one boundary probe only after established accepted-token history; partial low yield and consecutive misses pause, and shadow lookup remains active for recovery. The server also re-primes after the first proposal-free ask and admits the measured paged context envelope through 4608 tokens. CLI defaults, graded adaptation, and nonadaptive behavior remain unchanged.
Test plan
Evidence
Final benchmark report.
Raw attempts, diagnostics, provenance, and checksum manifests.
Technical reports: English, Korean.
Closes #2264