Skip to content

tech-debt(ci): the file-size ratchet says 'raising a ceiling must fail' but only blocks raises above the frozen baseline, so a shrink can be spent back #14498

Description

@mrveiss

The comment and the test disagree about what the ratchet forbids

repo_tests/python_file_size_ratchet_test.py:43-44 states the rule plainly:

Lowering a ceiling in the hook is fine and needs no edit here. Raising one, or adding a fourth file, must fail — so DO NOT "sync" this dict to make a test pass.

But test_no_ceiling_may_be_raised only fails when a ceiling exceeds the frozen RATCHET_BASELINE:

if rel in RATCHET_BASELINE and ceiling > RATCHET_BASELINE[rel]

So a ceiling can be raised freely as long as it stays under the original baseline. The two statements are not the same rule, and the gap opens exactly when the ratchet has been working.

The concrete case that exposed it

tool_handler.py was at its 4063 ceiling. PR #14497 (#14495) extracts a cohesive unit and brings it to 3863, lowering KNOWN_LARGE to match — as test_a_shrunk_file_must_lower_its_ceiling requires, so the removed lines are not re-licensed.

RATCHET_BASELINE stays 4063, deliberately — the header says not to sync it.

The consequence: the file may now grow from 3863 back to 4063 — 200 lines of regrowth — with every test passing, because each step is "raising a ceiling but staying under the baseline". The 218-line gain that PR just made is immediately available to be spent, and nothing reports that it is being spent.

PR #14492 (#14469) is the first case in the queue: it adds ~59 net lines to this file, taking it to roughly 3922. Under the test as written, bumping KNOWN_LARGE 3863 → 3922 passes. Under the comment as written, it must fail.

Why this is not just pedantry

test_a_shrunk_file_must_lower_its_ceiling exists precisely so that a shrink is locked in rather than re-licensed. Permitting a later raise back toward the baseline undoes that guarantee one PR at a time, and each individual step looks compliant. A ratchet that can be wound backwards inside its own tolerance is a slower version of no ratchet.

It is also the same shape as several defects found this week: the guard's stated subject is wider than what it actually enforces, so it reads as coverage.

Options — this needs a decision, not a default

  1. Make the test match the comment. Compare each ceiling against its own previous committed value (git show HEAD:scripts/check_python_file_size.py) rather than against the frozen baseline, so any raise fails. Strictest, and it means a PR adding to one of these three files must offset its additions — which is arguably the point of the exemption list.
  2. Re-baseline on every shrink. Lower RATCHET_BASELINE alongside KNOWN_LARGE when a file shrinks, keeping "raise above baseline fails" as the mechanism while locking in each gain. Contradicts the current header instruction, so the header changes with it.
  3. Make the comment match the test. Accept that growth within the original baseline is intended, and reword :43-44 so nobody reads it as a stricter promise than it makes. Cheapest and honest, but gives up the locked-in gain.

I recommend 2: it preserves the fixed-reference-point property the header argues for, while making each shrink permanent. 1 is stricter but would block a security fix like #14492 behind an unrelated refactor.

Whichever is chosen, #14492's ceiling bump should be settled explicitly rather than by whichever test happens to pass.

Related

Activity

  1. mrveiss commented on Aug 18, 2026

    @mrveiss
    OwnerAuthor

    Implemented option 2 (re-baseline on every shrink) in #14527.

    RATCHET_BASELINE is now lowered to today's sizes — tool_handler.py 4063 → 3719, locking in the 344 lines recovered by #14497 and #14492 — and is pinned to the files themselves by a new test rather than derived from KNOWN_LARGE, so the second-reference-point property survives: a raise in either file alone now fails against the other.

    Mutation evidence in the PR body: the regrowth scenario (file 3719 → 3800 with the ceiling following) passes on the merge base and fails on the branch; the new test fails against the tree as it stands today (baseline 4063 > actual 3719) and passes after. The header instruction that said not to sync the baseline changed with the rule.

    #14492's ceiling bump is settled by landing: it merged at 3719 and the baseline now records that, so there is no headroom left over from it.

  2. mrveiss commented on Aug 18, 2026

    @mrveiss
    OwnerAuthor

    Closed by PR #14527, squash-merged as 83f28b13ab28 and verified in origin/Dev_new_gui.

    The invariant is now algebraic rather than conventional. Whenever both suites pass:

    • test_no_ceiling_may_be_raised forces ceiling ≤ baseline
    • audit_ceilings' exact-match forces ceiling == actual
    • the new test_the_baseline_is_relowered_by_every_shrink forces baseline ≤ actual

    Together: ceiling == baseline == actual, always. Measured in base, all three agree on all three files:

    orchestrator.py            actual=1114 ceiling=1114 baseline=1114
    chat_workflow/manager.py   actual=4068 ceiling=4068 baseline=4068
    chat_workflow/tool_handler.py  actual=3719 ceiling=3719 baseline=3719
    

    The 344 spendable lines are gone. tool_handler.py shrank 4081 → 3863 (#14497) → 3719 (#14492) today while RATCHET_BASELINE stayed at 4063, so every one of those lines could have been regrown with the whole suite green — each step being "a raise that stays under the baseline". test_a_shrunk_file_must_lower_its_ceiling exists precisely so a shrink is not re-licensed; the frozen baseline was undoing that one PR at a time.

    The second-copy property survived, which is where this fix could most easily have gone wrong. RATCHET_BASELINE is a deliberate second copy so the list cannot drift by being compared only against itself. The new test does not take the hook fixture and counts lines itself — it is pinned to the tree, not derived from the thing it checks. Syncing it from KNOWN_LARGE would have made the two agree by construction and left the protection in name only.

    Verified by mutation in both directions, each guarded by assert mutated != original:

    • raise KNOWN_LARGE alone → fails test_recorded_ceilings_match_the_files_today and test_no_ceiling_may_be_raised
    • raise RATCHET_BASELINE alone → fails the new test
    • spend the shrink (regrow 3719 → 3800, ceiling follows, baseline frozen) → passes on the merge base — the defect — and fails on the branch

    That last pair is what makes it a fix rather than an assertion; a change that failed on the branch but had also failed before would have proved nothing.

    The three existing direction tests were re-verified rather than assumed orthogonal — two assert on message text this PR rewrote, and their substrings survived.

    Two things fixed beyond the ask:

    • The hook's own guidance messages named only KNOWN_LARGE, so a maintainer following them exactly would edit one file and land a red test. They now name both edits via RATCHET_REL.
    • The header comments described the old behaviour — the "DO NOT sync this dict" line was half the original defect. They now describe what the tests enforce, with no contradicting residual instruction.

    Not a finding, recorded for the next reader: a coordinated same-commit edit to all three of {source file, ceiling, baseline} still passes. That is a real growth with a visible three-file diff, not a quiet spend — exactly the distinction this issue set out to draw.

  3. 9 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions