build: add a docs constant drift check - #1383
Open
davidberenstein1957 wants to merge 1 commit into
Open
Conversation
The methodology rewrite found that the docs had been describing a CPU fallback ladder the code does not implement, along with several stale or undocumented constants. The docs were correct when written and drifted silently afterwards. This makes the numeric half of that drift impossible to ship again. Nine constants are imported from the codebase (or loaded from the data file) and asserted to appear verbatim in the page documenting them. Nothing re-parses Python source with regexes, so the check cannot drift from the code it guards. Scope is deliberately narrow: no prose checking, no link checking, and no assertion about the order of any fallback ladder. A check that tries to verify narrative structure false-fails, and one that false-fails gets disabled. The rationale is recorded in the module docstring so the next contributor does not extend it the wrong way. `docs-check-drift` pointed at scripts/check-docs-drift.py, which never existed; it now points at the real file and runs in the docs CI job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## docs/methodology-rewrite #1383 +/- ##
===========================================================
Coverage ? 91.77%
===========================================================
Files ? 49
Lines ? 5057
Branches ? 0
===========================================================
Hits ? 4641
Misses ? 416
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacks on #1377.
Adds a check that fails when a numeric constant changes in the code while the docs keep the old value.
Why
The methodology rewrite in #1377 found that
methodology.mdhad been documenting a CPU fallback ladder the code does not implement, plus several constants that were stale, wrong, or undocumented. Those docs were correct when written. They drifted silently, and nothing caught it.This closes the mechanical half of that gap. It cannot catch prose going stale, but it can catch a number going stale, and a number going stale is what turned
85 Winto a value users were reading off a page while the code did something else.What it does
scripts/check_docs_drift.pyguards nine constants:POWER_CONSTANTcodecarbon/external/hardware.py:23CONSUMPTION_PERCENTAGE_CONSTANTcodecarbon/external/hardware.py:26DEFAULT_POWER_PER_COREcodecarbon/core/cpu.py:29RAM_SLOT_POWER_X86+ the x86 floorcodecarbon/external/ram.py:14world_averagedata/private_infra/carbon_intensity_per_source.jsoncodecarbon/viz/data.py:54-96Every value is imported from the codebase, or loaded from the actual data file, then asserted to appear verbatim in the page documenting it. Nothing re-parses Python source with regexes — a checker that reads the code textually can drift from the code it is meant to guard.
Scope, deliberately narrow
Not in scope, and the module docstring says so to stop the next contributor extending it wrongly:
scripts/check_docs_links.pyalready does it.resource_tracker.py.Verification
Passes on this branch:
And it has been seen to fail. Changing the documented value to 999 in a scratch copy:
Exit 1, and the message names the constant, both values, and the file to edit, so a failure is fixable without reading the script.
tests/test_docs_drift.pycovers this with temp fixtures rather than touching the real docs — 3 tests, passing.Note for review
pyproject.tomlhad adocs-check-drifttask pointing atscripts/check-docs-drift.py, a file that never existed. This repoints it at the real script (underscored, matchingcheck_docs_links.py) and wires it into the docs CI job.PR #1369, which targets master, deletes that dead task. Whichever lands second will conflict on that line; the resolution is to keep the corrected task from this PR.
🤖 Generated with Claude Code