docs: correct the estimation chain reference - #1377
Open
davidberenstein1957 wants to merge 4 commits into
Open
Conversation
The documented CPU fallback ladder did not match resource_tracker.py, the equivalence constants were sourced to a file absent from the repo, Electricity Maps was undocumented, and the RAM heuristic was presented as empirically motivated. Rewrite methodology.md against the source and split the bibliography and equivalences into their own pages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tracking_mode silently swaps the power model on the estimation path: cubic with a 10% floor in machine mode (hardware.py:287-288), linear with no floor in process mode (hardware.py:346). Documenting only the cubic invited readers to blame the divergence on attribution scope. Also scope accuracy.md's deviation figures to machine mode, which is what the profiling script actually measured. Refs #1378 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The overview page states the formula PUE-inflated; this page stated E x C and introduced PUE later, so the two read as disagreeing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The GPU row named only NVML, but AMD devices are read through AMDSMI in codecarbon/core/gpu_amd.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrites
docs/explanation/methodology.mdagainst the source, and splits the bibliography and the dashboard equivalences into their own pages.The page is the one a skeptical evaluator reads before deciding whether to trust CodeCarbon's numbers, and several of its central claims did not match the code. Every constant now carries the
file.py:LINEit comes from so a reviewer can spot-check.Factual corrections
1. The CPU fallback ladder was wrong (
resource_tracker.py:249-279)force_cpu_powercomes first and skips every platform backend (:255-260, guard at:272); thenforce_mode_cpu_load(:263-270); then the platform backend (:221-247); then_setup_fallback_tracking(:159-219), where cpu_load is preferred over constant whenever psutil is present.2. The 50%-of-TDP figure was attributed to the wrong mode
CONSUMPTION_PERCENTAGE_CONSTANT = 0.5(hardware.py:26) applies only in the no-psutil"constant"mode (hardware.py:363).cpu_loadmode does something else entirely — see the next item.2b.
cpu_loadhas TWO power models, selected bytracking_mode(hardware.py:274-352)Before: undocumented in either form. The first revision of this PR documented only the machine-mode cubic, which was arguably worse than documenting neither: it invited a reader comparing a machine-mode run against a process-mode run to attribute the whole divergence to attribution scope and conclude the tool is inconsistent.
After: both documented side by side, keyed to
tracking_mode, in a new The two cpu_load models section:tracking_mode="machine"tracking_mode="process"psutil.cpu_percent()(:280-282):294-330):345)0.1 + 0.9 × (load/100)³(:287-288)tdp × load_norm / 100(:346)With a worked divergence: at 50% utilisation machine mode gives 21% of TDP, process mode gives 50% — more than double, from the same underlying load. Flagged in an admonition so a user seeing that gap knows it is two models, not a bug.
The exponent, the 0.1 floor, and the absence of a floor in process mode are all stated as uncited — same treatment as the RAM multipliers; none has a comment, citation or fitting procedure anywhere in the module.
2c.
accuracy.mddeviation figures were unscoped, and its curve description was wrongcompare_cpu_load_and_RAPL.py:289-292constructs its tracker withforce_mode_cpu_load=Trueand notracking_modeargument, so every figure was gathered in the default machine mode. Stated in an admonition, with an explicit note that the numbers do not transfer to process mode, which is uncharacterised. The "interpolates linearly" sentence now names both curves correctly.3. Apple Silicon was documented backwards
powermetrics… if you do not want to give sudo rights, CodeCarbon will fall back to constant mode", with sudoers instructions._setup_cpu_load_fastis tried beforepowermetrics(resource_tracker.py:228-232), so with psutil installed — the normal case — cpu_load wins andpowermetricsis effectively unreachable. Stated explicitly.4.
DEFAULT_POWER_PER_CORE = 4was undocumentedthreads × 4 W(cpu.py:29, used:1024) — the actual common case. No model or no psutil →POWER_CONSTANT = 85(hardware.py:23, applied:457).5. Equivalences cited a file that does not exist in this repo
webapp/src/helpers/constants.ts— there is nowebapp/directory here.explanation/equivalences.mddocumenting what actually ships incodecarbon/viz/data.py: 0.409 kg CO₂e/mile (:65), 0.097 kg CO₂/hour (:76), 160.58 kg CO₂/week (:96), with the EPA derivations from their docstrings, and a note that the TV figure has no source in the code at all.6. Carbon intensity resolution order was scattered and incomplete
impact.csv) → Electricity Maps (emissions.py:162-177) → regional US/CAN/Nordic (:182-195,:234-290) → country (:292-326) → 475 gCO₂eq/kWh world average. Plus an explicit warning that a cloud-region miss falls back silently to country then world average (emissions.py:76-97,:303-315) with nothing in the output recording which level answered.7. The RAM model was presented as empirically motivated
Before: "this approach significantly improves the accuracy… a more reasonable estimate."
After: stated as a two-stage heuristic, with both stages tabulated, and: "Neither the 5 W figure nor any of the 0.9/0.8/0.7 multipliers has a source in the code." The only citation in the module is a pre-v3 Crucial FAQ for the replaced rule (
ram.py:20-22).force_ram_poweris pointed at prominently.Two worked examples were also wrong: 128 GB and 1 TB were given as ~40 W; the model yields 38 W (
5×4 + 5×0.9×4).8. PUE applies to the per-component columns
energy *= self._pueruns inside the per-hardware measurement loop (emissions_tracker.py:1194), socpu_energy,gpu_energyandram_energyare already inflated. Do not multiply again.9. GitHub issue #457 was cited as if it were a source
10. Bibliography moved to
explanation/references.mdEvery entry verified against the actual paper before pasting. Corrections made against the draft: Henderson et al. is JMLR 21(248):1–43, 2020 (confirmed at jmlr.org/papers/v21/20-312.html); Khan et al. is TOMPECS 3(2), Article 9, 1–26, 2018 (doi 10.1145/3177754); Lacoste et al. and Patterson et al. are listed as arXiv preprints, since neither arXiv record carries the workshop/venue the draft asserted. Author lists, years and arXiv IDs for Strubell, Lottick and Luccioni all check out as drafted. Ends with a link to
CITATION.cff.Also
accuracy.md,alternatives.md,faq.md,troubleshooting.md); repointed.mkdocs.ymlnav entries for the two new pages.cpu_fallback.pngdiagram reference was dropped: it illustrates the old, incorrect ladder. Redrawing it is tracked in Redraw docs/images/cpu_fallback.png — it shows a CPU fallback ladder the code does not implement #1378, which specifies what the corrected diagram must show.Verification
uv run task docspasses — build clean, link check clean (4898 attributes).Not verified
RAM_SLOT_POWER_X86 = 5, the 1.5 W ARM figure and the 0.9/0.8/0.7 multipliers have no citation anywhere in the codebase; documented as asserted.cpu_loadcurve's exponent, its 10% floor, and process mode's lack of a floor are likewise uncited in the code.cpu_loadaccuracy has never been profiled; only machine mode has.239 external URLs not checked), though every new citation URL was opened during drafting.🤖 Generated with Claude Code