Correct the provenance of two replication anchors (CBO 2005; DYNASIM4)

[MaxGhenis]

Two replication anchors carried the wrong model label. This PR corrects the labels in the scripts and the paper and adds an erratum that governs the committed artifacts, which keep their original bytes.

1. Mermin (2005) Table 1, 75-year deficit/surplus row: CBO 2005, not DYNASIM3. Mermin's text (411260-benefit-reductions.txt:212-214) says the last row of Table 1 "presents Congressional Budget Office (CBO) estimates". The table's source line (:521) credits both DYNASIM3 (Runid: 432) and CBO (2005): the benefit rows are DYNASIM3, and the deficit row is CBO. This row is the T2 anchor of the cost-ordering synthesis, so T2 is a comparison against CBO's 2005 estimates. It was registered but not blind, because the five figures it uses were already in committed artifacts before the registration.
2. Five Democratic Approaches (Urban 103050) Tables 3 and 15: DYNASIM4 ID980, not DYNASIM3 ID980. Table 3 reads "Source: DYNASIM4 ID980." (103050-five-dem.txt:1052), Table 15 reads "Source: DYNASIM ID980." (:3299), and the report says its analysis "is based on DYNASIM4" (:906).

What changed

• scripts/replication_cost_ordering.py: MERMIN_ANCHOR_CITE now attributes the row to CBO (2005) as reported in Mermin (2005). It also quotes the row label exactly ("percentage of", where the old string had "percent of"). CAREGIVER_ANCHOR_CITE now reads DYNASIM4 ID980. The docstring, the anchor comments and the T2-swap named delta were updated to match. The named delta now says the flip is against CBO's order, and that DYNASIM3's 2050 benefit row orders the pair the same way (it takes 81.8 and 85.2 from MERMIN_PCT_SCHEDULED_2050).
• scripts/replication_caregiver.py: the docstring, the paper field and the Table 15 citation now read DYNASIM4 ID980. The paper field also gets the report's actual subtitle ("Estimated Impact of Plans from the 2020 Presidential Campaign", :11-13) and its 2019 Trustees assumptions (:330-331).
• scripts/replication_mermin_rows.py: the Table 1 citation and the comment on ANCHOR_TABLE1_PAYROLL_PCT now name CBO (2005) for the payroll row. The citation now gives the page (PDF p.15), which the old string left blank.
• paper/paper.qmd: the cost-ordering paragraph said "DYNASIM ranks them the other way". It now says, in two sentences, that the anchor column is CBO's 2005 scoring, reported in the same table as DYNASIM3's benefits, and that DYNASIM3's own 2050 row for ages 62 to 67 orders the pair the same way (81.8 and 85.2 percent of scheduled, the table's existing values).
• docs/errata/2026-09-29-anchor-provenance.md (new) covers:
  • what was mislabelled, with the exact source quotes and their file:line locations, plus SHA-256 hashes of the source files;
  • every committed artifact that keeps an old label, by JSON pointer: runs/replication_cost_ordering_v1.json, runs/replication_caregiver_v1.json, runs/replication_mermin_rows_v1.json, and the benchmark registry, history and wall row dynasim.mermin.four_reform_cost_ordering;
  • the identifiers kept as aliases rather than renamed;
  • the results that stay the same.
• tests/test_anchor_provenance_erratum.py (new, unit tier, 10 tests):
  • pins the corrected strings in the three scripts and the paper exactly, and fails on the old ones (checked: 7 of the 10 fail against origin/master's scripts and paper; the other 3 test the erratum itself);
  • scans scripts/, src/, paper/ and docs/ outside the erratum for the old labels;
  • checks the erratum's quote blocks and hashes against ~/PolicyEngine/dynasim-refs when that directory is present, and skips the check otherwise.
• tests/tier_counts.json and the tests/README-tiers.md table: unit rises from 4,979 to 4,989 and the total from 10,353 to 10,363.

What did not change

• No numeric value, anchor value, result, forecast grade or pass/fail status changed. T1 is still 100% (met), T2 tau 0.667 (forecast 1.0, not met) and T3 tau 0.913 (forecast at least 0.8, met). The committed T2 cross-check against DYNASIM3's own benefit row is also 0.667.
• runs/*.json, gates.yaml, engine/loop.py and engine/steps.py were not touched.
• benchmarks/registry.json and build_registry.py were not touched. The registry's append protocol needs a spec-revision note, and build_wall.py requires the latest history set to use the current registry SHA, so a relabel of dynasim.mermin.four_reform_cost_ordering has to ride on the next evaluation append. Until then the erratum governs.
• No row id or output key was renamed: dynasim.mermin.four_reform_cost_ordering, T2_mermin_kendall_tau, mermin_cite, mermin_payroll_pct and seventy_five_year_payroll_pct keep their names, and the erratum records them as aliases.
• docs/references.bib was not touched. Its mermin2005benefitreductions note ("DYNASIM3, Runid 432") correctly names the report's own run, and smith2020fivedem already read DYNASIM4.

Verification

All commands used .venv/bin/python -m pytest -p no:xdist with OMP_NUM_THREADS=1, on macOS with the PSID data and ~/PolicyEngine/dynasim-refs staged:

• tests/test_anchor_provenance_erratum.py: 10 passed, including the exact-quote and hash check against the staged sources. Against origin/master's scripts and paper, with the erratum present, 7 of the 10 fail, as they should.
• tests/test_replication_cost_ordering.py, test_replication_caregiver.py, test_replication_mermin_rows.py and test_benchmarks.py, together with the new module and test_paper_figures.py: 83 passed, 2 skipped. The skips are the policyengine-us revision pins in the caregiver and Mermin-rows reproductions; the local checkout is at e4363903f3 and the artifacts pin bf71be3b. The PSID reproduction pin test_ppi_delta_reproduces_committed ran and passed.
• The downstream importers of the edited scripts (test_m2_pseudo_projection.py, the two test_w2_seam_caregiver*.py modules, the two test_gate_w1_candidate1*.py modules, test_gate_w1_candidate2.py and the three test_gate_w1_forensics*_reproduction.py modules): 52 passed, 17 skipped. The skips are environmental: no PE-US directory variable, no certified frame export or pickle, and no MX23REL.
• Tier policy over the full collection (pytest -p no:xdist tests -k test__given_collected_suite__then_tiers_match_policy_manifest): passed. The per-tier recounts with pytest --collect-only -q -p no:xdist -m <tier> were unit 4,989, artifact 3,331, integration_psid 1,341, reproduction_legacy 520 and oracle_policyengine 182, for 10,363 in total.
• black . -l 79 --check: 876 files unchanged. ruff check .: all checks passed.
• git grep -n "Runid 432\|DYNASIM3 ID980" outside runs/ finds only four kinds of hit:
  • uses that name the report's own DYNASIM3 run for its benefit rows, which are correct;
  • docs/references.bib, which is correct;
  • the benchmark registry's document strings (benchmarks/registry.json and its builder benchmarks/build_registry.py), which the erratum governs;
  • the erratum and its test, which quote the old labels on purpose.

Fixes #488

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
social-security-model	Ready	Preview	Sep 29, 2026 4:27pm UTC

[MaxGhenis]

Independent review (fresh Opus 5.5 agent, blind to the build): APPROVE at 484eed6. It checked every corrected label and erratum quote against the source texts (411260-benefit-reductions and 103050-five-dem, SHA-256 matched). It confirmed that no runs/*.json, gates.yaml, engine file, numeric value or status changed, and it proved the new pins fail on the old strings in a scratch copy. Tests, black and ruff pass. Merging on green CI at this head.

🤖 Generated with Claude Code