Relabel the Mermin cost-ordering benchmark row as CBO 2005 at the next evaluation append #497

The erratum docs/errata/2026-09-29-anchor-provenance.md (#488, #489) corrects the provenance of the Mermin (2005) Table 1 deficit row: it is Congressional Budget Office (2005) estimates as Mermin reports them, not DYNASIM3 output. The benchmark registry still labels that row DYNASIM3. The relabel cannot land on its own, so it must ride on the next evaluation append. This issue says exactly what to change when that append happens.

Why it waits for an append

• benchmarks/README.md (registry append protocol): changing an existing specification requires a new spec_revisions note, and the registry, evaluation artifact, run-manifest entry, history append and regenerated wall are committed together.
• benchmarks/build_wall.py (render()) requires every latest history record to carry the current registry SHA-256 ("latest wall records do not use the current registry"). Any change to registry.json bytes therefore fails build_wall.py --check until a new record set is appended under the new registry.
• No evaluation append is due now. benchmarks/history.jsonl holds two record sets, the seed (719604ca…, 42 rows) and tranche 2 (ead482c5…, 101 rows, registry 7231b306…), both committed on 1–2 August 2026, and no open PR or issue schedules a third.

Scope

• One row changes: dynasim.mermin.four_reform_cost_ordering (legacy seed row; T2 anchor of the cost-ordering synthesis).
• The other 19 dynasim.mermin.* rows stay as they are. They take benefit cells from Mermin's Tables 2 and 4. Those are the report's benefit projections, which it says "are based on the Urban Institute’s DYNASIM3 model" (411260-benefit-reductions.txt line 127), and neither table has a deficit row. Their external_reference "DYNASIM3" and their locator document "… report 411260, DYNASIM3 Runid 432" stay. The source lines of Tables 2 and 4, like Table 1's, also credit "the Congressional Budget Office (2005)" (lines 574 and 691). The report does not say what CBO contributed to those two tables, and this issue does not change them.
• No Five Approaches rows exist. The registry and its builder have no row sourced from Smith, Johnson and Favreault (2020), Urban report 103050: grep -rn "103050\|ID980\|Five Democratic\|caregiver" benchmarks/ finds nothing. There is nothing to relabel for the DYNASIM4 ID980 correction.
• The row id stays the same. dynasim.mermin.four_reform_cost_ordering is kept as an alias (the erratum's "Identifiers kept, not renamed"), and so are the history lines that point to it.

Field changes for dynasim.mermin.four_reform_cost_ordering

These fields do not change: gap_class (unverified_source), gap_note, tier (model_triangulation), verification_class, every value and pointer, the locator's page, table, row and column paths, and its derivation. Because gap_note is unchanged, the history records' gap_note needs no change either.

Revision note, as it would sit in SPEC_REVISIONS in benchmarks/build_registry.py:

"dynasim.mermin.four_reform_cost_ordering": [
    {
        "changed_fields": [
            "/concept_mismatch/frame",
            "/concept_mismatch/population",
            "/external_reference",
            "/source_pin/exact_locators/0/document",
        ],
        "note": (
            "Relabeled the published order as CBO's 2005 estimates as "
            "reported in Mermin (2005) Table 1, not DYNASIM3 output, per "
            "the 2026-09-29 anchor-provenance erratum."
        ),
        "revision": 1,
    }
],

Three couplings the change must handle

1. The Cross-model validation matrix: SSA, CBO, DYNASIM, WISH statutory #352 matrix must still rebuild byte-exact. The row is in LEGACY_ROW_IDS. schema.reconstruct_legacy_matrix() rebuilds the merged Cross-model validation matrix: SSA, CBO, DYNASIM, WISH statutory #352 matrix from the registry, and tests/test_benchmarks.py requires the result to hash to MERGED_MATRIX_SHA256. Add a compatibility override for this row in reconstruct_legacy_matrix() that restores the four old strings. This follows the existing cbo.tax_revenue.share_of_taxable_payroll precedent in the same function.
2. Two DYNASIM counts key on the label. build_registry.py asserts len(dynasim_rows) == 32 and 10 derivations, and test__benchmark_registry__retains_exact_dynasim_locators asserts 32 entries. Both select rows with external_model / external_reference .startswith("DYNASIM"). After the relabel that selection finds 31, and the builder aborts. Select by the immutable row-id family instead (row_id.startswith("dynasim.")). That keeps 32 rows and 10 derivations, and DERIVED_DYNASIM_ROW_IDS stays unchanged. Then assert that the relabelled row is the only one whose label no longer starts with DYNASIM.
3. The append moves the pinned SHA constants. The append updates CURRENT_REGISTRY_SHA256, CURRENT_HISTORY_SHA256, CURRENT_RUN_MANIFEST_SHA256 and WALL_SHA256 in tests/test_benchmarks.py, and regenerates wall.md, as any append does.

diff --git a/benchmarks/build_registry.py b/benchmarks/build_registry.py
--- a/benchmarks/build_registry.py
+++ b/benchmarks/build_registry.py
@@ -928,19 +928,30 @@ def build_cbo_rows() -> list[dict[str, Any]]:
     ]
 
 
+MERMIN_DOCUMENT = (
+    "Mermin (2005), Distributional Effects of Reforming Social "
+    "Security through Benefit Reductions, report 411260, DYNASIM3 Runid 432"
+)
+# Table 1's 75-year deficit/surplus row is CBO's, not DYNASIM3 output
+# (docs/errata/2026-09-29-anchor-provenance.md).
+MERMIN_CBO_ROW_DOCUMENT = (
+    "Mermin (2005), Distributional Effects of Reforming Social "
+    "Security through Benefit Reductions, report 411260; Table 1's 75-year "
+    "deficit/surplus row reports Congressional Budget Office (2005) estimates"
+)
+
+
 def mermin_locator(
     table: str,
     pdf_page: int,
     row_path: str,
     column_path: str,
     derivation: str | None = None,
+    document: str = MERMIN_DOCUMENT,
 ) -> list[dict[str, Any]]:
     locator = {
         "publisher": "Urban Institute",
-        "document": (
-            "Mermin (2005), Distributional Effects of Reforming Social "
-            "Security through Benefit Reductions, report 411260, DYNASIM3 Runid 432"
-        ),
+        "document": document,
         "page": {"pdf": pdf_page},
         "table": table,
         "row_path": row_path,
@@ -1681,11 +1692,12 @@ def build_sharing_rows() -> list[dict[str, Any]]:
 ORDERING_MISMATCH = {
     "frame": (
         "Our ordinal result is scored once on a common Phase-A frame; the "
-        "published order comes from DYNASIM/CBO actuarial scoring. No level alignment is claimed."
+        "published order comes from CBO's 2005 actuarial scoring, as "
+        "reported in Mermin (2005). No level alignment is claimed."
     ),
     "population": (
-        "Our common restricted PSID career frame differs from DYNASIM's projected "
-        "population and from the national actuarial population."
+        "Our common restricted PSID career frame differs from the national "
+        "population in CBO's 2005 actuarial projection."
     ),
     "year_basis": (
         "Our result is a common-evaluation-age per-capita analogue; the published "
@@ -1711,7 +1723,9 @@ def build_ordering_row() -> dict[str, Any]:
     test = ordering["tests"]["T2_mermin_kendall_tau"]
     return {
         "row_id": "dynasim.mermin.four_reform_cost_ordering",
-        "external_model": "DYNASIM3 / CBO 2005 actuarial scoring",
+        "external_model": (
+            "CBO 2005 actuarial scoring, as reported in Mermin (2005)"
+        ),
         "quantity": "Ordinal benefit-reduction ordering across four reforms",
         "comparison_scope": ["ordering"],
         "our": {
@@ -1746,6 +1760,7 @@ def build_ordering_row() -> dict[str, Any]:
                     "obtain deficit reductions (2.37, 1.55, 1.19, 0.57), "
                     "then sort descending."
                 ),
+                document=MERMIN_CBO_ROW_DOCUMENT,
             ),
         },
         "deviation": {
@@ -3280,12 +3295,17 @@ def build_audited_tranche_rows() -> list[dict[str, Any]]:
 assert len(legacy_rows) == 22
 assert len(legacy_reported_not_verified) == 20
 assert not any(".mermin." in row["row_id"] for row in legacy_rows)
+# Select the Urban-published rows by their immutable row-id family, not by
+# the display label: the cost-ordering row keeps its dynasim.mermin id but
+# is labelled as CBO's estimates (docs/errata/2026-09-29-anchor-provenance.md).
 dynasim_rows = [
-    row
-    for row in legacy_all_rows
-    if row["external_model"].startswith("DYNASIM")
+    row for row in legacy_all_rows if row["row_id"].startswith("dynasim.")
 ]
 assert len(dynasim_rows) == 32
+assert (
+    sum(row["external_model"].startswith("DYNASIM") for row in dynasim_rows)
+    == 31
+)
 for row in dynasim_rows:
     for locator in row["published"]["source_locators"]:
         assert locator["row_path"]
@@ -3581,7 +3601,23 @@ def build_audited_tranche_rows() -> list[dict[str, Any]]:
             ),
             "revision": 1,
         }
-    ]
+    ],
+    "dynasim.mermin.four_reform_cost_ordering": [
+        {
+            "changed_fields": [
+                "/concept_mismatch/frame",
+                "/concept_mismatch/population",
+                "/external_reference",
+                "/source_pin/exact_locators/0/document",
+            ],
+            "note": (
+                "Relabeled the published order as CBO's 2005 estimates as "
+                "reported in Mermin (2005) Table 1, not DYNASIM3 output, per "
+                "the 2026-09-29 anchor-provenance erratum."
+            ),
+            "revision": 1,
+        }
+    ],
 }
 LABEL_MISMATCH_ROWS = {
     "ssa.reported_taxable_earnings_per_worker",
diff --git a/benchmarks/schema.py b/benchmarks/schema.py
--- a/benchmarks/schema.py
+++ b/benchmarks/schema.py
@@ -2059,6 +2059,25 @@ def reconstruct_legacy_matrix(
             entry["source_pin"]["exact_locators"][0][
                 "observation_range"
             ] = "A40:A47+B40:B47 (calendar years 2015-2022)"
+        if row_id == "dynasim.mermin.four_reform_cost_ordering":
+            entry["external_reference"] = (
+                "DYNASIM3 / CBO 2005 actuarial scoring"
+            )
+            entry["concept_mismatch"]["frame"] = (
+                "Our ordinal result is scored once on a common Phase-A "
+                "frame; the published order comes from DYNASIM/CBO "
+                "actuarial scoring. No level alignment is claimed."
+            )
+            entry["concept_mismatch"]["population"] = (
+                "Our common restricted PSID career frame differs from "
+                "DYNASIM's projected population and from the national "
+                "actuarial population."
+            )
+            entry["source_pin"]["exact_locators"][0]["document"] = (
+                "Mermin (2005), Distributional Effects of Reforming Social "
+                "Security through Benefit Reductions, report 411260, "
+                "DYNASIM3 Runid 432"
+            )
         require(row_id in seed_by_id, f"legacy seed row is missing: {row_id}")
         record = seed_by_id[row_id]
         our = {
diff --git a/tests/test_benchmarks.py b/tests/test_benchmarks.py
--- a/tests/test_benchmarks.py
+++ b/tests/test_benchmarks.py
@@ -1077,9 +1077,15 @@ def test__benchmark_registry__retains_exact_dynasim_locators():
     dynasim = [
         entry
         for entry in registry["entries"]
-        if entry["external_reference"].startswith("DYNASIM")
+        if entry["row_id"].startswith("dynasim.")
     ]
     assert len(dynasim) == 32
+    relabelled = [
+        entry["row_id"]
+        for entry in dynasim
+        if not entry["external_reference"].startswith("DYNASIM")
+    ]
+    assert relabelled == ["dynasim.mermin.four_reform_cost_ordering"]
 
     derived_ids = set()
     for entry in dynasim:

What the prototype showed

I applied the patch above to a throwaway checkout of origin/master at dd222b9, ran python benchmarks/build_registry.py to regenerate registry.json (new SHA-256 1a4df3d8…), and ran:

• build_registry.py --check: passes.
• build_history.py --check: passes. The frozen 42-line seed prefix does not depend on the current registry.
• The legacy Cross-model validation matrix: SSA, CBO, DYNASIM, WISH statutory #352 matrix round trip (_check_benchmark_migration_round_trip): passes. With the override, the reconstruction still hashes to MERGED_MATRIX_SHA256.
• schema.validate_append_mostly_registry(HEAD registry, new registry): passes. The revision's changed_fields exactly cover the four changed leaves.
• test__benchmark_registry__retains_exact_dynasim_locators (patched) and test__benchmark_registry_and_history__are_append_only_across_commits: pass.
• build_wall.py --check fails with "latest wall records do not use the current registry", as expected. Six tests/test_benchmarks.py tests fail for the same reason: the registry SHA moved without a record set, and CURRENT_REGISTRY_SHA256 still pins 7231b306….

The regenerated registry.json differs from master only in this entry's four fields and its spec_revisions.

I then rehearsed the append in the same throwaway checkout. The run artifact was a local placeholder committed only there and never pushed, and it is not a real evaluation. The candidate set copied the tranche-2 records with the new run SHA and registry SHA. Steps and results:

• append_history.py --check and --append: pass.
• Regenerating wall.md: build_registry.py, build_history.py and build_wall.py --check all pass.
• After updating the four pinned SHA constants, 8 of the 10 tests/test_benchmarks.py tests pass, including all six that failed before the append.

The two that still fail would fail on any third record set, relabel or not. The append PR has to handle them either way. I did not rehearse a fix for them.

• test__benchmark_history__reproduces_frozen_seed_prefix treats tranche 2 as the latest record set in three places in tests/test_benchmarks.py:

  • line 725: len(history) == SEED_ROW_COUNT + CURRENT_ROW_COUNT;
  • lines 746–749: manifest[-1] is the tranche-2 artifact;
  • lines 755–757: the latest CURRENT_ROW_COUNT records carry TRANCHE2_RUN_SHA256.

  Fixing only line 725 moves the failure to line 746, then to line 755. Those assertions have to be generalized, or made to address tranche 2 by position (history lines 43–143, manifest line 2).

• test__benchmark_builders__check_without_mutating_artifacts feeds the last CURRENT_ROW_COUNT history lines to scripts/build_benchmark_tranche2_history_candidate.py --check. The script's build() reads benchmarks/registry.json through load_registry(), derives registry_sha from those bytes, and hard-codes EXPECTED_REGISTRY_ROW_COUNT = 101. Under a changed registry it rejects the committed tranche-2 lines as stale, and registry_sha is the only key that differs. So the script must change as well as the test. For example, add an option to rebuild against tranche 2's committed registry bytes (7231b306…, from 5347bdc), and have the test feed it history lines 43–143.

Checklist for the append PR

• Apply the builder, schema and test changes above, then regenerate registry.json with python benchmarks/build_registry.py.
• Evaluate every active row under the new registry SHA per the README's append protocol. This row's gap_class and gap_note are unchanged.
• Append the record set with benchmarks/append_history.py, regenerate wall.md, and update the four CURRENT_*/WALL_SHA256 constants in tests/test_benchmarks.py.
• Update every check that assumes tranche 2 is the latest record set:
  • the three assertions in test__benchmark_history__reproduces_frozen_seed_prefix (tests/test_benchmarks.py:725, :746-749, :755-757);
  • scripts/build_benchmark_tranche2_history_candidate.py, which must be able to rebuild against tranche 2's own registry bytes;
  • the test that replays it, which must feed history lines 43–143.
• Add an addendum to docs/errata/2026-09-29-anchor-provenance.md saying that from this append the registry carries the corrected label. Leave its table of committed artifacts as the record of the earlier state. History lines 41 and 83 and wall.md carry only the row id, which stays.
• Run pytest -p no:xdist tests/test_benchmarks.py tests/test_anchor_provenance_erratum.py tests/test_mermin_report_title.py and the tier policy test.

Optional companion: the Mermin publisher capture

/external_capture_review/missing_after_refresh/mermin_2005_publisher_capture/status says publisher bytes "were not retrieved; the modern Urban Institute URLs returned 404", and so keeps all 20 Mermin rows reported_not_verified. On 2026-09-30 the exact PDF URL (…/publication/51966/411260-Distributional-Effects-of-Reforming-Social-Security-through-Benefit-Reductions.PDF) returned HTTP 200. Its bytes are identical to the archived copy (SHA-256 88934782…). The landing URL in the old capture request, which was built from a wrong title, returned 404 (docs/errata/2026-09-30-mermin-report-title.md).

A capture refresh under benchmarks/SOURCES-NEEDED.md's protocol (sidecar manifest, REFRESH) could therefore now land the publisher bytes. Whether the 20 rows then move to verified is a registry change, so it would ride on an append too. It is noted here, not required for the relabel.

Refs #488, #489.