Skip to content

fix(ai): restore ensemble registry license metadata + defer prod flip to one-shot retrain (ADR-1105) - #902

Merged
lusoris merged 1 commit into
masterfrom
fix/rc-polish-registry-changelog
Jun 13, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/rc-polish-registry-changelog

Conversation

@lusoris

@lusoris lusoris commented Jun 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

RC-polish: fixes the model/tiny/registry.json defects that PR #865
(8b7ae731a) introduced when it regenerated the five
fr_regressor_v2_ensemble_v1_seed{0..4} ONNX files in --smoke mode to
repair a codec_vocab 14→6 input-dim mismatch.

  1. License metadata restored (pure regression). fix: iter6 bundle — 6 fixes (wave32 comment, UBSan, TSAN, vmaf-tune CRF, AI scripts, MCP conformance) #865 dropped
    license / license_url / sigstore_bundle from the five rows,
    breaking test_every_entry_has_license_metadata (every entry must
    carry license metadata, smoke or not). Restored.
  2. Production flip deferred to the one-shot retrain (ADR-1105). The
    rows now ship smoke weights, but ADR-0321's
    test_fr_regressor_v2_ensemble_seed_rows_are_production asserts
    smoke: false. The ADR-0321 production weights were LOSO-validated at
    codec_vocab=14; the trim to 6 made them stale, and real production
    weights at width 6 require re-running export_ensemble_v2_seeds.py —
    part of the locked one-shot post-RC retrain (ensemble is in scope;
    piecemeal retraining is explicitly out of scope). For the RC we keep
    smoke: true (truthful about the shipped weights) and mark the
    production test xfail(strict=True). strict=True flips it back to a
    hard failure the moment the retrain lands real weights, forcing
    removal of the marker.

No Netflix golden-data assertion modified — model_registry_schema_test.py
is fork-local (CLAUDE.md §8 untouched).

Reproducer / smoke test

python3 -m pytest python/test/model_registry_schema_test.py -q
# -> 10 passed, 1 xfailed   (the deferred production assertion)
python3 ai/scripts/validate_model_registry.py
# -> OK: 26 registry entries valid against registry.schema.json

Deliverables (ADR-0108)

  • Research digest — no digest needed: metadata/test fix, no novel research.
  • Decision matrix — in ADR-1105 ## Alternatives considered (retrain-now / revert-ONNX / leave-red / delete-sidecars all evaluated).
  • AGENTS.md invariant note — no rebase-sensitive invariants: no code/ABI touched; the existing ADR-0321 "hand-flipping forbidden" invariant is unchanged.
  • Reproducer / smoke-test command — see the commands above.
  • CHANGELOG fragment — changelog.d/fixed/ensemble-v2-registry-license-restore-smoke-honest.md.
  • rebase-notes entry — no rebase impact: fork-local model metadata + fork-local test; no upstream-mirrored code touched.

Docs / state

  • Docs (§12 r10) — docs/ai/models/fr_regressor_v2_probabilistic.md status block corrected (production → smoke-placeholder-pending-retrain).
  • state.md (§12 r13) — T-ENSEMBLE-V2-PROD-FLIP-DEFERRED row added to Open bugs.
  • ffmpeg-patches (§12 r14) — N/A: no libvmaf C-API / CLI / header / meson_options change.
  • ADR — ADR-1105 (Accepted) + README index row.

… to one-shot retrain (ADR-1105)

PR #865 regenerated the five fr_regressor_v2_ensemble_v1_seed{0..4} ONNX
files in --smoke mode to fix a codec_vocab 14->6 input-dim mismatch, but
two side effects regressed the registry:

1. license / license_url / sigstore_bundle were dropped from the five
   rows, breaking test_every_entry_has_license_metadata. Restored here
   (every entry must carry license metadata, smoke or not).
2. The rows now ship smoke weights while ADR-0321's test asserts
   production (smoke=false). The ADR-0321 production weights were
   LOSO-validated at codec_vocab=14; the trim to 6 made them stale, and
   real production weights at width 6 require re-running
   export_ensemble_v2_seeds.py -- part of the locked one-shot post-RC
   retrain (ensemble is in scope; piecemeal retrain is explicitly out).

For the RC: keep smoke=true (matches the shipped weights honestly), and
mark test_fr_regressor_v2_ensemble_seed_rows_are_production
xfail(strict=True) with an ADR-1105 reason. strict=True flips it back to
a hard failure the moment the one-shot retrain lands real weights,
forcing removal of the marker.

No Netflix golden-data assertion modified (model_registry_schema_test.py
is fork-local).

- ADR-1105 (Accepted) + README index row
- docs/state.md: T-ENSEMBLE-V2-PROD-FLIP-DEFERRED row
- docs/ai/models/fr_regressor_v2_probabilistic.md: status block corrected
- changelog.d/fixed fragment

Local: python3 -m pytest python/test/model_registry_schema_test.py
       -> 10 passed, 1 xfailed; validate_model_registry.py -> OK 26 entries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@lusoris
lusoris merged commit bc88056 into master Jun 13, 2026
77 of 80 checks passed
@lusoris
lusoris deleted the fix/rc-polish-registry-changelog branch June 13, 2026 13:26
@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant