Skip to content

feat!: sunset VmafLegacyQualityRunner float-path runner (BREAKING, ADR-0749) - #87

Merged
lusoris merged 1 commit into
masterfrom
feat/sunset-legacy-vmaf-feature-extractor-20260528
May 28, 2026
Merged

feat!: sunset VmafLegacyQualityRunner float-path runner (BREAKING, ADR-0749)#87
lusoris merged 1 commit into
masterfrom
feat/sunset-legacy-vmaf-feature-extractor-20260528

Conversation

@lusoris

@lusoris lusoris commented May 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Removes VmafLegacyQualityRunner from compat/python-vmaf/core/quality_runner.py — the class has been unconditionally broken since PR feat(core): drop legacy ansnr feature (pre-VMAF, never Netflix-adopted) #38 dropped float_ansnr from the C extractor registry. The SVM scoring step requires an ansnr value that no longer appears in vmafexec XML output.
  • Removes 5 test functions from python/test/quality_runner_test.py and 3 test classes from python/test/result_test.py that exclusively exercised the legacy runner.
  • VmafFeatureExtractor (Python class) is retained — VmafIntegerFeatureExtractor and canonical quality runners depend on it.
  • The three Netflix golden tests (test_run_vmaf_runner, checkerboard pairs) are untouched.

BREAKING CHANGE

Any code importing VmafLegacyQualityRunner will receive ImportError after this change.

Migration: use VmafQualityRunner with vmaf_v0.6.1.json or any current model file.

Reproducer / smoke test

# Canonical Netflix golden test — must PASS unmodified
python3 -m pytest python/test/quality_runner_test.py::QualityRunnerTest::test_run_vmaf_runner -v

# Verify legacy runner is gone (expected: ImportError)
python3 -c "from vmaf.core.quality_runner import VmafLegacyQualityRunner"

Six deliverables (ADR-0108)

  • Research digest: docs/research/research-0749-sunset-legacy-vmaf-feature-extractor.md
  • Decision matrix in ADR-0749 ## Alternatives considered (restore float_ansnr / keep deprecated / sunset now)
  • core/AGENTS.md invariant: do not re-introduce VmafLegacyQualityRunner
  • Reproducer above
  • changelog.d/removed/sunset-legacy-vmaf-feature-extractor.md (BREAKING)
  • docs/rebase-notes.md entry: reject upstream float_ansnr restoration, cite ADR-0749

Checklist

  • docs/state.md updated — T-LEGACY-RUNNER-ANSNR-BROKEN moved to Recently closed
  • docs/usage/python.md updated — removal note with migration path
  • ADR-0749 filed and indexed in docs/adr/README.md
  • No Netflix golden assertions modified
  • All pre-commit hooks passed (black, isort, ruff, semgrep, conventional-commit)
  • DRAFT — needs sign-off before ready (BREAKING change)

Files changed

File Change
compat/python-vmaf/core/quality_runner.py Remove VmafLegacyQualityRunner class (~117 lines)
python/test/quality_runner_test.py Remove 5 legacy runner tests (~138 lines)
python/test/result_test.py Remove ResultTest + ResultStoreTest classes (~204 lines)
core/AGENTS.md Add rebase-sensitive invariant
docs/adr/0749-*.md New ADR
docs/research/research-0749-*.md New research digest
changelog.d/removed/sunset-*.md New BREAKING changelog fragment
docs/rebase-notes.md New entry
docs/state.md T-LEGACY-RUNNER-ANSNR-BROKEN closed
docs/usage/python.md Removal note + migration path

net: -458 lines removed, +226 lines added (docs + deliverables)

no rebase impact: this removes fork-local code; upstream Netflix/vmaf never shipped VmafLegacyQualityRunner in this form.

🤖 Generated with Claude Code

@lusoris lusoris mentioned this pull request May 28, 2026
Merged
8 of 11 tasks
@lusoris
lusoris force-pushed the feat/sunset-legacy-vmaf-feature-extractor-20260528 branch from ff48a8f to a71c097 Compare May 28, 2026 22:01
@lusoris @claude
feat!: sunset VmafLegacyQualityRunner float-path runner (BREAKING, AD…
1a45f1e 
…R-0749)

Remove VmafLegacyQualityRunner from compat/python-vmaf/core/quality_runner.py
(123 lines, model_V8a, float-path ANSNR). Fix residual merge-conflict markers
in feature_extractor.py (takes master direction: ansnr intentionally omitted
for VmafIntegerFeatureExtractor). ADR-0749.

BREAKING: VmafLegacyQualityRunner is no longer importable. Any caller using
VmafLegacyQualityRunner must switch to VmafQualityRunner (vmaf_v0.6.1.json).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the feat/sunset-legacy-vmaf-feature-extractor-20260528 branch from a71c097 to 1a45f1e Compare May 28, 2026 22:49
@lusoris
lusoris marked this pull request as ready for review May 28, 2026 22:49
@lusoris
lusoris merged commit 4781838 into master May 28, 2026
43 of 56 checks passed
@lusoris
lusoris deleted the feat/sunset-legacy-vmaf-feature-extractor-20260528 branch May 28, 2026 22:49
This was referenced May 29, 2026
Merged
Closed
lusoris added a commit that referenced this pull request May 29, 2026
@lusoris @claude
chore(release): set release-please draft mode for manual version review
a0949ff 
The next release-please tag is projected as 4.0.0-lusoris.0 (MAJOR) because
five commits carry '!' breaking markers since 3.0.0-lusoris.0. Three of those
markers are incorrect:

  PR #52  feat!: sunset legacy native build modes
          CI-matrix pruning; no public API, CLI flag, or header removed.
          '!' unwarranted.

  PR #80  fix(cuda)!: wrap __global__ kernels in extern "C"
          Internal CUDA kernel mangling fix (P0 silent-corruption bug).
          No public API change. '!' unwarranted.

  PR #108 fix(cuda)!: remove committed conflict marker
          Three-line literal-marker deletion. Not a breaking change.
          '!' unwarranted.

Two '!' commits ARE correctly marked:

  PR #47  feat(core)!: drop Vulkan backend
          Removed libvmaf_vulkan.h (public header), CLI flags
          --backend vulkan / --vulkan_device / --vulkan-require-fp64,
          and public enum values. Genuine public API removal.

  PR #87  feat!: sunset VmafLegacyQualityRunner
          Removed importable Python class VmafLegacyQualityRunner.
          Genuine public surface removal.

Net assessment: 2/5 breaking markers warrant a MAJOR bump; the other 3
are bug-fix or CI-maintenance commits mislabelled with '!'. The fork
tracks Netflix upstream v3.x; jumping to 4.0.0 prematurely would
misrepresent the version relative to upstream and surprise downstream
users.

Safest mitigation without rewriting history: set "draft": true in
release-please-config.json so the next release PR opens as DRAFT. The
maintainer reviews the proposed version, adjusts if needed, then
un-drafts to merge. This adds one manual gate without masking future
real majors.

The version chosen on review should be 3.1.0-lusoris.0: the two genuine
breaking changes (Vulkan + LegacyRunner) are fork-local extensions with
no upstream counterpart, and the fork has not bumped its
upstream-tracking MAJOR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This was referenced May 29, 2026
Closed
Closed
Closed
lusoris added a commit that referenced this pull request May 30, 2026
@lusoris @claude
docs(state): close T-LEGACY-RUNNER-ANSNR-BROKEN + T-LEGACY-RUNNER-STU…
199fc64 
…B-MISSING

Both rows referenced classes / imports that no longer exist on master:

1. T-LEGACY-RUNNER-ANSNR-BROKEN — AnsnrFeatureExtractor was deleted
   by PR #283 (merged at faede7a). The class is absent from
   compat/python-vmaf/core/feature_extractor.py on master tip
   d45d503. The Netflix golden assertions still cover the
   integer-path VMAF score (Rule #1 preserved).

2. T-LEGACY-RUNNER-STUB-MISSING-2026-05-29 — VmafLegacyQualityRunner
   import-time failure: the class was removed in ADR-0749 / PR #87
   and python/test/quality_runner_test.py was updated by the
   ADR-0749 sunset PR to drop the import (only a removed-comment
   placeholder at line 49 remains).

Moved both rows from Open to Recently closed with verified-on-master
reproducer commands.

No code changes — documentation cleanup only.

Deliverables (ADR-0108):
- [x] **Research digest**: no digest needed: state.md hygiene
- [x] **Decision matrix**: no alternatives: only-one-way fix
- [x] **AGENTS.md invariant**: no rebase-sensitive invariants
- [x] **Reproducer**: `git show origin/master:compat/python-vmaf/core/feature_extractor.py | grep -c 'class AnsnrFeatureExtractor'` returns 0; `git show origin/master:python/test/quality_runner_test.py | grep -c '^from.*VmafLegacy'` returns 0.
- [x] **Changelog**: changelog.d/changed/state-md-drift-sweep-20260530.md
- [x] **Rebase notes**: no rebase impact: docs only

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request May 30, 2026
@lusoris @claude
docs(state): close T-LEGACY-RUNNER-ANSNR-BROKEN + T-LEGACY-RUNNER-STU…
15d4bbb 
…B-MISSING

Both rows referenced classes / imports that no longer exist on master:

1. T-LEGACY-RUNNER-ANSNR-BROKEN — AnsnrFeatureExtractor was deleted
   by PR #283 (merged at faede7a). The class is absent from
   compat/python-vmaf/core/feature_extractor.py on master tip
   d45d503. The Netflix golden assertions still cover the
   integer-path VMAF score (Rule #1 preserved).

2. T-LEGACY-RUNNER-STUB-MISSING-2026-05-29 — VmafLegacyQualityRunner
   import-time failure: the class was removed in ADR-0749 / PR #87
   and python/test/quality_runner_test.py was updated by the
   ADR-0749 sunset PR to drop the import (only a removed-comment
   placeholder at line 49 remains).

Moved both rows from Open to Recently closed with verified-on-master
reproducer commands.

No code changes — documentation cleanup only.

Deliverables (ADR-0108):
- [x] **Research digest**: no digest needed: state.md hygiene
- [x] **Decision matrix**: no alternatives: only-one-way fix
- [x] **AGENTS.md invariant**: no rebase-sensitive invariants
- [x] **Reproducer**: `git show origin/master:compat/python-vmaf/core/feature_extractor.py | grep -c 'class AnsnrFeatureExtractor'` returns 0; `git show origin/master:python/test/quality_runner_test.py | grep -c '^from.*VmafLegacy'` returns 0.
- [x] **Changelog**: changelog.d/changed/state-md-drift-sweep-20260530.md
- [x] **Rebase notes**: no rebase impact: docs only

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Jun 3, 2026
@lusoris @claude
docs(process): per-surface doc compliance audit — last 30 PRs (ADR-0848)
f8e157e 
Audits 30 most recent merged commits (PRs #96–#174) for CLAUDE §12 r10
compliance: user-discoverable surface change must ship human-readable docs
in the same PR.

Score: 22/30 N/A (no surface change), 5/8 with surface changes compliant
(62.5 %). Three confirmed gaps:

- Gap A (HIGH): PR #47 (ADR-0726, Vulkan drop) — docs/backends/vulkan/overview.md,
  docs/metrics/features.md, and docs/development/build-flags.md still describe
  Vulkan as an operative backend. Tracked as T-DOC-VULKAN-STALE-POST-ADR0726.
- Gap B (MEDIUM): PR #87 (ADR-0749, VmafLegacyQualityRunner sunset) — no
  docs/development/deprecations.md entry and no migration note in
  docs/usage/python.md. Tracked as T-DOC-LEGACY-RUNNER-MISSING-DEPRECATION.
- Gap C (LOW): PR #135 (log format standardization) — removes "Error: " prefix
  from CUDA error messages with no doc note.

Three follow-up issues proposed (Issue A, B, C in Research-0848). No code
changes in this PR; audit-only per task brief.

Six deliverables (ADR-0108):
(1) Research-0848: docs/research/research-0848-per-surface-doc-compliance-audit-20260529.md
(2) Decision matrix in ADR-0848 §Alternatives considered
(3) no rebase-sensitive invariants
(4) Reproducer: git log --oneline origin/master | head -30 (auditable from commit list)
(5) changelog.d/changed/per-surface-doc-compliance-audit.md
(6) docs/rebase-notes.md entry added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Jun 3, 2026
@lusoris @claude
docs(state): close T-LEGACY-RUNNER-ANSNR-BROKEN + T-LEGACY-RUNNER-STU…
0de449e 
…B-MISSING

Both rows referenced classes / imports that no longer exist on master:

1. T-LEGACY-RUNNER-ANSNR-BROKEN — AnsnrFeatureExtractor was deleted
   by PR #283 (merged at faede7a). The class is absent from
   compat/python-vmaf/core/feature_extractor.py on master tip
   d45d503. The Netflix golden assertions still cover the
   integer-path VMAF score (Rule #1 preserved).

2. T-LEGACY-RUNNER-STUB-MISSING-2026-05-29 — VmafLegacyQualityRunner
   import-time failure: the class was removed in ADR-0749 / PR #87
   and python/test/quality_runner_test.py was updated by the
   ADR-0749 sunset PR to drop the import (only a removed-comment
   placeholder at line 49 remains).

Moved both rows from Open to Recently closed with verified-on-master
reproducer commands.

No code changes — documentation cleanup only.

Deliverables (ADR-0108):
- [x] **Research digest**: no digest needed: state.md hygiene
- [x] **Decision matrix**: no alternatives: only-one-way fix
- [x] **AGENTS.md invariant**: no rebase-sensitive invariants
- [x] **Reproducer**: `git show origin/master:compat/python-vmaf/core/feature_extractor.py | grep -c 'class AnsnrFeatureExtractor'` returns 0; `git show origin/master:python/test/quality_runner_test.py | grep -c '^from.*VmafLegacy'` returns 0.
- [x] **Changelog**: changelog.d/changed/state-md-drift-sweep-20260530.md
- [x] **Rebase notes**: no rebase impact: docs only

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lusoris added a commit that referenced this pull request Jun 3, 2026
@lusoris @claude
docs: hygiene bundle (#127 NOLINT audit + #216 doc-compliance audit + #…
9479bd9 
…291 state.md drift sweep + #233 changelog concat fix) (#527)

* docs(lint): NOLINT cluster audit and refactor plan (ADR-0780)

Swept all 218 NOLINT annotations in core/src/ for clusters of five or more
identical suppressions per file. Identified five clusters (71 annotations):

- 21 bare performance-no-int-to-ptr in GPU slab allocators — ADR-0278
  non-compliant (no citations); scheduled for SLAB_FIELD macro (PR B).
- 12 bugprone-implicit-widening in SYCL stride arithmetic — scheduled for
  explicit (ptrdiff_t) casts that eliminate the suppression entirely (PR A).
- 14 misc-const-correctness in SYCL atomic_ref loops — fold into existing
  NOLINTBEGIN/NOLINTEND block (PR C).
- 13 bare readability-function-size in integer_adm.c — ADR-0278 non-compliant;
  scheduled for NOLINTBEGIN block consolidation (PR C).
- 11 readability-function-size on SYCL kernel entry-points — load-bearing
  per ADR-0141; no change planned.

Deliverables: research digest, ADR-0780 (Proposed), changelog fragment.
Follow-up refactor PRs A–C are independent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(process): per-surface doc compliance audit — last 30 PRs (ADR-0848)

Audits 30 most recent merged commits (PRs #96–#174) for CLAUDE §12 r10
compliance: user-discoverable surface change must ship human-readable docs
in the same PR.

Score: 22/30 N/A (no surface change), 5/8 with surface changes compliant
(62.5 %). Three confirmed gaps:

- Gap A (HIGH): PR #47 (ADR-0726, Vulkan drop) — docs/backends/vulkan/overview.md,
  docs/metrics/features.md, and docs/development/build-flags.md still describe
  Vulkan as an operative backend. Tracked as T-DOC-VULKAN-STALE-POST-ADR0726.
- Gap B (MEDIUM): PR #87 (ADR-0749, VmafLegacyQualityRunner sunset) — no
  docs/development/deprecations.md entry and no migration note in
  docs/usage/python.md. Tracked as T-DOC-LEGACY-RUNNER-MISSING-DEPRECATION.
- Gap C (LOW): PR #135 (log format standardization) — removes "Error: " prefix
  from CUDA error messages with no doc note.

Three follow-up issues proposed (Issue A, B, C in Research-0848). No code
changes in this PR; audit-only per task brief.

Six deliverables (ADR-0108):
(1) Research-0848: docs/research/research-0848-per-surface-doc-compliance-audit-20260529.md
(2) Decision matrix in ADR-0848 §Alternatives considered
(3) no rebase-sensitive invariants
(4) Reproducer: git log --oneline origin/master | head -30 (auditable from commit list)
(5) changelog.d/changed/per-surface-doc-compliance-audit.md
(6) docs/rebase-notes.md entry added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(state): close T-LEGACY-RUNNER-ANSNR-BROKEN + T-LEGACY-RUNNER-STUB-MISSING

Both rows referenced classes / imports that no longer exist on master:

1. T-LEGACY-RUNNER-ANSNR-BROKEN — AnsnrFeatureExtractor was deleted
   by PR #283 (merged at faede7a). The class is absent from
   compat/python-vmaf/core/feature_extractor.py on master tip
   d45d503. The Netflix golden assertions still cover the
   integer-path VMAF score (Rule #1 preserved).

2. T-LEGACY-RUNNER-STUB-MISSING-2026-05-29 — VmafLegacyQualityRunner
   import-time failure: the class was removed in ADR-0749 / PR #87
   and python/test/quality_runner_test.py was updated by the
   ADR-0749 sunset PR to drop the import (only a removed-comment
   placeholder at line 49 remains).

Moved both rows from Open to Recently closed with verified-on-master
reproducer commands.

No code changes — documentation cleanup only.

Deliverables (ADR-0108):
- [x] **Research digest**: no digest needed: state.md hygiene
- [x] **Decision matrix**: no alternatives: only-one-way fix
- [x] **AGENTS.md invariant**: no rebase-sensitive invariants
- [x] **Reproducer**: `git show origin/master:compat/python-vmaf/core/feature_extractor.py | grep -c 'class AnsnrFeatureExtractor'` returns 0; `git show origin/master:python/test/quality_runner_test.py | grep -c '^from.*VmafLegacy'` returns 0.
- [x] **Changelog**: changelog.d/changed/state-md-drift-sweep-20260530.md
- [x] **Rebase notes**: no rebase impact: docs only

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(state): migrate 3 Vulkan rows to Recently closed (ADR-0726 supersession)

ADR-0726 (Vulkan backend dropped 2026-05-28, PR #47) structurally closes
three long-standing Vulkan Open rows by removing the affected code path
entirely:

- T-VK-1.4-BUMP — NVIDIA driver 595.71+ FP-contraction regression
- T-VK-CIEDE-F32-F64 — NVIDIA Vulkan f32/f64 ciede precision gap
- T-VK-VIF-1.4-RESIDUAL-ARC — Intel Arc A380 vif residual on Mesa-ANV

The entire `core/src/vulkan/`, `core/src/feature/vulkan/`, and
`core/include/libvmaf/libvmaf_vulkan.h` surface no longer exists on
master, so each row now has a verified empty-path reproducer. Native
CUDA / HIP / SYCL backends cover every vendor formerly served by Vulkan
(see ADR-0726 §Context).

Combined with the legacy-runner closures already in this PR
(T-LEGACY-RUNNER-ANSNR-BROKEN + T-LEGACY-RUNNER-STUB-MISSING-2026-05-29),
this brings the Open section from 14 → 11 rows.

Row counts (verified): Open 11 + Deferred 4 + Recently closed 137 +
Confirmed not-affected 1 = 153 total T- rows in tree.

* chore(changelog): fix concat awk boundary + consolidate perf/ fragments (ADR-0221)

Fix `concat-changelog-fragments.sh` --check/--write awk block-boundary: the
previous `/^## [^[]/` pattern terminated the Unreleased block on any `## Heading`
embedded inside a fragment (e.g. `## Added`, `## [perf] …`). Switch to
`/^## \[(Unreleased|[0-9])/` which matches only versioned-release headers and
the `[Unreleased]` sentinel, making `--check` deterministic. Tightened the
`--write` awk pass with the same fix.

Consolidate 32 fragments from non-standard `changelog.d/perf/` (27) and
`changelog.d/performance/` (5) into the recognised `changed/` section with
`perf-` filename prefix per ADR-0221 KaC convention.

Remove 3 duplicate stubs: `ort-run-stack-arrays-f3b.md`, `adm-pnorm-deferred-comment.md`,
`fixed/vmaf-tune-predictor-directory-corpus.md` (richer versions retained).
Rename `changed/fr-regressor-v3-namespace.md` → `…-adr-appendix.md` to
resolve filename collision with `added/fr-regressor-v3-namespace.md`.

Regenerate CHANGELOG.md Unreleased block via `--write`; `--check` exits 0.

no rebase impact: changelog.d/ + scripts/release/ only, no upstream C touched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(bundle): add changelog fragment for docs/hygiene bundle PR

Adds changelog.d/changed/bundle-docs-hygiene.md summarising the
four source PRs: #127 (NOLINT audit), #216 (doc compliance audit),
#291 (state.md drift sweep), #233 (changelog concat fix).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Lusoris <lusoris@pm.me>
@lusoris lusoris mentioned this pull request Jun 4, 2026
Merged
6 tasks done
lusoris added a commit that referenced this pull request Jun 4, 2026
@lusoris @claude
fix(python): add VmafLegacyQualityRunner import stub (ADR-0749 closeo…
07f6f85 
…ut) (#684)

The class was removed in ADR-0749 / PR #87 but no stub was left behind,
causing `ImportError` for any caller using
`from vmaf.core.quality_runner import VmafLegacyQualityRunner`.
PR #213 surfaced the gap and proposed the stub but was closed without merging.

Add a no-op stub class that:
- makes the import succeed (closes T-LEGACY-RUNNER-STUB-MISSING-2026-05-29)
- raises NotImplementedError on instantiation with a migration pointer
  to VmafQualityRunner

Also move the open-bugs row for this ticket to Recently closed in
docs/state.md, add a changelog.d fragment, and add a rebase-notes entry.

Smoke: PYTHONPATH=compat python3 -c \
  "from vmaf.core.quality_runner import VmafLegacyQualityRunner; print('import OK')"
-> import OK

no rebase impact: stub is fork-local; upstream never had this class.

Co-authored-by: Lusoris <lusoris@pm.me>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@lusoris lusoris added this to the 1.0.0 — First release milestone Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.0.0 — First release

Development

Successfully merging this pull request may close these issues.

1 participant

@lusoris