Repository navigation
fix(test): sunset VmafLegacyQualityRunner test methods per ADR-0749 - #241
Merged
Merged
Conversation
ADR-0749 (Accepted 2026-05-28) removed VmafLegacyQualityRunner from compat/python-vmaf/core/quality_runner.py. The ADR's stated decision was to also "delete all Python tests that exclusively exercise it" but the test deletion never landed, leaving: ImportError: cannot import name 'VmafLegacyQualityRunner' from 'vmaf.core.quality_runner' …which blocks the Netflix CPU Golden Tests (D24) gate. Removes from python/test/quality_runner_test.py: - Import of VmafLegacyQualityRunner (line 26) - test_executor_id (line 50) - test_run_vmaf_legacy_runner (line 62) - test_run_vmaf_legacy_runner_10le (line 86) - test_run_vmaf_legacy_runner_12le (line 114) - test_run_vmaf_legacy_runner_with_result_store (line 142) The canonical Netflix golden test `test_run_vmaf_runner` (uses VmafQualityRunner) is UNTOUCHED per CLAUDE.md §1 / §8 — those bit-exact assertions remain the numerical-correctness ground truth. Verified locally: python3 -c "import ast; ast.parse(...)" succeeds; no remaining VmafLegacyQualityRunner references; canonical test_run_vmaf_runner still present. Unblocks: Netflix CPU Golden Tests (D24). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lusoris
added a commit
that referenced
this pull request
May 30, 2026
…-0749 ADR-0749 removed VmafLegacyQualityRunner; PR #241 deleted exclusively-legacy tests in quality_runner_test.py but missed result_test.py. Removes ResultTest + ResultStoreTest classes (both exclusively legacy-runner driven). 4 unaffected Result test classes remain: ResultFormattingTest, ResultStoreTestWithNone, ResultAggregatingTest, ScoreAggregationTest. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
6 of 11 tasks
lusoris
added a commit
that referenced
this pull request
May 30, 2026
…-0749 (#253) ADR-0749 removed VmafLegacyQualityRunner; PR #241 deleted exclusively-legacy tests in quality_runner_test.py but missed result_test.py. Removes ResultTest + ResultStoreTest classes (both exclusively legacy-runner driven). 4 unaffected Result test classes remain: ResultFormattingTest, ResultStoreTestWithNone, ResultAggregatingTest, ScoreAggregationTest. Co-authored-by: lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
3 of 6 tasks
lusoris
added a commit
that referenced
this pull request
May 31, 2026
…migration (#408) Cross-reference docs/state.md against VMAFx/vmafx GitHub Issues. The repo carries 0 issues (only PRs); the historical bug-tracker numbers cited in state.md (#239, #857, plus the resolving PR refs #241, #310, #870) live on the archived lusoris/vmaf repo and now collide with unrelated PR numbers on VMAFx/vmafx — e.g. VMAFx PR #239 is a Cython rename hotfix, not the Vulkan async-fence work; VMAFx PR #241 is a test-sunset PR, not the v2 pending-fence ring; VMAFx has no #870 at all. This change qualifies every bare cite as `lusoris/vmaf#NNN` so future maintainers don't follow ambiguous numbers to a different PR on the active repo. No GitHub issues were closed and no rows added — repo state was already aligned; this PR is documentation hygiene only. Co-authored-by: Lusoris <lusoris@pm.me> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ADR-0749 removed the runner but never deleted the tests. The dangling
ImportErrorblocks Netflix CPU Golden Tests. Removes the 5 legacy-runner test methods + import. Canonicaltest_run_vmaf_runner(Netflix golden gate) untouched.Test plan
VmafLegacyQualityRunnerreferencestest_run_vmaf_runnerstill at line 89Deep-dive deliverables (ADR-0108)
docs/adr/0749-*.md## Alternatives consideredpytest python/test/quality_runner_test.py::QualityRunnerTest::test_run_vmaf_runnersucceeds (was ImportError)Lint clean (CLAUDE §12 r12)
python/test/quality_runner_test.pytouchedState drift (CLAUDE §12 r13)
🤖 Generated with Claude Code