Repository navigation
fix(model): read a model collection's stored score instead of predicting a frame twice - #2206
Merged
Merged
Conversation
…ing a frame twice (#2206) * fix(model): read a model collection's stored score instead of predicting a frame twice vmaf_score_at_index_model_collection() predicted every member model and wrote the members' and the four named bootstrap scores of the frame into the feature collector, which refuses a second write of a frame. A second per-frame call of a frame, or vmaf_score_pooled_model_collection() over a range holding a frame already scored per frame (its loop predicts every frame of the range again), therefore failed with -EINVAL ("feature ... cannot be overwritten"). No score was wrong; the call failed. A frame whose four named bootstrap scores are already in the collector now returns them, as vmaf_score_at_index() reads a single model's stored score first. The values are the first prediction's, bit for bit, so a pooled score equals a fresh session's. Upstream Netflix/vmaf has the same code. test_model_collection_score_repeat fails on master without the fix: the second per-frame call and the pooled call after a per-frame call both return -EINVAL. Golden gate 280 passed, 3 skipped.
12 of 18 tasks
lusoris
force-pushed
the
fix/model-set-score-idempotent
branch
from
October 5, 2026 23:12
f504696 to
6663208
Compare
13 of 18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A model collection's score can now be read more than once.
vmaf_score_at_index_model_collection()failed with-EINVALfor a frame it had scored before, and so didvmaf_score_pooled_model_collection()over a range holding a frame already scored per frame: the prediction writes the members' scores and the four named bootstrap scores of the frame into the feature collector, which refuses a second write (feature "vmaf_0001" cannot be overwritten at index 2), and the pooled call predicts every frame of its range again. No score was wrong; the call failed.The fix (
core/src/libvmaf.c,read_predicted_collection_score()): a frame whose four named bootstrap scores are already in the collector returns them, asvmaf_score_at_index()reads a single model's stored score first. The returned values are the first prediction's, bit for bit, so the pooled score after a per-frame call equals a fresh session's. Upstream Netflix/vmaf has the same code.Found by the VMAFx core API tests of RC4 work package 2 (draft #2199,
rc4/api-wp2-core), where it is row T-MODEL-SET-SCORE-NOT-IDEMPOTENT-2026-10-05.Type
fix— bug fixChecklist
make format && make lintis green locally — the commit hooks pass.python3 scripts/ci/run_meson_test.py -- -C build --suite=fast --num-processes 4→ 354 OK, 0 fail on a CPU build (-Db_lto=false)./cross-backend-diffand the worst ULP is ≤ 2. — not applicable: no SIMD/GPU code touched..c/.cpp/.cu/.h/.hpp, it has the appropriate license header (EUPL-1.2, fork-authored test).!orBREAKING CHANGE:and the migration path is documented below. — not a breaking change: a call that failed now succeeds; signatures and values unchanged.docs/adr/_index_fragments/<NNNN-slug>.mdand the slug is appended todocs/adr/_index_fragments/_order.txt— no ADR: a bug fix with one way to fix it (read the stored values, as the single-model path does).tidy: cpu
core/src/libvmaf.c,core/test/test_model_collection_score_repeat.c0 findings, 0 uncited NOLINT (dev container, clang-tidy 22.1.8,scripts/dev/tidy-lane.sh --only ... cpu).scripts/dev/preflight.sh --stage msvcism: pass.Bug-status hygiene (ADR-0165)
docs/state.mdupdated — row T-MODEL-SET-SCORE-NOT-IDEMPOTENT-2026-10-05 under "Recently closed" (opened and closed by this PR; the open row on draft feat(api): implement the VMAFx core API on the engine (RC4 WP2, ADR-1852) #2199 is dropped when it rebases onto this).Netflix golden-data gate (ADR-0024)
assertAlmostEqual(...)score in the Netflix golden Python tests.make test-netflix-golden(GOLDEN_NINJA_JOBS=4,core/build-goldenbuilt with gcc): 280 passed, 3 skipped.Cross-backend numerical results
Not applicable: no extractor or kernel changed; a first prediction is unchanged, a repeat returns its stored values.
Performance (if
perforfeat)Not applicable (
fix). A repeated read skips the prediction (four collector look-ups instead of predicting every member).Deep-dive deliverables (ADR-0108)
AGENTS.mdinvariant note —core/src/AGENTS.d/pooling-and-bootstrap.md, "Collection per-frame score reads stored values first".changelog.d/fixed/model-collection-score-repeat.md.docs/rebase-notes.md, "A model collection's per-frame score reads its stored values first".Reproducer
Failing first (measured on master
782eba01fwith the test and without the fix):test_second_frame_scorefails at "second" (-EINVAL), and with that case removedtest_pooled_after_frame_scorefails at "pooled over frame 2" (-EINVAL). With the fix both pass, the second per-frame score is bit-identical to the first, and the pooled score bit-identical to a fresh session's.Known follow-ups