Skip to content

fix(model): read a model collection's stored score instead of predicting a frame twice - #2206

Merged
lusoris merged 1 commit into
masterfrom
fix/model-set-score-idempotent
Oct 5, 2026
Merged

lusoris merged 1 commit into
masterfrom
fix/model-set-score-idempotent

Conversation

@lusoris

@lusoris lusoris commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Summary

A model collection's score can now be read more than once. vmaf_score_at_index_model_collection() failed with -EINVAL for a frame it had scored before, and so did vmaf_score_pooled_model_collection() over a range holding a frame already scored per frame: the prediction writes the members' scores and the four named bootstrap scores of the frame into the feature collector, which refuses a second write (feature "vmaf_0001" cannot be overwritten at index 2), and the pooled call predicts every frame of its range again. No score was wrong; the call failed.

The fix (core/src/libvmaf.c, read_predicted_collection_score()): a frame whose four named bootstrap scores are already in the collector returns them, as vmaf_score_at_index() reads a single model's stored score first. The returned values are the first prediction's, bit for bit, so the pooled score after a per-frame call equals a fresh session's. Upstream Netflix/vmaf has the same code.

Found by the VMAFx core API tests of RC4 work package 2 (draft #2199, rc4/api-wp2-core), where it is row T-MODEL-SET-SCORE-NOT-IDEMPOTENT-2026-10-05.

Type

  • fix — bug fix

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally — the commit hooks pass.
  • Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build --suite=fast --num-processes 4 → 354 OK, 0 fail on a CPU build (-Db_lto=false).
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. — not applicable: no SIMD/GPU code touched.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. — not applicable: no extractor touched.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (EUPL-1.2, fork-authored test).
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. — not a breaking change: a call that failed now succeeds; signatures and values unchanged.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to docs/adr/_index_fragments/_order.txt — no ADR: a bug fix with one way to fix it (read the stored values, as the single-model path does).

tidy: cpu core/src/libvmaf.c, core/test/test_model_collection_score_repeat.c 0 findings, 0 uncited NOLINT (dev container, clang-tidy 22.1.8, scripts/dev/tidy-lane.sh --only ... cpu).

scripts/dev/preflight.sh --stage msvcism: pass.

Bug-status hygiene (ADR-0165)

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below — not applicable.

make test-netflix-golden (GOLDEN_NINJA_JOBS=4, core/build-golden built with gcc): 280 passed, 3 skipped.

Cross-backend numerical results

Not applicable: no extractor or kernel changed; a first prediction is unchanged, a repeat returns its stored values.

Performance (if perf or feat)

Not applicable (fix). A repeated read skips the prediction (four collector look-ups instead of predicting every member).

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: trivial, a repeated read returns stored values.
  • Decision matrix — no alternatives: only-one-way fix (the single-model path already reads its stored score first).
  • AGENTS.md invariant note — core/src/AGENTS.d/pooling-and-bootstrap.md, "Collection per-frame score reads stored values first".
  • Reproducer / smoke-test command — below.
  • CHANGELOG fragment — changelog.d/fixed/model-collection-score-repeat.md.
  • Rebase note — docs/rebase-notes.md, "A model collection's per-frame score reads its stored values first".

Reproducer

meson setup build core -Db_lto=false
ninja -C build -j4 test/test_model_collection_score_repeat
python3 scripts/ci/run_meson_test.py -- -C build test_model_collection_score_repeat

Failing first (measured on master 782eba01f with the test and without the fix): test_second_frame_score fails at "second" (-EINVAL), and with that case removed test_pooled_after_frame_score fails at "pooled over frame 2" (-EINVAL). With the fix both pass, the second per-frame score is bit-identical to the first, and the pooled score bit-identical to a fresh session's.

Known follow-ups

@lusoris lusoris added the type:bug Something isn't working label Oct 5, 2026
…ing a frame twice (#2206)

* fix(model): read a model collection's stored score instead of predicting a frame twice

vmaf_score_at_index_model_collection() predicted every member model and
wrote the members' and the four named bootstrap scores of the frame into
the feature collector, which refuses a second write of a frame. A second
per-frame call of a frame, or vmaf_score_pooled_model_collection() over a
range holding a frame already scored per frame (its loop predicts every
frame of the range again), therefore failed with -EINVAL ("feature ...
cannot be overwritten"). No score was wrong; the call failed.

A frame whose four named bootstrap scores are already in the collector
now returns them, as vmaf_score_at_index() reads a single model's stored
score first. The values are the first prediction's, bit for bit, so a
pooled score equals a fresh session's. Upstream Netflix/vmaf has the same
code.

test_model_collection_score_repeat fails on master without the fix: the
second per-frame call and the pooled call after a per-frame call both
return -EINVAL. Golden gate 280 passed, 3 skipped.
@lusoris
lusoris force-pushed the fix/model-set-score-idempotent branch from f504696 to 6663208 Compare October 5, 2026 23:12
@lusoris
lusoris merged commit 6663208 into master Oct 5, 2026
5 of 79 checks passed
@lusoris
lusoris deleted the fix/model-set-score-idempotent branch October 5, 2026 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants