feat(motion): derive motion2 and motion3 frame by frame so VMAF windows complete before the flush (ADR-2090)

.config/clang-tidy/measured-sources.txt

@@ -45,6 +45,7 @@ core/src/dnn/ort_backend.c
 core/src/dnn/tensor_io.c
 core/src/feature/adm.c
 core/src/feature/adm_tools.c
+core/src/feature/adm_view_dist.c
 core/src/feature/alias.c
 core/src/feature/arm64/adm_neon.c
 core/src/feature/arm64/cambi_neon.c
@@ -674,6 +675,7 @@ core/test/test_motion_min_dim.c
 core/test/test_motion_neon.c
 core/test/test_motion_pipeline_neon.c
 core/test/test_motion_v2_simd.c
+core/test/test_motion_window_incremental.c
 core/test/test_ms_ssim_decimate.c
 core/test/test_ms_ssim_decimate_coverage.c
 core/test/test_niqe.c

.config/lint-exceptions.d/clang-tidy-coverage.toml

@@ -193,6 +193,11 @@ path = "core/test/test_rust_cambi_kernels.c"
 reason = "Built only with -Denable_rust_features=true, which needs cargo; the dev image has no Rust toolchain yet (ADR-1713 follow-up: a pinned toolchain in dev/Containerfile), so no lane configures it. Needs a Rust-enabled lane."
 expires = 2026-12-31
 
+[[exception]]
+path = "core/test/test_rust_motion_window_incremental.c"
+reason = "Built only with -Denable_rust_features=true, which needs cargo; the dev image has no Rust toolchain yet (ADR-1713 follow-up: a pinned toolchain in dev/Containerfile), so no lane configures it. Needs a Rust-enabled lane."
+expires = 2026-12-31
+
 [[exception]]
 path = "core/tools/compat/win32/getopt.c"
 reason = "Windows-only translation unit (built only for host_machine.system() == 'windows'); no Linux or macOS compile database has an entry. Needs a Windows lane with clang-tidy."

.standards.yaml

@@ -227,6 +227,10 @@ exceptions:
     path: "core/test/test_rust_cambi_kernels.c"
     reason: "Built only with -Denable_rust_features=true, which needs cargo; the dev image has no Rust toolchain yet (ADR-1713 follow-up: a pinned toolchain in dev/Containerfile), so no lane configures it. Needs a Rust-enabled lane."
     expires: "2026-12-31"
+  - rule: clang-tidy-coverage
+    path: "core/test/test_rust_motion_window_incremental.c"
+    reason: "Built only with -Denable_rust_features=true, which needs cargo; the dev image has no Rust toolchain yet (ADR-1713 follow-up: a pinned toolchain in dev/Containerfile), so no lane configures it. Needs a Rust-enabled lane."
+    expires: "2026-12-31"
   - rule: clang-tidy-coverage
     path: "core/test/test_rust_predict.c"
     reason: "Built only with -Denable_rust_features=true, which needs cargo; the dev image has no Rust toolchain yet (ADR-1713 follow-up: a pinned toolchain in dev/Containerfile), so no lane configures it. Needs a Rust-enabled lane."

changelog.d/added/adm-cuda-shared-viewing-distances.md

@@ -0,0 +1,6 @@
+- **`adm_cuda` evaluates two viewing distances in one instance (ADR-2795).**
+  With `adm_norm_view_dist_extra`, or when two models such as
+  `vmaf_v1.0.16_3d0h` and `_5d0h` run on `--backend cuda`, the CUDA twin runs
+  the wavelet transform once per scale and the other kernels per distance, and
+  returns the CPU's scores for both distances bit for bit
+  ([Two viewing distances share one `adm`](docs/metrics/adm.md#two-viewing-distances-share-one-adm)).

changelog.d/added/api-window-scores.md

@@ -15,5 +15,5 @@
   into windows of `n_stats` seconds or `n_stats_frames` frames (#2138), and
   `vmafx_context_max_in_flight` reports the most frames a context holds after
   a submit (#2238). Windows over `motion2` / `motion3`, and so over VMAF
-  models, complete at the flush in this release. ABI 0.1.8. See
+  models, complete one frame after their last frame (ADR-2090). ABI 0.1.8. See
   [window scores](docs/api/vmafx/windows.md).

changelog.d/changed/motion-window-incremental.md

@@ -0,0 +1,12 @@
+- **`motion2` / `motion3` are final one frame after their frame, not at the
+  flush (RC4, ADR-2090).** The integer motion extractors (`motion`,
+  `motion_v2`) and every GPU twin that derived `motion2` / `motion3` at the
+  end of the stream now write a frame's scores as soon as the frame after it
+  is scored (frame 2 for frames 0 and 1 with `motion_five_frame_window`); the
+  last frame's scores still come with the flush. The values are unchanged,
+  bit for bit: the derivation runs the same statements in the same order and
+  carries the moving average from frame to frame. A per-frame model score, a
+  metadata callback for a model, `vmaf_score_pooled()` over the frames read
+  so far, and a VMAFx window over a VMAF model are therefore available while
+  the stream runs; on CUDA, `motion_cuda` completes frames per readback
+  batch of eight. See [motion](docs/metrics/motion.md#when-motion2-and-motion3-are-final).

changelog.d/changed/rust-motion-twin-advance.md

@@ -0,0 +1,8 @@
+- **`motion_rust` writes `motion2` / `motion3` frame by frame, as the C `motion`
+  does (ADR-2090).** With the Rust twin, a frame's `motion2` and `motion3`
+  arrive as soon as the frame after it is scored instead of at the flush, so a
+  window over a VMAF model completes during the stream with
+  `VMAF_FEATURE_IMPL=rust` too; the values are unchanged, bit for bit. A Rust
+  twin now gets its own `advance` callback where its C extractor has one,
+  instead of inheriting the C callback, which made `motion_rust` fail at the
+  flush. See [Motion](docs/metrics/motion.md#rust-implementation).

changelog.d/fixed/engine-read-fed-frame-fence.md

@@ -0,0 +1,7 @@
+- **A score read of a frame still on a worker thread answered "invalid"
+  (RC4, ADR-2090).** `vmaf_feature_score_at_index()` waited for the worker
+  threads only when the frame's slot existed but was unwritten; for a fed
+  frame whose feature had no slot yet (its first score, or a frame past the
+  first eight) it returned `-EINVAL` at once. It now waits for the frames in
+  flight before it answers, as the documentation of `-EAGAIN` describes; a
+  feature name no extractor writes still returns `-EINVAL`.

core/api/vmafx.toml

@@ -2068,7 +2068,7 @@ name = "vmafx_window_submit"
 header = "vmafx/score.h"
 since = "0.1"
 returns = "status"
-doc = "Ask for `request.target` pooled with every method of `request.pool_mask` over frames `first` to `last`, and return at once. The window completes when every scored frame of its range is final, or at vmafx_flush(), which completes it over the frames the stream had and flags it VMAFX_WINDOW_PARTIAL when the stream ended before `last`. A device backend collects a frame's scores one submit later; motion2 / motion3 of the integer motion extractors, and so every VMAF model, are final only at vmafx_flush() in this release. The context's completion thread (started by its first window) finds completion: the worker that finishes a frame, a submit, a flush, an import and this call wake it, so a window completes whether or not the feeding thread calls again; a window already final completes right after this call. At most 1024 windows of a context are open; one more is VMAFX_E_BUSY naming `context`. The caller holds the window until vmafx_window_release(). Added in ABI 0.1.8."
+doc = "Ask for `request.target` pooled with every method of `request.pool_mask` over frames `first` to `last`, and return at once. The window completes when every scored frame of its range is final, or at vmafx_flush(), which completes it over the frames the stream had and flags it VMAFX_WINDOW_PARTIAL when the stream ended before `last`. A device backend collects a frame's scores one submit later. motion2 / motion3 of a frame, and so every VMAF model, are final once the frame after it is scored (ADR-2090); the last frame's at vmafx_flush(). The context's completion thread (started by its first window) finds completion: the worker that finishes a frame, a submit, a flush, an import and this call wake it, so a window completes whether or not the feeding thread calls again; a window already final completes right after this call. At most 1024 windows of a context are open; one more is VMAFX_E_BUSY naming `context`. The caller holds the window until vmafx_window_release(). Added in ABI 0.1.8."
 params = [
   { name = "context", type = "VmafxContext", pass = "in" },
   { name = "request", type = "VmafxWindowRequest", pass = "in" },

core/include/vmafx/score.h

@@ -397,13 +397,13 @@ VMAFX_EXPORT VmafxStatus vmafx_score_pooled_model_set(VmafxContext *context,
  * `last`, and return at once. The window completes when every scored frame of its range is final,
  * or at vmafx_flush(), which completes it over the frames the stream had and flags it
  * VMAFX_WINDOW_PARTIAL when the stream ended before `last`. A device backend collects a frame's
- * scores one submit later; motion2 / motion3 of the integer motion extractors, and so every VMAF
- * model, are final only at vmafx_flush() in this release. The context's completion thread (started
- * by its first window) finds completion: the worker that finishes a frame, a submit, a flush, an
- * import and this call wake it, so a window completes whether or not the feeding thread calls
- * again; a window already final completes right after this call. At most 1024 windows of a context
- * are open; one more is VMAFX_E_BUSY naming `context`. The caller holds the window until
- * vmafx_window_release(). Added in ABI 0.1.8.
+ * scores one submit later. motion2 / motion3 of a frame, and so every VMAF model, are final once
+ * the frame after it is scored (ADR-2090); the last frame's at vmafx_flush(). The context's
+ * completion thread (started by its first window) finds completion: the worker that finishes a
+ * frame, a submit, a flush, an import and this call wake it, so a window completes whether or not
+ * the feeding thread calls again; a window already final completes right after this call. At most
+ * 1024 windows of a context are open; one more is VMAFX_E_BUSY naming `context`. The caller holds
+ * the window until vmafx_window_release(). Added in ABI 0.1.8.
  * @since 0.1
  */
 VMAFX_EXPORT VmafxStatus vmafx_window_submit(VmafxContext *context,

core/src/AGENTS.d/picture-ownership-and-dispatch.md

@@ -2,11 +2,29 @@
 paths:
   - core/src/libvmaf.c
   - core/src/feature/feature_extractor.cpp
-invariant: PREV_REF window uses counted references, n-2 kept only for reader; vmaf_read_pictures owns both pictures.
+invariant: PREV_REF counted refs, n-2 kept only for reader; vmaf_read_pictures owns both pictures; advance per frame.
 ---
 <!-- markdownlint-disable MD013 -->
 # Picture ownership, batch dispatch, and SYCL upload synchronization
 
+## Extractor advance after each frame and read fence (ADR-2090)
+
+`advance_extractors()` calls `fex->advance()` (optional; motion window
+extractors) on every registered context at the end of a successful
+`vmaf_engine_read_pictures()` (via `read_pictures_owned()`),
+`vmaf_read_pictures_sycl()`, `fence_for_read()` and `vmaf_engine_advance()`
+(VMAFx completion thread, engine lock held); never after flush, never
+concurrent with that context's extract / collect / flush, never from a worker. Pooled CPU
+extractor (worker pool): advance runs on the registered context (never
+extracts), engine sets `is_initialized` so close frees what advance built, as
+the threaded flush does. Other contexts: only once initialised. Keep the call
+on every frame-feeding entry point; a new entry point without it makes
+motion2 / motion3 final only at flush there. `vmaf_engine_feature_score_at_index()`
+fences on `-EINVAL` too when `index <= last_index` (fed frame without a slot
+yet), then re-reads; unknown name still `-EINVAL`.
+Guards: `test_motion_window_incremental` (threaded runs fail without the
+fence), `test_vmafx_window`, `test_motion_window_advance_contract.py`.
+
 ## PREV_REF window: earlier reference frames, counted references (ADR-1072, ADR-0778, ADR-1478)
 
 Context keeps reference picture of frame n-1 (`vmaf->prev_ref`). Frame n-2

core/src/AGENTS.d/rust-extractor-framework.md

@@ -6,52 +6,58 @@ paths:
   - core/src/feature/tad_rust.c
   - core/src/libvmaf.c
   - core/src/meson.build
-invariant: One Rust archive in libvmaf only; registry reaches Rust via the shim accessor; twins inherit C descriptor.
+invariant: One Rust archive, engine library only; registry reaches Rust via shim accessor; twins inherit C descriptor.
 ---
 <!-- markdownlint-disable MD013 -->
 # Rust extractor framework (ADR-1713)
 
-- **Separate, dependency-free workspace.** `core/src/rust/Cargo.toml` is its
-  own workspace (the root one `exclude`s it and the TAD crate, which names it
-  with `package.workspace`). Its `Cargo.lock` holds no external crate; adding
-  one breaks Meson's `cargo build --offline --locked` and the empty-CARGO_HOME
-  step of `rust-ci.yml`, on purpose. Never move these crates back into the
-  root workspace: cargo resolves the whole workspace and the bindings need
-  `bindgen` from the registry.
-- **One archive, libvmaf only.** `core/src/meson.build` builds
-  `vmafx-core-rs` (`core/src/rust/staticlib`) with
-  `core/src/rust/build_staticlib.py` (offline, `--locked`, depfile) and
-  links it, the shim `core/src/rust/shim/rust_twins.cpp` and
-  `core/src/feature/tad_rust.c` into the `libvmaf` library target only. Never
-  add a Rust symbol to `libvmaf_feature_static_lib`, `predict_c_lib` or
-  `model.c`: test binaries extract those objects without the archive. A
-  second Rust staticlib in the same link duplicates the Rust std symbols.
-- **Registry through an accessor.** `vmaf_init()` calls
-  `vmaf_rust_twins_install()`, which installs `rust_extractor_at` with
+- **Separate, dependency-free workspace.** `core/src/rust/Cargo.toml` = own
+  workspace (root workspace `exclude`s it and TAD crate, which names it via
+  `package.workspace`). Its `Cargo.lock` holds no external crate; adding one
+  breaks Meson's `cargo build --offline --locked` and empty-CARGO_HOME step of
+  `rust-ci.yml`, on purpose. Never move these crates back into root workspace:
+  cargo resolves whole workspace, bindings need `bindgen` from registry.
+- **One archive, engine library only.** `core/src/meson.build` builds
+  `vmafx-core-rs` (`core/src/rust/staticlib`) via
+  `core/src/rust/build_staticlib.py` (offline, `--locked`, depfile) and links
+  it, shim `core/src/rust/shim/rust_twins.cpp` and
+  `core/src/feature/tad_rust.c` into engine library target only (`libvmafx`
+  since WP6 split, ADR-2094). Never add Rust symbol to
+  `libvmaf_feature_static_lib`, `predict_c_lib` or `model.c`: test binaries
+  extract those objects without archive. Second Rust staticlib in same link
+  duplicates Rust std symbols.
+- **Registry through accessor.** `vmaf_init()` calls
+  `vmaf_rust_twins_install()`, which installs `rust_extractor_at` via
   `vmaf_feature_extractor_install_rust_registry()`. Every registry walk in
-  `feature_extractor.cpp` goes through `registry_at()`; a new walk over
-  `feature_extractor_list[]` directly misses the Rust extractors and their
+  `feature_extractor.cpp` goes through `registry_at()`; new walk over
+  `feature_extractor_list[]` directly misses Rust extractors and their
   duplicate-name audit.
-- **Twins inherit the C descriptor.** `add_twin()` copies the C extractor and
-  replaces only the name, callbacks, `priv_size` (C size rounded up plus one
-  instance pointer at the tail) and adds `VMAF_FEATURE_EXTRACTOR_RUST`. The
-  option table, provided features and `reads_prev_prev_ref` are the C
-  extractor's, which the option parser and the feature-name dictionary rely
-  on. Do not give a twin options of its own. Copy includes `merge` and
-  `extend_name_dict` (ADR-2795): `twin_init()` calls `extend_name_dict` after
-  building the dictionary; the registry keeps the C extractor and the twin
-  apart by name, never by callback.
+- **Twins inherit C descriptor.** `add_twin()` copies C extractor, replaces
+  only name, callbacks, `priv_size` (C size rounded up plus one instance
+  pointer at tail), adds `VMAF_FEATURE_EXTRACTOR_RUST`. Option table, provided
+  features, `reads_prev_prev_ref` = C extractor's; option parser and
+  feature-name dictionary rely on them. No twin-own options. Copy includes
+  `merge` and `extend_name_dict` (ADR-2795): `twin_init()` calls
+  `extend_name_dict` after building dictionary; registry keeps C extractor and
+  twin apart by name, never by callback.
+- **Twin advance = shim trampoline (ADR-2090, MI-1).** `add_twin()` sets
+  `advance` to `twin_advance` iff C extractor defines one, else NULL; never
+  inherited C callback (C advance derives from Rust SADs into C priv, Rust
+  flush appends again, `-EINVAL`). `advance_one_extractor()`
+  (`core/src/libvmaf.c`) runs `init_shared_rust_twin()` before pooled twin's
+  first advance; keep both. Guard: `test_rust_motion_window_incremental`.
 - **C stays default.** `first_pass_eligible()` skips Rust twins when no flag
-  is asked for; only `vmaf_feature_extractor_impl_select()` (env
-  `VMAF_FEATURE_IMPL=rust`, read once through `vmaf_gpu_dispatch_env_get()`)
+  asks; only `vmaf_feature_extractor_impl_select()` (env
+  `VMAF_FEATURE_IMPL=rust`, read once via `vmaf_gpu_dispatch_env_get()`)
   swaps one in. Keep that call on every registration path
   (`vmaf_use_feature`, `vmaf_use_features_from_model`,
   `create_context_fallback`).
-- **ABI header is generated.** `core/src/rust/include/vmafx_rs.h` comes from
-  `scripts/dev/rust-abi-header.sh` (cbindgen 0.29.4); on a conflict take
-  either side and regenerate. `test_rust_abi_layout` compares every size and
-  offset with Rust's table, `test_rust_twin_registry` the registration,
-  `scripts/ci/rust_twin_diff.py` the scores (bit identity, no tolerance).
-- Symbols of the archive stay out of `libvmaf.so` via
-  `-Wl,--exclude-libs,libvmafx_core_rs.a`; `nm -D build/src/libvmaf.so | grep _RN`
-  must print nothing.
+- **ABI header generated.** `core/src/rust/include/vmafx_rs.h` comes from
+  `scripts/dev/rust-abi-header.sh` (cbindgen 0.29.4); on conflict take either
+  side, regenerate. `VMAFX_RS_ABI_VERSION` stays 1 until release ships ABI.
+  `test_rust_abi_layout` compares every size and offset with Rust's table,
+  `test_rust_twin_registry` registration, `scripts/ci/rust_twin_diff.py`
+  scores (bit identity, no tolerance).
+- Archive symbols stay out of engine shared library via
+  `-Wl,--exclude-libs,libvmafx_core_rs.a`;
+  `nm -D build/src/libvmafx.so | grep _RN` must print nothing.