Skip to content

test(tools): cover frames with an odd width or height - #1664

Merged
lusoris merged 2 commits into
masterfrom
port/upstream-1604-odd-dimension-readback-test
Oct 1, 2026
Merged

lusoris merged 2 commits into
masterfrom
port/upstream-1604-odd-dimension-readback-test

Conversation

@lusoris

@lusoris lusoris commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Summary

This adds the regression test the fork was missing for the defect of our upstream Netflix/vmaf PR #1604. No reader code changes.

A subsampled plane occupies ceil(width / dec) * ceil(height / dec) samples in a raw or y4m file. Upstream's direct readers read the floor, so part of every odd-sized frame stays in the stream, the next frame is read from the wrong offset, and the tool crashes. The fork is not affected: VmafPicture carries ceiling chroma (vmaf_chroma_extent(), Research-0094), and the buffered reader the vmaf tool uses reads a frame in one piece. Nothing tested that, so a rebase that restored floor chroma would have gone unnoticed here.

test_video_input_odd_dims derives every sample from its plane, row, column and frame, reads three frames and then requires the end of the clip, through both reader entry points (video_input_fetch_into_vmaf_picture() and video_input_fetch_frame()):

Type

  • feat — new feature
  • fix — bug fix
  • perf — performance improvement
  • refactor — no behavior change
  • docs — documentation only
  • test — test-only
  • build / ci — tooling / infra
  • port — cherry-pick from upstream Netflix/vmaf
  • sycl / cuda / simd — backend-specific

Checklist

  • Commits follow Conventional Commits (the commit-msg hook enforces this).
  • make format && make lint is green locally. Not run in full. Run instead: clang-format --dry-run -Werror on the new file (clean), scripts/ci/tidy-ratchet.py --lane cpu --only core/test/test_video_input_odd_dims.c (0 warnings), and the pre-commit hook set (passed).
  • Unit tests pass: fast suite on a CPU build, 191 passed, 0 failed, 2 skipped, and on a CUDA build (gcc, nvcc, RTX 4090), 249 passed, 0 failed, 1 skipped; the new test also passes under ASan + UBSan + LeakSanitizer.
  • If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. None touched.
  • If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. None touched.
  • If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (see CONTRIBUTING.md).
  • If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below.
  • If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to docs/adr/_index_fragments/_order.txt.

Bug-status hygiene (ADR-0165)

  • docs/state.md updated: T-VIDINPUT-ODD-DIMENSION-READBACK-UNTESTED-2026-10-01 under Recently closed.

Netflix golden-data gate (ADR-0024)

  • I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
  • If I believe a golden value must change, I have explained why below AND pinged @lusoris for a CODEOWNERS exception.

Deep-dive deliverables (ADR-0108)

  • Research digest — no digest needed: test-only change.
  • Decision matrix — no alternatives: only-one-way fix already in tree; upstream's test could not be taken as it is, because it expects floor chroma in the picture.
  • AGENTS.md invariant note — no rebase-sensitive invariants beyond the rebase note below.
  • Reproducer / smoke-test command — pasted below under "Reproducer".
  • CHANGELOG fragment — changelog.d/changed/test-video-input-odd-dimensions.md.
  • Rebase note — entry added to docs/rebase-notes.md; the older note that offered upstream's test file is corrected.

Reproducer

CC=gcc meson setup build-san core -Db_lto=false -Denable_cuda=false -Denable_sycl=false \
  -Denable_float=true -Db_sanitize=address,undefined -Dbuildtype=debugoptimized
ninja -C build-san test/test_video_input_odd_dims
ASAN_OPTIONS=detect_leaks=1:halt_on_error=1 UBSAN_OPTIONS=halt_on_error=1 \
  build-san/test/test_video_input_odd_dims
# 4 tests run, 4 passed

Negative controls, run locally and not committed:

  • floor chroma in vmaf_chroma_extent() (upstream's picture geometry): test_odd_420_reads_back fails with the picture does not carry the planes the file stores;
  • floor chroma in yuv_input_set_plane_geometry(): AddressSanitizer stops the test in the buffered reader.

End to end on master c7f28317f: three 19x19 4:2:0 frames in a y4m container give psnr_y 32.360917 / 34.161521 / 33.848468, the values upstream reports with its fix.

Known follow-ups

  • Observed, not changed: the tool's odd-dimension refusal (validate_chroma_alignment() in core/tools/vmaf.cpp) tests frame_w / frame_h. The y4m reader pads those to a multiple of 16, so the refusal fires for raw input only (odd width 19 not allowed for chroma-subsampled format), and odd-sized y4m input is read and scored. Both readers are frame-aligned, so no score is wrong; whether odd y4m input should be refused as well is a maintainer decision. The docs/state.md row records it.
  • The y4m 4:2:2 case is left out on purpose: the y4m reader resamples C422 chroma to the jpeg siting, so its output is not the file's samples.

@github-actions github-actions Bot added the type:test Test-only change label Oct 1, 2026
A subsampled plane occupies ceil(width / dec) * ceil(height / dec)
samples in a raw or y4m file. Upstream Netflix/vmaf's direct readers read
the floor, lose framing on every odd-sized frame and crash; our upstream
PR #1604 fixes that there. The fork is not affected, because VmafPicture
carries ceiling chroma and the buffered reader reads a frame in one
piece, but nothing tested it.

test_video_input_odd_dims derives every sample from its plane, row,
column and frame, reads three frames and then requires the end of the
clip, through video_input_fetch_into_vmaf_picture() and
video_input_fetch_frame(): 20x20 as the control, 19x19, 19x20 and 20x19
4:2:0 in raw and y4m, 19x19 4:2:2 raw, and a clip cut short inside its
last frame, which must return -1.

With floor chroma put back into vmaf_chroma_extent() the test fails with
"the picture does not carry the planes the file stores".

No reader code changes.
@lusoris
lusoris force-pushed the port/upstream-1604-odd-dimension-readback-test branch 2 times, most recently from bd2f5d6 to 0da4827 Compare October 1, 2026 08:23
@lusoris
lusoris merged commit ed469cb into master Oct 1, 2026
42 of 86 checks passed
@lusoris
lusoris deleted the port/upstream-1604-odd-dimension-readback-test branch October 1, 2026 08:23
lusoris added a commit that referenced this pull request Oct 1, 2026
Each of the fork's fifteen open Netflix/vmaf pull requests (#1588 to
against the fork's code, with ASan + UBSan builds where the report is
about memory or undefined behaviour.

The fork needs none of the six commits. It carries the fix of every pull
request except the second revision of #1602, which is a score change left
to the maintainer. Three gaps found on the way are in their own pull
requests: #1662 (a negative threshold shift in the scalar adm_cm), #1663
and #1664 (regression tests for #1590 and #1604).

docs/state.md gains seventeen rows under "Confirmed not-affected", each
with the fork file and function, the test, and what was run; the #1604
row is corrected (the tool refuses odd 4:2:0 dimensions for raw input
only). docs/rebase-notes.md says what a sync can skip and what it must
keep. docs/development/known-upstream-bugs.md lists all fifteen pull
requests with the fork's status and records that they are no longer
updated while upstream does not act on them.
lusoris added a commit that referenced this pull request Oct 1, 2026
* docs(state): record the 2026-10-01 upstream reconciliation

Each of the fork's fifteen open Netflix/vmaf pull requests (#1588 to
against the fork's code, with ASan + UBSan builds where the report is
about memory or undefined behaviour.

The fork needs none of the six commits. It carries the fix of every pull
request except the second revision of #1602, which is a score change left
to the maintainer. Three gaps found on the way are in their own pull
requests: #1662 (a negative threshold shift in the scalar adm_cm), #1663
and #1664 (regression tests for #1590 and #1604).

docs/state.md gains seventeen rows under "Confirmed not-affected", each
with the fork file and function, the test, and what was run; the #1604
row is corrected (the tool refuses odd 4:2:0 dimensions for raw input
only). docs/rebase-notes.md says what a sync can skip and what it must
keep. docs/development/known-upstream-bugs.md lists all fifteen pull
requests with the fork's status and records that they are no longer
updated while upstream does not act on them.

* docs(upstream): keep the reconciliation page to the technical status
lusoris added a commit that referenced this pull request Oct 1, 2026
…s (ADR-1398)

The vmaf CLI formerly refused odd-sized raw YUV 4:2:0 and 4:2:2 inputs with
"odd width/height %d not allowed..." via validate_chroma_alignment()
(ADR-0461), whereas .y4m inputs were accepted because frame_w was padded
to a multiple of 16. The C engine and raw readers handle odd dimensions
with ceiling chroma (vmaf_chroma_extent(), picture_geometry.h, PR #1643,
PR #1664).

Per user decision on 2026-10-01 ("Accept both (Recommended)"), the CLI
now accepts raw YUV inputs with odd dimensions using ceiling chroma,
matching .y4m.

- core/tools/vmaf.cpp: validate_chroma_alignment() returns 0.
- core/tools/test/test_vmaf_raw_odd_dims.sh: add positive (19x19,
  1921x1081, 19x20, 20x19, 19x19 422), boundary (1x1 420/422), and
  negative (file size mismatch exits 2 cleanly) regression tests.
- core/tools/test/test_vmaf_option_dict_ownership.sh: test Case 2 with
  mismatched dimensions instead of odd height to preserve early-exit
  dictionary leak coverage.
- python/test/vmafx_cli_test.py: add unit tests asserting raw YUV scores
  match Y4M bit-identically for odd dimensions, 1x1 boundary, and file
  size error handling.
- docs: document odd dimensions support in docs/usage/cli.md and
  core/tools/AGENTS.md, record ADR-1398, update docs/state.md,
  docs/rebase-notes.md, changelog fragment, and citations registry.

Closes T-CLI-RAW-ODD-DIMENSIONS-REFUSED-2026-10-01.
lusoris added a commit that referenced this pull request Oct 1, 2026
…s (ADR-1398) (#1672)

* fix(tools): accept odd dimensions for raw YUV chroma-subsampled inputs (ADR-1398)

The vmaf CLI formerly refused odd-sized raw YUV 4:2:0 and 4:2:2 inputs with
"odd width/height %d not allowed..." via validate_chroma_alignment()
(ADR-0461), whereas .y4m inputs were accepted because frame_w was padded
to a multiple of 16. The C engine and raw readers handle odd dimensions
with ceiling chroma (vmaf_chroma_extent(), picture_geometry.h, PR #1643,
PR #1664).

Per user decision on 2026-10-01 ("Accept both (Recommended)"), the CLI
now accepts raw YUV inputs with odd dimensions using ceiling chroma,
matching .y4m.

- core/tools/vmaf.cpp: validate_chroma_alignment() returns 0.
- core/tools/test/test_vmaf_raw_odd_dims.sh: add positive (19x19,
  1921x1081, 19x20, 20x19, 19x19 422), boundary (1x1 420/422), and
  negative (file size mismatch exits 2 cleanly) regression tests.
- core/tools/test/test_vmaf_option_dict_ownership.sh: test Case 2 with
  mismatched dimensions instead of odd height to preserve early-exit
  dictionary leak coverage.
- python/test/vmafx_cli_test.py: add unit tests asserting raw YUV scores
  match Y4M bit-identically for odd dimensions, 1x1 boundary, and file
  size error handling.
- docs: document odd dimensions support in docs/usage/cli.md and
  core/tools/AGENTS.md, record ADR-1398, update docs/state.md,
  docs/rebase-notes.md, changelog fragment, and citations registry.

Closes T-CLI-RAW-ODD-DIMENSIONS-REFUSED-2026-10-01.

* docs: regenerate the indexes and the citation map after rebasing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:test Test-only change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant