feature/adm: fix out-of-bounds read at scale 3 for frame dimensions 17 to 32

[lusoris]

dwt2_src_indices_filt() fills the DWT source-index table in three ranges: output 0, the interior, and a mirrored tail that starts at n_half - 2. For an input of 3 or 4 samples n_half is 2, so the tail restarts at output 0 and replaces its { 1, 0, 1, 2 } with { -1, 0, 1, 2 }. adm_dwt2_s123_combined() and its AVX2 and AVX-512 versions then read row -1 and column -1 of their input. Scale 3 sees a 3- or 4-sample input for any frame width or height from 17 to 32. Column -1 is the int32_t in front of the tmp_ref allocation, which AddressSanitizer reports as a heap-buffer-overflow; row -1 lies inside data_buf, so it only shows as a wrong score.

Start the tail at 1 when n_half is 2. The table is unchanged for every larger input. The float implementation (dwt2_src_indices_filt_s()) mirrors index -1 to 1 and is not affected.

The regression test test_adm_dwt2_indices compares every table entry for inputs of 3 to 66 samples with the symmetric extension. On the unpatched tree it fails with n=3 output=0 tap=0: index -1, expected 1.

Reproducer, using the first 2592 bytes of the src01 test clips as three 24x24 frames:

head -c 2592 python/test/resource/yuv/src01_hrc00_576x324.yuv > ref_24x24.yuv
head -c 2592 python/test/resource/yuv/src01_hrc01_576x324.yuv > dis_24x24.yuv
build-sanitize/tools/vmaf -r ref_24x24.yuv -d dis_24x24.yuv -w 24 -h 24 -p 420 -b 8 \
  --feature adm --no_prediction --cpumask -1 -q -o /dev/null
# ==ERROR: AddressSanitizer: heap-buffer-overflow ... READ of size 4
#     #0 adm_dwt2_s123_combined ../libvmaf/src/feature/integer_adm.c:2752
#     #1 integer_compute_adm ../libvmaf/src/feature/integer_adm.c:2948
# ... is located 4 bytes before 384-byte region ... allocated by adm_buffer_alloc ../libvmaf/src/feature/integer_adm.c:3071
# Without --cpumask the same report names adm_dwt2_s123_combined_avx512 (adm_avx512.c:2705).

Over single-frame synthetic 8-bit clips with widths 17 to 34 and heights 17, 18, 24, 31, 32, 33 and 40, each under the default dispatch, --cpumask 16 and --cpumask -1, 336 of 378 runs end in that report on the unpatched tree (every run with a width of 17 to 32) and none with the patch.

Effect on integer_adm_scale3 for the clip above, with float_adm as the reference:

	frame 0	frame 1	frame 2
unpatched	0.925844	0.920199	0.913376
patched	0.986957	0.973522	0.945018
float_adm adm_scale3	0.986972	0.973548	0.944989

Scales 0 to 2 are identical before and after; integer_adm2 follows scale 3.

Rebased on master 9e48141 (2026-10-02).

Validation against upstream 9e48141bd1eb8d2329e09d3744e7c24af53017ca, x86-64 Linux (AVX-512 host), GCC 16.2.1, Meson 1.12.1:

meson setup build-release libvmaf --buildtype=release -Denable_cuda=false -Denable_float=true -Denable_docs=false
ninja -C build-release
meson test -C build-release --print-errorlogs
# 25/25 passed

meson setup build-sanitize libvmaf --buildtype=debug -Db_sanitize=address,undefined -Denable_cuda=false -Denable_float=true -Denable_docs=false
ninja -C build-sanitize
ASAN_OPTIONS=detect_leaks=1:halt_on_error=1 UBSAN_OPTIONS=halt_on_error=1 meson test -C build-sanitize test_adm_dwt2_indices test_feature_extractor --print-errorlogs
# both passed

No golden value changes. The three reference pairs (src01_hrc00/src01_hrc01 at 576x324, and checkerboard_1920_1080_10_3_0_0 against _1_0 and _10_0) give identical per-frame and pooled JSON output from the unpatched and the patched release build, under the default dispatch and with --cpumask -1; pooled vmaf_v0.6.1 means are 76.667831, 35.068667 and 7.985899 in all cases.

This does not make frames below 17 pixels work. A 16x16 frame still fails after this patch: UBSan reports shift exponent 4294967295 in adm_cm() and the release build crashes. The remaining UBSan reports in the size sweep (shift exponent -1 is negative in get_best15_from32(), adm_avx2.c:1350) are present before and after and are not touched here.