Skip to content

feature/adm: clip the integer AIM score at 1 like the float extractor - #1634

Closed
lusoris wants to merge 1 commit into
Netflix:masterfrom
VMAFx:fix/adm-integer-aim-clip
Closed

lusoris wants to merge 1 commit into
Netflix:masterfrom
VMAFx:fix/adm-integer-aim-clip

Conversation

@lusoris

@lusoris lusoris commented Oct 1, 2026 •

Copy link
Copy Markdown

The integer ADM extractor can report an AIM above 1; the float extractor clips the same ratio to 1. This PR clips the integer one. It is a behaviour change, so the options are spelled out below.

What is inconsistent

/* libvmaf/src/feature/integer_adm.c:3007 */
*score_aim = aim_num / den;

/* libvmaf/src/feature/adm.c:323 */
// normalize AIM score by the DLM denominator and clip values larger than 1
*score_aim = MIN(aim_num / aim_den, 1.0f);

The integer line came in with 966be8d5 (2026-04-17) and the float line, with the clip and its comment, with 4dcc2f7c (2026-04-27). AIM is the additive impairment that survives contrast masking, divided by the DLM denominator, which is the reference's own detail. When the distorted picture has far more detail than the reference, the ratio exceeds 1.

Reproducer

Master 6ec23e8f2 (reproduced there; the code is unchanged in 9e48141), release build, default options, default dispatch. The pictures are 8-bit 4:2:0 and generated, so anyone can rebuild them:

  • reference: every sample 128, at 64x64 and at 576x324;
  • patches: the reference with a 4x2 block of luma samples 255 0 0 0 (two rows of 255 0 0 0) every 16 samples from (1, 1), at both sizes;
  • noise: the 576x324 reference with uniform noise in [-amp, amp] added to luma (xorshift32 with seed 1, 128 + state % (2 * amp + 1) - amp), for amp 8 and 24.
vmaf -r flat_ref.yuv -d distorted.yuv -w 64 -h 64 -p 420 -b 8 \
     --feature adm --feature float_adm --no_prediction -q --json -o out.json
distorted picture integer_aim master aim (float) master integer_adm3 master adm3 (float) master integer_aim, integer_adm3 this branch
64x64 patches 3.175585 1.000000 0.000000 0.500000 1.000000, 0.500000
576x324 patches 2.602848 1.000000 0.000000 0.500000 1.000000, 0.506536
576x324 noise +-24 1.268618 1.000000 0.365691 0.500000 1.000000, 0.500000
576x324 noise +-8 0.435699 0.435678 0.782151 0.782161 0.435699, 0.782151

The integer adm3 is max(adm2 * w + (1 - aim) * (1 - w), adm_min_val), so an AIM above 1 lowers it. The 0.506536 is 0.5 * adm2 with an integer adm2 of 1.013; that is the scale 0 centre-tap wrap that #1602 fixes (with #1602 applied the same picture gives an adm2 of 1.0), not an AIM effect.

With the model the shipped vmaf_v1.0.16 files set (adm_dlm_weight 0.7, adm_min_val 0.5) on the noise pictures, vmaf_v1.0.16_3d0h:

noise model AIM model integer_adm3 master model integer_adm3 this branch VMAF master VMAF this branch
+-18 1.017921 0.694624 0.700000 0.000000 0.000000
+-24 1.337371 0.598789 0.700000 0.000000 0.000000

Fix

*score_aim = MIN(aim_num / den, 1.0f); in integer_compute_adm(). That is the only place the integer AIM is concluded. libvmaf/src/feature/cuda has no AIM pass (integer_adm_cuda never computes integer_aim), and nothing else reads the score.

What changes, and where

  • Changes: integer_aim, and everything computed from it: integer_adm3 (VMAF_integer_feature_adm3_score) and the VMAF of every model that reads it. Those are the eight vmaf_v1.0.16 and vmaf_v1.0.16_hfr models (model/vmaf_v1.0.16/*.json, model/vmaf_v1.0.16_hfr/*.json); no other file under model/ mentions adm3 or an AIM feature. vmaf_v0.6.1 and the other shipped models read adm2 only.
  • Inputs: only frames where the unclipped AIM exceeds 1, i.e. a distorted picture with much more detail than its reference (flat or very smooth reference against noise, patches, texture). Every value of 1 or less is unchanged.
  • Python goldens: python/test asserts an integer AIM in 48 places; the largest expected value is 0.026560104166666664 (vmafexec_feature_extractor_test.py:1998). The 49 integer adm3 assertions (0.9423 to 1.0526) are computed with those AIM values. None involves an AIM above 1, so no assertion changes.
  • The three Netflix reference pairs (.engagement/golden_compare.py, default dispatch and --cpumask -1): identical per-frame and pooled output on master and on this branch.
  • VMAF of the four models I scanned (vmaf_v1.0.16_3d0h, _5d0h, _1d5h_2160, _hfr_3d0h) on the 576x324 reference against noise of +-8, 12, 16, 17, 18, 19, 20, 22, 24, 28, 32 and 48 (model AIM up to 2.63): identical before and after in all 48 runs, because the score reaches 0 before the AIM reaches 1 on this family of pictures. This is not exhaustive; a picture with an AIM above 1 and a score above 0 would change.

Alternative

Drop the clip from adm.c instead, so that both extractors return the raw ratio. That changes the float aim and the float adm3 on the same inputs, keeps the shipped models' scores as they are, and makes float_adm and adm report values above 1. The adm.c comment says the clip is deliberate there; nothing says its absence from integer_adm.c is. I think the clip is the intended behaviour, but the models read the integer feature, so it is your call; I can turn this into the other change if you prefer.

Tests

test_integer_adm_aim_clip (new): on the flat reference against noise of +-24, integer AIM is exactly 1 on the SIMD and on the scalar path, equals the float AIM, and integer adm3 with weight 0.7 and floor 0.5 is within 1e-4 of the float one; identical pictures give an AIM of 0; noise of +-8 gives an AIM between 0.4 and 0.5 that matches the float one within 1e-4. On master it fails: integer aim 1.2686181234336971, expected 1. The float comparisons are skipped when float_adm is not built. The test uses the noise picture rather than the patches because the patches also reach the centre-tap wrap that #1602 fixes: on the 64x64 patches master gives a scalar adm2 of 1.0355 against 1.0 on the SIMD paths, and UBSan reports a negative left shift in adm_cm() (integer_adm.c:1680). With #1602 applied the scalar adm2 is 1.0.

Validation

Rebased on master 9e48141 (2026-10-02). x86-64 Linux, GCC 16.2.1, on master 9e48141b.

  • Release build meson test (-Denable_float=true -Denable_checkasm=true): 25/25 on master, 26/26 here (the new test).
  • -Db_sanitize=address,undefined -Db_lto=false build: 22 pass and 3 fail on master, 23 pass and 3 fail here. test_predict and test_pic_preallocation abort on LeakSanitizer reports and checkasm on a heap-buffer-overflow in adm_dwt2_16 on both; this change does not touch them.
  • CUDA: not built; the change is in the CPU extractor and CUDA has no AIM.

The workflow run on this PR needs a maintainer's approval.

integer_compute_adm() normalised the AIM numerator with aim_num / den, while compute_adm() in adm.c returns MIN(aim_num / aim_den, 1.0f). On a picture whose distorted side has much more detail than the reference, the integer extractor therefore reports an AIM above 1 where the float extractor reports exactly 1. A flat grey 64x64 reference against the same picture with isolated 4x2 luma patches gives integer_aim 3.175585 and aim 1.000000; 576x324 patches give 2.602848 and 1.0; flat against uniform noise of +-24 gives 1.268618 and 1.0.

The unclipped value also reaches integer adm3, which VMAF_integer_feature_adm3_score computes as MAX(adm2 * w + (1 - aim) * (1 - w), adm_min_val). With the weight 0.7 and floor 0.5 of the vmaf_v1.0.16 models it sat on the floor where the float extractor gives 0.7. Clip the integer AIM in the one place that concludes it. integer_adm_cuda has no AIM pass and no other extractor reads the score.

This changes integer aim, integer adm3 and the vmaf_v1.0.16 scores only where the unclipped AIM exceeds 1. No Python golden assertion has an integer AIM above 0.02656, and the three Netflix reference pairs score identically. test_integer_adm_aim_clip checks, on a flat reference against noise of +-24 at 576x324, the clipped value (1.268618 before, 1 after) on the SIMD and the scalar path and the agreement of integer and float adm3 with the model's weight and floor, and that identical pictures give an AIM of 0 and noise of +-8 an AIM below 1 that passes unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lusoris
lusoris force-pushed the fix/adm-integer-aim-clip branch from af94fc9 to 530be8e Compare October 2, 2026 18:43
@lusoris

lusoris commented Oct 7, 2026

Copy link
Copy Markdown
Author

Closing this. I proposed the clip to make the integer extractor match the float one, but I could not find a case where it changes a shipped model's output: on every picture I built where integer AIM exceeds 1 (flat reference with patches or noise), the default model's VMAF is already 0 before and after the clip, and the other published models read adm3 unchanged once AIM is below 1.

Since integer AIM feeds adm3, and adm3 feeds the published scores, changing it is a behaviour change that should come from you rather than from a consistency cleanup. The inconsistency stays as it is upstream (integer: unclipped ratio; float: clipped at 1). If you want the clip, it is the one line in the description and I am glad to reopen.

@lusoris lusoris closed this Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant