HDR full-reference metrics: HDR-SSIM and HDR-MS-SSIM via a local nonlinearity

[lusoris]

Summary. Add an HDR full-reference family: HDR-SSIM and HDR-MS-SSIM first (per-pixel nonlinearity, then the existing float_ssim / float_ms_ssim), HDR-VMAF later.

Why. HDR content is scored today with SDR models. An open research implementation of the family exists (HDRMAX, MIT-style licence with a citation requirement). The nonlinearity is a per-pixel transform, so the SSIM variants are a small addition on top of shipped extractors.

Scope.

• The local expansive nonlinearity as a transform stage, on CPU first.
• HDR-SSIM and HDR-MS-SSIM via the existing extractors.
• Licence and citation notice recorded.
• Docs state "research metric, not a standard".
• HDR-VMAF: separate item, needs its own features and a retrained model; training only after RC8 evidence is accepted (ADR-1421). No claim of an HDR VMAF model before then.

Acceptance (failing first).

• The transform reproduces the reference's values on three clips, with the tolerance named in an ADR. Fails first because the transform is missing.
• Negative test: an SDR-range input is handled as documented, not silently scored.

Evidence. utlive/HDRMAX (Python, MIT-style plus citation clause). Related: #2145, #1247, #1248. Candidate: RC5 (#1724, ADR-1868); the 1.1 epic #2148 is related, not the parent. Milestone is 1.0.0 for HDR-SSIM and HDR-MS-SSIM; HDR-VMAF belongs to 2.0 and is split out when this is scheduled.

Plan (2026-10-07)

• Build on the ported conversion pipeline (colorimetry flags, conversion_target, zimg): the HDRMAX non-linearity is a transform stage after conversion. The device-side question is Colour-conversion pipeline on the device: define exactness for the zimg-based conversion_target path (RC5) #2462.
• HDR-VMAF is split out of this issue when scheduled (post-1.0 per the text above; no HDR VMAF model claim before RC9 evidence).
• Acceptance (failing first): transform values equal the reference on three clips within the tolerance named in an ADR; SDR-range input handled as documented. Depends on the zimg pin row and Colour-conversion pipeline on the device: define exactness for the zimg-based conversion_target path (RC5) #2462. Size L.

[lusoris]

Upstream is building HDR-VMAF. On 2026-10-05 Netflix/vmaf merged colorimetry CLI flags (#1671), a per-model conversion_target (#1672), conversion of pictures to that target through zimg (#1673) and approximate gamma (#1677); on 2026-10-03 Netflix/vmaf_resource added 576x324 HDR dock fixtures. The fork is porting the batch now. Build this issue's HDR work on that conversion pipeline (and its device-side twin in RC5) instead of a separate one.

[lusoris]

Research radar, first sweep 2026-10-07 (ADR-2171).

HDRMAX (utlive/HDRMAX, Python with retrained models, last push 2025-09-25) is the reference for the HDR-SSIM, HDR-MS-SSIM and HDR-VMAF family. Class: metric and oracle. Licence class: permissive, an MIT text plus a clause asking users to cite three listed papers (a notice to carry if code is ported). Patent note: not checked. HDR-SSIM and HDR-MS-SSIM stay "per-pixel nonlinearity plus the existing float_ssim / float_ms_ssim"; HDR-VMAF needs its own features and is out of this issue.

VMAFx shape if adopted (triage procedure):

• one CPU reference extractor written from the paper, with no code copied from the original (where a permissively licensed implementation exists, porting it directly with its notice is the alternative, and the two routes are compared first);
• GPU twins (CUDA, SYCL, HIP) and SIMD paths under the exact-twin contract: bit-identical to the CPU extractor, or a measured tolerance recorded in an ADR;
• conformance measured against the original implementation, run locally as an oracle and never vendored or shipped (the RC7 method, Reference conformance: prove every extractor against its original implementation; reference-exact default with a Netflix compat mode (RC7) #2286);
• a patent check before work starts, recorded here;
• a speed comparison against the original as an RC7 / 1.x benchmark row; the aim is a faster and more accurate implementation, and nothing is claimed before it is measured.