feat(api): implement the VMAFx core API on the engine (RC4 WP2, ADR-1852)

[lusoris]

Summary

RC4 work package 2: the VMAFx core API, implemented on the engine. A program can now score videos through vmafx/*.h alone, and every score equals the libvmaf.h call's bit for bit. WP1 #2187 is on master; this PR's diff against master is its own commits (core API, tidy, full log routing, the rebase fix-ups). It implements design sections 2.2 to 2.6 and the synchronous part of 2.8 of ADR-1852 / Research-2158 in one PR: context, options, errors, models, frames, submission and scores (no stacked series needed).

What the definition gains (core/api/vmafx.toml, ABI 0.1.0 -> 0.1.1, additions only, all in node VMAFX_0.1 per ADR-1897):

• Contexts and logging: full routing (maintainer popup 2026-10-05, "Full routing in RC4"). VmafxContextConfig gains log_callback / log_user (appended; a 0.1.0-sized struct still works). A context with a callback receives every message the library raises for it at or below its level, on the calling thread (a per-thread sink in core/src/log.cpp around every engine call) and on its worker threads (every job captures the submitting call's sink, ThreadDataBatch.log_sink in core/src/libvmaf.c; thread_pool.c unchanged), and nothing of it reaches the process log; it never changes the process level. The float ADM / SSIM / MS-SSIM / motion / VIF error prints went to stdout and now go through vmaf_log(). A model load routes to the callback of its VmafxModelConfig (new log_level / log_callback / log_user). Without a callback a context logs to stderr and sets the level as vmaf_init() does. vmafx_context_set_option (perceptual_weight, perceptual_weight_strength).
• Options and registration. VmafxOptions (vmafx_options_set / free, the libvmaf feature dictionary under its new name), vmafx_context_use_feature, vmafx_context_use_model, vmafx_context_use_model_set, vmafx_context_import_score, vmafx_context_extractor_count, vmafx_feature_resolve (twin lookup of ADR-1359; on a context without a device backend the answer is the CPU extractor), vmafx_context_frame_retention (1, or 2 with an n-2 reader, ADR-1478).
• Models. vmafx_model_load / load_file / override_feature / ref / unref / name / feature_count / feature_name / hash / builtin_next / default_version, and model sets (vmafx_model_set_load / load_file / override_feature / ref / unref / lead / size / hash). The hash is the SHA-256 of the bytes as loaded (built-in == sha256sum model/<file>.json). A context holds a reference to every model and set it uses (ADR-1755); an override is refused with VMAFX_E_BUSY once the model is shared.
• Devices and host frames. VmafxDevice skeleton (CPU only; WP3 fills the backends), vmafx_frame_create_host, vmafx_frame_wrap_host (borrowed planes, release callback, no copy), vmafx_frame_planes, vmafx_frame_ref / unref. A frame reference is one count of the engine picture's counter.
• Submission and scores. vmafx_submit (consumes both references on every path; refuses non-increasing indices and geometry changes naming reference / distorted), vmafx_flush, vmafx_score_frame, vmafx_score_frame_model_set, vmafx_score_pooled, vmafx_feature_score_pooled, vmafx_score_pooled_model_set. VMAFX_PENDING where libvmaf returns -EAGAIN, without an error.
• One frame, many contexts (ADR-1880 consequence 1, PR docs(adr): add the format envelope and device-targeted scoring to the 1.0.0 candidates (ADR-1880) #2185): one frame submitted to two contexts with different ADM viewing distances, one reference each, no copy; each context's scores equal a separate run's, and each frame is released once after both.
• Errors. vmafx_error_subject_kind (VmafxSubjectKind), vmafx_error_function; every failure names its subject; with a NULL error pointer the message reaches the log at ERROR whatever the level. New status VMAFX_E_ABI (-11): an input struct below its introduction size.
• Engine entry points. 14 more libvmaf bodies in core/src/libvmaf.c become vmaf_engine_* (core/src/vmafx/engine.h); the libvmaf names forward to them until WP6 generates them as shims. Return values and behaviour of every libvmaf.h function are unchanged.
• ADR-1906 (Accepted, maintainer popups 2026-10-05: "Full routing in RC4", "Override while unshared (Recommended)") records the semantics above (full log routing, failure delivery, size negotiation, frame and model references, entry points); the calling-thread-only logging option is under Alternatives, rejected.

--abi-check --against-ref origin/rc4/api-generation-prototype: definition is an append-only successor of origin/rc4/api-generation-prototype (95 additions); same against origin/rc4/api-wp1-generator. No break of a prototype entry, so no new minor.

Generator changes (WP1-owned, needed here, filed as request WP1-2): Python records keep to_c() when they gain pointer fields (else ContextConfig with its callback broke the binding), and the generator tests no longer assume the live definition's ABI version or function set. Request WP1-1 asks the generator to emit the struct introduction sizes the C code keeps by hand (core/src/vmafx/internal.h, VMAFX_MIN_*).

Landing (Q-083)

Lands bottom-up per Q-083: v1.0.0-rc.3 is tagged, so RC4 lands through the merge train, one API PR at a time. Its base #2187 is on master; this PR was rebased onto master 18b04cf8b (the base's commits dropped), retargeted to master, and #2213 (WP3 common) follows once it lands. The API docs keep marking the VMAFx API as a preview (docs/api/vmafx/index.md, ABI 0.x) until the rc.4 cut. ADR-1906 is Accepted.

ABI check against master: python3 scripts/codegen/vmafx-api.py --abi-check --against-ref origin/master -> definition is an append-only successor of origin/master (95 additions). Additions only, node VMAFX_0.1, abi_version 0.1.1 (patch bump over master's).

Rebase: conflicts only in generated files (docs/adr/titles.md, ADR by-tag pages, CHANGELOG.md, citations), taken from master and regenerated once at the tip (vmafx-api.py --write, make docs-fragments-write, check-source-adr-citations.py --write); docs/state.md by scripts/dev/resolve-state-md-conflict.py.

Local gate on the rebased stack (CPU, -Db_lto=false, -j4, warnings as errors): build 0 warnings; run_meson_test.py --suite=fast 390 OK, 0 fail, 1 skipped (test_vmafx_api_abi_append_only before master had a definition); codegen tests 61 passed; affected suites: tooling 2210 passed, 0 failed; make test-netflix-golden GOLDEN_NINJA_JOBS=4 280 passed, 3 skipped; preflight.sh --stage msvcism pass; train gates (deliverables, state-md touch and rows, silent revert, praetorctl audit) pass.

Type

• feat — new feature

Checklist

• Commits follow Conventional Commits (the commit-msg hook enforces this).
• make format && make lint is green locally — the commit hooks pass on both commits (clang-format, markdownlint, semgrep, source ADR citations, generated-index freshness, FFmpeg patch stack, HISS audit).
• Unit tests pass: python3 scripts/ci/run_meson_test.py -- -C build --suite=fast --num-processes 4 → 360 OK, 0 fail, 1 skipped (test_vmafx_api_abi_append_only: the merge base with origin/master has no definition yet) on a CPU build (-Db_lto=false).
• If I touched any SIMD/GPU code path, I ran /cross-backend-diff and the worst ULP is ≤ 2. — not applicable: no SIMD/GPU code touched.
• If I touched a feature extractor with SIMD/GPU twins, I either updated every twin or listed the gap under "Known follow-ups" below. — not applicable: no extractor touched.
• If I added a new .c / .cpp / .cu / .h / .hpp, it has the appropriate license header (EUPL-1.2, fork-authored).
• If this is a breaking change, the commit message uses ! or BREAKING CHANGE: and the migration path is documented below. — not a breaking change: additions only; VmafxContextConfig grew at its end and a 0.1.0-sized struct is accepted.
• If this PR adds an ADR, the ADR row lives in docs/adr/_index_fragments/<NNNN-slug>.md and the slug is appended to docs/adr/_index_fragments/_order.txt — ADR-1906: docs/adr/_index_fragments/1906-vmafx-core-api-semantics.md, slug appended, indexes regenerated.

tidy: cpu core/src/vmafx/{context,device,error,frame_host,model,options,register,score,sha256,sized,submit,status_gen,compat_libvmaf_gen}.c, core/src/libvmaf.c, core/src/log.cpp, core/src/model.c, core/src/picture.c, core/src/feature/{adm,ssim,motion,ms_ssim,vif}.c, core/test/test_vmafx_{context,model,frame,score,bitexact,lifetime,sha256,log_routing,api_slice,abi_layout}.c (and through them core/src/log.h, core/src/vmafx/*.h, core/include/vmafx/*.h, core/test/vmafx_test_util.h) 0 findings, 0 uncited NOLINT (dev container, clang-tidy 22.1.8, scripts/dev/tidy-lane.sh --only ... cpu, measured per commit). The lane reports 7 findings in core/include/libvmaf/picture.h that this branch inherits from its base and master fixed in #2186; they go away on the rebase onto master. GPU lanes not measured: no GPU translation unit touched.

scripts/dev/preflight.sh --stage msvcism: pass. praetorctl audit -base origin/rc4/api-wp1-generator: governance gates passed; HISS debt ratchet 6 -> 6, no finding in a touched file.

Bug-status hygiene (ADR-0165)

• docs/state.md updated — no state delta: the two defects WP2 found are master defects already closed on master with their own rows: T-MODEL-SET-SCORE-NOT-IDEMPOTENT-2026-10-05 (fix(model): read a model collection's stored score instead of predicting a frame twice #2206; the rebase dropped this branch's open row and the separate-session workaround in test_vmafx_score) and T-LOG-LEVEL-GLOBAL-DATA-RACE-2026-10-06 (fix(log): make the process log level atomic so contexts on several threads do not race #2207). This branch keeps only the VMAFx part of the latter: the engine's init no longer sets the level, vmafx_context_create() sets it for a context without a callback only.

Netflix golden-data gate (ADR-0024)

• I did not modify any assertAlmostEqual(...) score in the Netflix golden Python tests.
• If I believe a golden value must change, I have explained why below — not applicable.

Golden gate on this branch (make test-netflix-golden with GOLDEN_NINJA_JOBS=4, core/build-golden built with gcc; the pytest step ran with the repository venv because the worktree venv the target bootstraps carries only meson and ninja): 280 passed, 3 skipped on b19003049; earlier heads too.

Cross-backend numerical results

Not applicable: no extractor or kernel changed. VMAFx against libvmaf on the same build (core/test/test_vmafx_bitexact.c): 8 cases (Netflix 576x324 pair 48 frames, 1080p checkerboard 1 px and 10 px 3 frames, 10-bit sparks 480x270 5 frames, each with vmaf_v1.0.16_3d0h and vmaf_v0.6.1), 2912 values compared bit for bit (every feature in the collector at every frame, every feature pooled with all 8 methods, the model score per frame and pooled with all 8 methods): 0 differ.

Performance (if perf or feat)

No engine path changed; the VMAFx layer adds argument checks and one thread-local pointer swap per engine call. Borrowed host frames remove the copy into vmaf_picture_alloc pictures that libvmaf.h callers make.

Deep-dive deliverables (ADR-0108)

• Research digest — no digest needed: implements Research-2158 sections 2.2 to 2.6 and 2.8 (docs(adr): accept the VMAFx API redesign and VMAFx filter names for RC4 (ADR-1852) #2176).
• Decision matrix — docs/adr/1906-vmafx-core-api-semantics.md ## Alternatives considered (logging scope, struct sizes, frame reference counting, model immutability).
• AGENTS.md invariant note — core/src/AGENTS.d/vmafx-api.md (index regenerated): core API files, failure and logging rules, size table, frame and model references, the model.c / dict.cpp bodies WP6 must rename, the engine defects found.
• Reproducer / smoke-test command — below.
• CHANGELOG fragment — changelog.d/added/api-core-context-model-frame-score.md.
• Rebase note — docs/rebase-notes.md, "VMAFx core API: engine entry points, per-thread log sink, shared picture helpers".

User documentation: docs/api/vmafx/index.md rewritten for the core API (a complete scoring program, contexts and logging, registration, models, frames, scores, the rules every call follows); the generated reference pages cover every function, struct and constant; docs/development/api-generation.md lists the new tests.

Reproducer

python3 scripts/codegen/vmafx-api.py --check
python3 scripts/codegen/vmafx-api.py --abi-check --against-ref origin/rc4/api-generation-prototype
meson setup build core -Denable_cuda=false -Denable_sycl=false -Denable_hip=false -Db_lto=false
ninja -C build -j4
python3 scripts/ci/run_meson_test.py -- -C build test_vmafx_context test_vmafx_model \
    test_vmafx_frame test_vmafx_score test_vmafx_bitexact test_vmafx_lifetime \
    test_vmafx_sha256 test_vmafx_api_slice test_vmafx_python_binding test_vmafx_abi_layout
# lifetimes under ASan / UBSan
meson setup build-asan core -Db_sanitize=address,undefined -Db_lundef=false -Db_lto=false
ninja -C build-asan -j4 test/test_vmafx_lifetime test/test_vmafx_frame test/test_vmafx_bitexact
python3 scripts/ci/run_meson_test.py -- -C build-asan test_vmafx_lifetime test_vmafx_frame test_vmafx_bitexact

test_vmafx_bitexact needs the fixtures in python/test/resource/yuv (skipped with 77 without them).

Tests

Test	Covers
test_vmafx_context (static link)	errors and their accessors, unclaimed failures at ERROR, engine messages routed to the callback, level filtering, process level untouched by a callback context, struct sizes (0.1.0-sized, newer and too-short config; older, newer and too-short outputs), options, context options, registration, imported scores, feature resolution (device twin on GPU builds), retention
test_vmafx_model	built-in and file loads, hash == sha256sum, every refusal (unknown version, set as model, pkl, missing file, directory, bad flags, short config), override only while unshared, references, built-in list, model sets
test_vmafx_frame	CPU device, allocated and borrowed frames, odd 4:2:0 geometry, every refusal, consumption on every submit path, geometry and index refusals, n-2 retention, one frame in two contexts
test_vmafx_score	per-frame and pooled scores against a separate libvmaf session for models, features and sets (all 8 pool methods), VMAFX_PENDING without an error, refusals
test_vmafx_bitexact (static link)	the 2912-value comparison above
test_vmafx_lifetime (Linux static, --wrap=vmaf_thread_pool_destroy)	model released before destroy, failed destroy retried with the context and its set still valid, planes freed by their release callback across n-2 retention
test_vmafx_sha256	FIPS 180-4 examples, padding boundaries, one million a
test_vmafx_python_binding	adds imported scores, pooling, context options, built-in models through the generated binding
test_vmafx_log_routing (Linux, static link)	worker-thread messages with n_threads = 4 reach the callback and nothing reaches stdout / stderr; two contexts on two threads in a deterministic interleaving (A logs again while B's sink is active) and concurrently with worker threads never get each other's lines; a context without a callback logs to stderr at its level; a model load routes to its config's callback
test_engine_log_routing_contract.py	no direct stdout / stderr write in core/src outside a declared exception table (exact counts; scanner self-tests plant each write form)

ASan + UBSan (-Db_sanitize=address,undefined): all nine VMAFx test programs clean. TSan (-Db_sanitize=thread): with #2207's atomic level applied, test_vmafx_log_routing 0 reports in 8 runs, test_vmafx_context and test_vmafx_lifetime clean; without it the concurrent case reports the master race (3 of 3 runs) until this branch rebases onto #2207.

Gates shown failing on planted defects (measured on this branch)

Planted defect	Test	Result
log sink not installed around engine calls	test_vmafx_context	test_engine_messages_reach_callback fails
callback context sets the process log level	test_vmafx_context	test_callback_context_keeps_process_level fails
input read past the caller's struct_size	test_vmafx_context	test_config_older_struct crashes (garbage callback pointer)
hash of the path instead of the bytes	test_vmafx_model	test_hash_equals_file_digest fails
override allowed on a shared model	test_vmafx_model	test_override_feature fails
refused submit keeps the caller's references	test_vmafx_frame	test_submit_and_flush fails (release count)
submit takes a reference instead of consuming the caller's	test_vmafx_frame	test_submit_and_flush fails (release count)
pool method mapped wrong (MEAN -> MEDIAN)	test_vmafx_score, test_vmafx_bitexact	both fail
VMAFX_PENDING reported as an error	test_vmafx_score	test_pending_is_not_an_error fails
borrowed chroma planes off by one row	test_vmafx_bitexact	fails
SHA-256 length field wrong	test_vmafx_sha256	fails
context takes no reference to a model it uses	test_vmafx_lifetime (ASan)	heap-use-after-free
context releases its models before the engine close	test_vmafx_lifetime (ASan)	heap-use-after-free
no log sink installed around worker jobs	test_vmafx_log_routing	test_worker_messages_reach_callback fails
one process-wide sink instead of one per thread	test_vmafx_log_routing	test_two_contexts_interleaved fails
model load without its config's sink	test_vmafx_log_routing	test_model_load_messages_routed fails
a context without a callback leaves the process level	test_vmafx_log_routing	test_plain_context_logs_to_process_log fails
a printf planted in feature/motion.c	test_engine_log_routing_contract.py	fails naming the file
plain-global process log level (master; fixed by #2207)	TSan on test_vmafx_log_routing	data race reported, 3 of 3 runs

Known follow-ups

• Engine defect T-MODEL-SET-SCORE-NOT-IDEMPOTENT-2026-10-05: fixed on master by fix(model): read a model collection's stored score instead of predicting a frame twice #2206 (in the merge train); on the rebase this branch drops its open row and the separate-session workaround in test_vmafx_score.
• Log routing exceptions (test_engine_log_routing_contract.py, each with reason and expiry): CUDA / SYCL / Metal device prints (WP3 lanes; no VMAFx context attaches a device before WP3), the libsvm model-text parser error at model load (needs the libsvm link change, WP6), vmaf_write_output() (WP5 vmafx_report_write), vifdiff() (no extractor calls it). libvmaf.h calls on a bridged handle keep the process log (compat).
• ADM viewing distance: the default CSF refuses adm_norm_view_dist below 3 (-EINVAL at the first submit, ADR-1191 / ADR-1325 table range); device-targeted scoring for close viewing (RC5) will need it. The VMAFx error names the frame index, not the extractor, because the engine does not report which extractor failed (WP5's producer record can).
• WP3: device backends, vmafx_context_attach_device, imports, fences and admission extend core/src/vmafx/device.c / frames; the frame reference rule (ADR-1906 item 4) is the base.
• WP6: generate the 14 forwarding libvmaf functions as shims, and rename the core/src/model.c / core/src/dict.cpp bodies the core API calls first (core/src/AGENTS.d/vmafx-api.md).
• WP1 requests: WP1-1 (introduction sizes), WP1-2 (the two generator fixes in this PR).
• C++ -Wextra -Werror missing-field-initializers on *_INIT (WP1 note): not hit, the WP2 tests are C.