Repository navigation
fix(build): include C++ standard headers outside extern "C" so macOS libc++ builds compile - #2596
Merged
Merged
Conversation
… on the VMAFx API (RC4 WP6) (#2303) * feat(api): split libvmaf into libvmafx.so.1 and a compat libvmaf.so.3 on the VMAFx API (RC4 WP6, ADR-2094) libvmafx.so.1 now holds the engine and the VMAFx API and exports vmafx_ symbols only. libvmaf.so.3 defines the libvmaf functions on exported vmafx_ symbols and links nothing else, so a libvmaf function the new API cannot express fails the link (ADR-1852 decision D3). - core/api/vmafx.toml lists all 107 libvmaf functions as [[compat]] entries: generated shims and glue, hand-written ones in core/src/compat/libvmaf/, and declared engine exceptions for the CUDA and SYCL functions (HIP and Metal while built) until their WP3 lanes land. - The VMAFx API gains what the compat library needs (ABI 0.1.6): the libvmaf bridge for pictures, models and model sets, context-owned preallocated frames, perceptual side data, the context backend, frame converters, the tiny-AI and MCP functions, and vmafx_backend_name(). - Every engine translation unit compiles with the generated engine_names_gen.h, which names the engine's own libvmaf bodies vmaf_engine_<stem>; the WP2 forwarders are gone. - test_compat_conformance runs every scenario through the old bodies and the compat library and requires equal traces (every score as %a) and a call of every compat function; planted divergence and coverage defects fail. check_exported_symbols checks both libraries against generated lists, and test_compat_library_gates shows an engine symbol does not link. The Netflix golden gate passes through the compat library: 280 passed, 3 skipped, assertions untouched. * feat(api): opt-in libvmaf deprecation, upstream consumer conformance and the migration guide (RC4 WP6) Every libvmaf declaration now names its VMAFx successor, upstream FFmpeg and GStreamer are checked against the compat library, and the split is documented for users and packagers. - core/include/libvmaf/*.h: each exported function carries VMAF_DEPRECATED("use <successor>"), a compiler warning only when the consumer defines VMAF_ENABLE_DEPRECATION_WARNINGS (opt-in in 1.0, ADR-1852 decision D7). test_libvmaf_deprecation holds every marker to the target core/api/vmafx.toml names, shows the opt-in warning, and shows that a default build and the library itself compile clean under -Werror. The public-API Doxyfile strips the marker as a function-like macro. - scripts/ci/upstream-ffmpeg-compat.sh and upstream-gstreamer-compat.sh build unpatched FFmpeg (FFMPEG_TAG) and the GStreamer vmaf element (GST_PLUGINS_BAD_VERSION, a new Renovate git-tags manager) against an installed libvmaf and compare their scores with the vmaf command line and a reference library as exact text. The Upstream Consumers workflow runs both (#2237). - The compat tensor records of vmaf_dnn_session_run() cross as member-wise copies instead of a pointer cast between distinct struct types, in the compat library and in libvmafx. vmafx_frame_convert() refuses a NULL destination itself if the engine ever accepts one. The conformance test records every return value it used to discard, reads its plant through the environment snapshot (ADR-0488), and covers a real tiny-AI session. - The MCP CI lane also runs the compat conformance and export checks, the only lane whose build has the vmaf_mcp_* compat functions. - Images, Go builds and the CLI artifact take libvmafx.so* next to libvmaf.so*; CGO_LDFLAGS link -lvmaf -lvmafx. - Docs: "Migrating from libvmaf.h" with the deliberate differences, the generated migration table docs/api/vmafx/compat.md, the upstream consumer guide, a changelog fragment, a rebase note and a docs/state.md row for the backend exceptions and the release staging WP12 owns. * fix(api): assert the compat outputs, give each conformance run its own report, plan the two newer libvmaf functions (RC4 WP6) The pre-push assertion-density gate and a parallel test run found two gaps in the compat layer; this closes them and records how the two libvmaf functions newer than the RC4 base map onto the VMAFx API. - The generated score shims assert what a successful call guarantees (`post` in core/api/vmafx.toml: the record's index is the one asked for), and vmaf_picture_convert(), vmafx_frame_from_picture() and vmafx_context_acquire_frame() assert the frame and picture they hand on. - The three conformance tests (plain and the two planted defects) ran at once and shared one report file, so a run could read another's XML as its JSON. Each run now writes its own file, named after its planted defect. - scripts/ci/upstream_consumer_scores.py gives its dict its type arguments (mypy type-arg). - ADR-2094, docs/state.md and the compat agent page record that master's vmaf_set_sample_range_check_enabled() (#2221) and #2300's vmaf_set_input_colorimetry() (ADR-2093) need compat entries when the RC4 chain moves onto master, colorimetry on the colour VMAFx frames carry with a context default; test_libvmaf_deprecation refuses either one missing. * docs(adr): accept ADR-2094 with the maintainer's answer on engine names The maintainer accepted ADR-2094 by popup on 2026-10-06, as recommended: the engine's libvmaf bodies get their engine names from the generated forced header (core/src/vmafx/engine_names_gen.h), not from renaming them in the source or from partial linking. The answer is cited in the ADR's References; the status, the deciders and the index row move to Accepted. * fix(api): give the compat library the two newer libvmaf functions on frame colour and a context option (RC4 WP6) On the chain restacked onto master the vmaf command line did not link: master's vmaf_set_sample_range_check_enabled() (#2221) and vmaf_set_input_colorimetry() (#2300) had no compat entry, and libvmafx.so.1 hides the engine's bodies. Both are now compat functions on the VMAFx API, as ADR-2094 planned. - vmaf_set_sample_range_check_enabled() sets the new context option check_sample_range of vmafx_context_set_option() (ADR-1918). - vmaf_set_input_colorimetry() calls the new vmafx_context_set_default_color() and keeps no state of its own. - VmafxFrameDesc gains color, a VmafxColor appended at its end (ABI 0.1.6, the patch this pull request already raises). Host, wrapped, pool and preallocated frames carry the colour of their desc; a frame whose colour is all UNKNOWN takes the context default. - vmafx_submit() hands each pair's colour to the engine's conversion state (vmaf_engine_set_pair_colorimetry()). The same colour is accepted at any time; another one after the first converted pair is VMAFX_E_BUSY, as libvmaf's -EBUSY, and the pair is not counted. - Both functions carry their VMAF_DEPRECATED marker and a conformance scenario (sample_range; colorimetry with a conversion_target model). - The Python emitter converts a record with one struct of numbers back to C, so FrameDesc keeps to_c() now that it holds a Color. - The lane's regenerated binding and master's ten newer tests linked through vmaf_test_link are part of this commit. - clang-tidy (cpu lane, container) on the 30 translation units this pull request touches: 0 findings after guarding the port write-back of vmaf_mcp_start_sse() against a NULL config (the analyzer's null dereference; a NULL config never reaches it) and splitting a test. The cpu baseline, the measured-source list and the ADR citation registry are updated with it; the shared vt_psnr_context() test helper replaces a second copy. Tests: test_vmafx_frame_input (new), test_vmafx_context, test_vmafx_python_binding, the conformance scenarios and the generator tests fail without the change; planted defects (desc minimum of the grown size, frames without their colour, the default overriding a frame's colour, and in a zimg build a dropped compat colour, a default setter without the engine's -EBUSY, an equal colour treated as a change) are each refused. Docs: the frame colour section of the VMAFx guide, the migration note, the sample range and pictures pages, the changelog fragment, the rebase note and state row T-VMAFX-COMPAT-NEWER-LIBVMAF-FUNCTIONS-2026-10-07. * ci(api): give each master push of the upstream consumer workflow its own concurrency group Master's concurrency contract (ADR-1673) refused the Upstream Consumers workflow this pull request adds: its group was shared between master pushes with cancel-in-progress on, so a later push could cancel an earlier master run. The group now names the commit on master and the ref elsewhere, as the other workflows do; test_master_concurrency_contract passes again (it failed on the previous group). * test(docker): pin both SONAME chains and pkg-config files the image builders copy after the library split WP6 builds libvmafx.so.1 next to the compat libvmaf.so.3 and the Go and node image builders copy both chains and both .pc files; the runtime contract test still pinned the libvmaf-only find command and failed on the split. * ci(api): run the upstream consumer workflow at the full CI tier Master's CI tiers (ADR-2169) route every pull-request workflow through ci-tier.yml; the routing contract test refused upstream-consumers.yml, which ran its 45-minute FFmpeg and GStreamer build on the release pull request too. The job now needs the tier job and runs at the full tier, like the FFmpeg lanes; the guide and the workflow table say so. * fix(api): clear the caller's pictures once vmaf_read_pictures() owns them in a CUDA build A CUDA build released host translations that are struct copies of the caller's pictures and left the caller's VmafPicture structs pointing at released storage; a CPU build clears them through vmaf_picture_unref(). The compat library (ADR-2094) clears them, so test_compat_conformance failed in every CUDA build. vmaf_engine_read_pictures() now clears both structs once the context owns the pictures, in every build; the body moves to read_pictures_owned(). * test(api): read the read-pictures body where it now lives in the GPU fallback contract vmaf_engine_read_pictures() wraps read_pictures_owned() since it clears the caller's pictures; the contract that fallbacks resolve before the CUDA translation reads the helper's body. * fix(api): keep the Windows-only SYCL importer out of the ELF legacy version script core/src/vmafx_legacy_sycl.map named vmaf_sycl_import_d3d11_surface, which only _WIN32 builds define, so every Linux SYCL link failed (--no-undefined-version: undefined version VMAF_LEGACY_SYCL). [[compat]] gains platform = "windows": the legacy map omits the name, its compat-list row ends in @windows and check_exported_symbols.py expects it only on Windows. The generator refuses another platform; both cases are tests that fail on the previous head. * build(test): link the Rust twin registry test through vmaf_test_link after the library split WP6 (ADR-2094) removed libvmaf.get_static_lib(): libvmaf is the compat library, and white-box tests link the engine through vmaf_test_link. test_rust_twin_registry (framework #2086, on master) still asked for the static libvmaf, so meson setup failed with -Denable_rust_features=true. The citation map loses the compat_libvmaf_gen.c sites the split removed and gains the sites of the restack. * docs(api): move the rebase notes to a fragment and leave the rendered files to the landing render (ADR-2197) * build(test): link the Rust predictor and CAMBI kernel tests through vmaf_test_link test_predict_rust_ops, test_rust_predict and test_rust_cambi_kernels came to master with #2085 and #2090 and link libvmaf.get_static_lib(). After the library split libvmaf.a is the compat library: a white-box test links vmaf_test_link (libvmaf.a with libvmafx.a, or both shared libraries), as every other white-box test of this PR does. * ci(tidy): measure the translation units of WP6 (library split) in the cpu lane The clang-tidy coverage rule on master requires every tracked translation unit to be read by a lane. The new and touched units of this pull request were measured in the dev container (scripts/dev/tidy-lane.sh --write --only ... cpu, clang-tidy 22.1.8): 0 findings, 0 uncited NOLINT; they join the cpu lane's measured sources. * fix(api): state the compat SSE start's invariants as assertions vmaf_mcp_start_sse() reached 20 lines without an assertion, which the pre-push Power-of-10 assertion-density gate (scripts/ci/assertion-density.sh) refuses for fork-added functions. It now asserts the two facts its tail relies on: a successful start had a configuration (a NULL one fails with -EINVAL before), and the bound port fits the libvmaf struct's 16-bit field. The guard on the write stays for release builds. assertion-density: pass; clang-tidy cpu lane on core/src/compat/libvmaf/mcp.c: 0 findings. Signed-off-by: Lusoris <lusoris@proton.me>
…libc++ builds compile (#2596) * fix(build): include C++ standard headers outside extern "C" so macOS libc++ builds compile Every macOS libvmaf build on master fails with "templates must have C++ linkage" (runs 37729977161, 37729977110, 37729977056). feature_collector.h opened extern "C" before including model.h, which includes <climits> and <cstddef> in C++ since #2199. libc++ declares templates in those headers; libstdc++ hides the defect on Linux. The headers are now included before the C-linkage block. metadata.h gets its own extern "C" guard because it was C-linked only through feature_collector.h. feature_collector.cpp, output.cpp and test_dict.cpp include headers outside their extern "C" blocks. The state row is T-MACOS-CXX-TEMPLATES-IN-EXTERN-C-2026-10-08. Signed-off-by: Lusoris <lusoris@proton.me>
lusoris
force-pushed
the
fix/master-red-macos-cxx-linkage
branch
from
October 8, 2026 09:54
45ae8db to
23cfafd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every macOS libvmaf build on master fails with
__cstddef/byte.h:57: templates must have C++ linkage. On head499438075this coversFFmpeg macOS clang [Build vmaf](run 37729977161),build.yml macOS Clang+Metal [Build libvmaf (macOS)](run 37729977110) and the matrixmacOS clang/macOS clang+DNN [Build libvmaf](run 37729977056).core/src/feature/feature_collector.h:24openedextern "C"before#include "model.h", and since #2199 (5af6f00f9)model.h:24-25includes<climits>and<cstddef>when compiled as C++. libc++ declares templates in those headers, so every C++ translation unit that includesfeature_collector.hfails. libstdc++ hides the defect on Linux, where the merge train runs.Headers are now included outside C-linkage blocks:
feature_collector.hincludes its headers beforeextern "C".metadata.h(Netflix BSD file, licence unchanged) carries its ownextern "C"guard. It declaresvmaf_register_metadata_handlerand was C-linked only throughfeature_collector.h.feature_collector.cpp,output.cppandcore/test/test_dict.cppinclude headers outside theirextern "C"blocks.Type
fix— bug fixChecklist
git commit -s; fix a branch withgit rebase --signoff origin/master). See DCO sign-off.make format && make lintis green locally: clang-format is clean on the changed files.python3 scripts/ci/run_meson_test.py -- -C build --suite=fastreports 390 OK, 0 failures and 2 skipped./cross-backend-diffand the worst ULP is ≤ 2. (no SIMD/GPU path touched).c/.cpp/.cu/.h/.hpp, it has the appropriate license header (seeCONTRIBUTING.md). (none added)!orBREAKING CHANGE:and the migration path is documented below. (no ABI change: same symbols, same C linkage)docs/adr/_index_fragments/<NNNN-slug>.mdand nothing else is touched for the index. (no ADR)Bug-status hygiene (ADR-0165)
docs/state.mdupdated in this PR with a row under Recently closed:T-MACOS-CXX-TEMPLATES-IN-EXTERN-C-2026-10-08.Netflix golden-data gate (ADR-0024)
assertAlmostEqual(...)score in the Netflix golden Python tests.Deep-dive deliverables (ADR-0108)
AGENTS.mdinvariant note — no rebase-sensitive invariants; the comment infeature_collector.hstates the rule where it applies.changelog.d/fixed/macos-libcxx-extern-c-headers.md.metadata.hgains only a guard that upstream lacks; the include move is fork-local.Reproducer
This is macOS-only. It was verified by reading the job log and by a local check of the same construct on Linux. The probe runs every compile command of the build with an include directory whose wrapper headers declare a template and then
#include_nextthe real standard header, so any standard header included under C linkage fails as it does with libc++. It finds 13 offending translation units onorigin/master, including thefeature_collector.h:24chain, and 0 with this PR. The full Linux build and the fast suite pass.meson setup build core && ninja -C build python3 scripts/ci/run_meson_test.py -- -C build --suite=fastKnown follow-ups
The failing logs show only this error, but Ninja stops at the first failed targets. A later macOS-only error, if there is one, will show up on this PR's macOS legs.
macOS Metal [Configure]fails earlier, in Meson's C++ standard-library probe, and is not addressed here.