Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6544,6 +6544,15 @@ The `--restore-tracked` step that drops unusable restored CI fixtures no longer
`core.eol=lf` as well.


- **The native Windows CUDA build compiles again, with one MSVC toolset.**
NVCC's host compiler was the first `cl.exe` a recursive search of the
Visual Studio install found, an older toolset (14.29 in Visual Studio 2026)
than the one building the rest of the library; its standard library cannot
compile the C++20 `<numbers>` header the CUDA `ciede` kernel uses. NVCC now
uses the build's own `cl.exe` when the build compiles with MSVC, otherwise the
newest installed toolset (`docs/getting-started/building-on-windows.md`).


- **Windows MSVC+SYCL compiles the exact-arithmetic SYCL headers again, and two
contract tests pass on Windows.** A `max()` macro from `<windows.h>` broke
`std::numeric_limits<float>::max()` in a SYCL header, and two Python tests
Expand Down
7 changes: 7 additions & 0 deletions changelog.d/fixed/windows-nvcc-ccbin-build-msvc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
- **The native Windows CUDA build compiles again, with one MSVC toolset.**
NVCC's host compiler was the first `cl.exe` a recursive search of the
Visual Studio install found, an older toolset (14.29 in Visual Studio 2026)
than the one building the rest of the library; its standard library cannot
compile the C++20 `<numbers>` header the CUDA `ciede` kernel uses. NVCC now
uses the build's own `cl.exe` when the build compiles with MSVC, otherwise the
newest installed toolset (`docs/getting-started/building-on-windows.md`).
18 changes: 13 additions & 5 deletions core/src/AGENTS.d/build-and-compiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,25 @@
paths:
- core/src/meson.build
- core/test/check_exported_symbols.py
invariant: Windows CUDA discovers compiler on vswhere and PATH; C++ targets take vmaf_cppflags_common with hidden inlines.
invariant: Windows nvcc host = build MSVC, else newest vswhere toolset, else PATH; C++ targets take vmaf_cppflags_common.
---
<!-- markdownlint-disable MD013 -->
# Build system, compiler discovery, and C++ profile invariants

## Windows CUDA compiler discovery

`meson.build` must assign `cl_path` on both `vswhere` and `PATH` discovery
routes. NVCC's `-ccbin` and MSVC include discovery consume that same path.
Keep configure regression in `../test/test_windows_cuda_compiler_discovery.py`
when rebasing Windows discovery block from Netflix PR #1472.
`meson.build` must assign `cl_path` on every discovery route. NVCC's `-ccbin`
and MSVC include discovery consume that same path. Order: the build's own
MSVC (`nvcc_build_msvc`, from `cxx` when its id is `msvc`), then the newest
toolset under the latest `vswhere` install (sorted by `[version]`, never the
first `cl.exe` of a recursive walk: that was the v142 toolset 14.29 of VS 18,
whose STL hides `<numbers>` from nvcc's C++20 host passes;
`T-WINDOWS-NVCC-CCBIN-OLDEST-TOOLSET-2026-10-06`), then `PATH`.
`nvcc_build_msvc` is assigned before the `host_machine.system() == 'windows'`
block because the regression extracts that block into a project without
compilers and sets the variable itself. Keep configure regression in
`../test/test_windows_cuda_compiler_discovery.py` when rebasing Windows
discovery block from Netflix PR #1472.

## C++ targets take `vmaf_cppflags_common` (ADR-0379)

Expand Down
2 changes: 1 addition & 1 deletion core/src/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Parent scope: [`../AGENTS.md`](../AGENTS.md) (core) and

| Touching | Read first | Invariant |
| --- | --- | --- |
| `meson.build`, `/core/test/check_exported_symbols.py` | [build-and-compiler](AGENTS.d/build-and-compiler.md) | Windows CUDA discovers compiler on vswhere and PATH; C++ targets take vmaf_cppflags_common with hidden inlines. |
| `meson.build`, `/core/test/check_exported_symbols.py` | [build-and-compiler](AGENTS.d/build-and-compiler.md) | Windows nvcc host = build MSVC, else newest vswhere toolset, else PATH; C++ targets take vmaf_cppflags_common. |
| `feature/ciede.c`, `feature/cuda/integer_ciede/ciede_device.h`, `feature/ciede_ff_math.h`, `/core/test/test_ciede_upstream_products.c` | [ciede-squares-are-products](AGENTS.d/ciede-squares-are-products.md) | ciede.c squares = products (ADR-1467); its chroma and rotation products = float, no double cast (ADR-1476). |
| `picture.h`, `picture_pool.h` | [doxygen-header-invariants](AGENTS.d/doxygen-header-invariants.md) | Internal core/src headers carry Doxygen file briefs and parameter comments. |
| `framesync.c`, `framesync.h` | [framesync-producer](AGENTS.d/framesync-producer.md) | Framesync buffer error paths invoke vmaf_framesync_abort to prevent consumer cond_wait hang. |
Expand Down
48 changes: 36 additions & 12 deletions core/src/meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -1393,23 +1393,47 @@ if is_cuda_enabled
# On Windows, nvcc requires MSVC's cl.exe as host compiler for preprocessing.
# We avoid adding cl.exe to PATH as that causes meson to pick MSVC as the
# default C compiler. Use vswhere + powershell to find cl.exe automatically.
#
# When the build itself compiles C++ with MSVC, nvcc's host compiler is that
# same cl.exe: a second toolset compiles the host half of every .cu against
# another STL than the objects it links with, and a Visual Studio install
# carries older toolsets beside the current one. The search used to take
# the first cl.exe of a recursive walk, the v142 toolset (14.29) of VS 18,
# whose STL hides <numbers> from nvcc's C++20 host passes, so
# ciede_score.cu stopped compiling (T-WINDOWS-NVCC-CCBIN-OLDEST-TOOLSET-2026-10-06).
# Without an MSVC build compiler the search takes the newest toolset of
# the latest install. test_windows_cuda_compiler_discovery.py runs this block.
nvcc_build_msvc = ''
if is_variable('cxx')
if cxx.get_id() == 'msvc'
nvcc_build_msvc = find_program(cxx.cmd_array()[0]).full_path()
endif
endif
if host_machine.system() == 'windows'
cl_find_result = run_command('powershell', '-NoProfile', '-Command',
'(Get-ChildItem -Path (& "${env:ProgramFiles(x86)}/Microsoft Visual Studio/Installer/vswhere.exe" -latest -products * -property installationPath) -Recurse -Filter cl.exe -ErrorAction SilentlyContinue | Where-Object { $_.DirectoryName -like "*HostX64*x64*" } | Select-Object -First 1).FullName',
check: false)
if cl_find_result.returncode() == 0 and cl_find_result.stdout().strip() != ''
cl_path = cl_find_result.stdout().strip()
message('Found MSVC cl.exe at: ' + cl_path)
nvcc_ccbin_flags = ['--allow-unsupported-compiler', '-ccbin', cl_path]
if nvcc_build_msvc != ''
cl_path = nvcc_build_msvc
message('nvcc host compiler: the build MSVC at ' + cl_path)
else
cl_exe = find_program('cl', required: false)
if cl_exe.found()
cl_path = cl_exe.full_path()
nvcc_ccbin_flags = ['--allow-unsupported-compiler', '-ccbin', cl_path]
cl_find_result = run_command('powershell', '-NoProfile', '-Command',
'$vs = & "${env:ProgramFiles(x86)}/Microsoft Visual Studio/Installer/vswhere.exe" -latest -products * -property installationPath; ' +
'(Get-ChildItem -Path "$vs/VC/Tools/MSVC" -Directory -ErrorAction SilentlyContinue | ' +
'Sort-Object { [version]$_.Name } -Descending | ' +
'ForEach-Object { Join-Path $_.FullName "bin/HostX64/x64/cl.exe" } | ' +
'Where-Object { Test-Path $_ } | Select-Object -First 1)',
check: false)
if cl_find_result.returncode() == 0 and cl_find_result.stdout().strip() != ''
cl_path = cl_find_result.stdout().strip()
message('Found MSVC cl.exe at: ' + cl_path)
else
error('MSVC cl.exe not found. nvcc on Windows requires Visual Studio Build Tools.')
cl_exe = find_program('cl', required: false)
if cl_exe.found()
cl_path = cl_exe.full_path()
else
error('MSVC cl.exe not found. nvcc on Windows requires Visual Studio Build Tools.')
endif
endif
endif
nvcc_ccbin_flags = ['--allow-unsupported-compiler', '-ccbin', cl_path]

# Find MSVC and Windows SDK include dirs so cl.exe can find system
# headers when invoked by nvcc outside a vcvars environment.
Expand Down
79 changes: 66 additions & 13 deletions core/test/test_windows_cuda_compiler_discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,21 @@ def discovery_block() -> str:
SDK_ROOT = "C:/Windows Kits/10/Include/10.0.26100.0"


def _fixture_responses(compiler: Path, vswhere: str) -> dict[str, str]:
BUILD_COMPILER = "C:/Visual Studio/VC/Tools/MSVC/14.51/bin/HostX64/x64/cl.exe"


def _selected(compiler: Path, vswhere: str, build_msvc: str) -> str:
if build_msvc:
return build_msvc
return DISCOVERED_COMPILER if vswhere == "found" else str(compiler)


def _fixture_responses(compiler: Path, vswhere: str, build_msvc: str = "") -> dict[str, str]:
"""Canned PowerShell answers for one discovery case."""
return {
"vswhere": vswhere,
"discovered": DISCOVERED_COMPILER,
"selected": DISCOVERED_COMPILER if vswhere == "found" else str(compiler),
"selected": _selected(compiler, vswhere, build_msvc),
"msvc_root": MSVC_ROOT,
"sdk_root": SDK_ROOT,
}
Expand All @@ -90,6 +99,8 @@ def _write_powershell_stub(binary_dir: Path) -> None:
cfg = json.loads((Path(__file__).parents[1] / 'responses.json').read_text())
command = sys.argv[-1]
if 'vswhere.exe' in command:
if cfg['vswhere'] == 'forbidden':
sys.exit('searched for a compiler although the build compiles with MSVC')
if cfg['vswhere'] == 'error':
sys.exit(1)
if cfg['vswhere'] == 'found':
Expand Down Expand Up @@ -120,8 +131,13 @@ def _write_cross_file(root: Path) -> Path:
return cross


def _write_project(root: Path, expected_compiler: str) -> None:
"""Write the tiny Meson project that runs the shipped discovery block."""
def _write_project(root: Path, expected_compiler: str, build_msvc: str = "") -> None:
"""Write the tiny Meson project that runs the shipped discovery block.

``nvcc_build_msvc`` is what core/src/meson.build derives from the build's
C++ compiler when that compiler is MSVC; a project without languages cannot
hold a compiler object, so the fixture sets the variable directly.
"""
checks = (
f"assert(cl_path == {expected_compiler}, 'wrong compiler selected')\n"
"assert(nvcc_ccbin_flags == ['--allow-unsupported-compiler', '-ccbin', cl_path])\n"
Expand All @@ -132,12 +148,17 @@ def _write_project(root: Path, expected_compiler: str) -> None:
f" '-I', '{SDK_ROOT}/um'])\n"
)
(root / "meson.build").write_text(
"project('windows-cuda-discovery')\n" + discovery_block() + checks,
"project('windows-cuda-discovery')\n"
+ f"nvcc_build_msvc = '{build_msvc}'\n"
+ discovery_block()
+ checks,
encoding="utf-8",
)


def _build_fixture(root: Path, *, vswhere: str, path_compiler: bool) -> tuple[Path, Path]:
def _build_fixture(
root: Path, *, vswhere: str, path_compiler: bool, build_msvc: str = ""
) -> tuple[Path, Path]:
"""Materialise one discovery case under ``root``.

Returns the stub binary directory and the cross file.
Expand All @@ -148,15 +169,18 @@ def _build_fixture(root: Path, *, vswhere: str, path_compiler: bool) -> tuple[Pa
if path_compiler:
compiler.write_text(f"#!{sys.executable}\n", encoding="utf-8")
compiler.chmod(0o700)
responses = _fixture_responses(compiler, vswhere)
responses = _fixture_responses(compiler, vswhere, build_msvc)
(root / "responses.json").write_text(json.dumps(responses), encoding="utf-8")
_write_powershell_stub(binary_dir)
# The compiler selected by the branch is checked against the exact
# executable found by Meson, including the PATH fallback.
expected_compiler = (
f"'{DISCOVERED_COMPILER}'" if vswhere == "found" else "find_program('cl').full_path()"
)
_write_project(root, expected_compiler)
if build_msvc:
expected_compiler = f"'{build_msvc}'"
elif vswhere == "found":
expected_compiler = f"'{DISCOVERED_COMPILER}'"
else:
expected_compiler = "find_program('cl').full_path()"
_write_project(root, expected_compiler, build_msvc)
return binary_dir, _write_cross_file(root)


Expand Down Expand Up @@ -187,13 +211,42 @@ def _run_meson(root: Path, binary_dir: Path, cross: Path) -> subprocess.Complete


class WindowsCudaCompilerDiscovery(unittest.TestCase):
def configure(self, *, vswhere: str, path_compiler: bool) -> subprocess.CompletedProcess[str]:
def configure(
self, *, vswhere: str, path_compiler: bool, build_msvc: str = ""
) -> subprocess.CompletedProcess[str]:
self.assertTrue(MESON_COMMAND, "Meson is required for this configure regression")
with tempfile.TemporaryDirectory(prefix="vmafx-windows-discovery-") as temporary:
root = Path(temporary)
binary_dir, cross = _build_fixture(root, vswhere=vswhere, path_compiler=path_compiler)
binary_dir, cross = _build_fixture(
root, vswhere=vswhere, path_compiler=path_compiler, build_msvc=build_msvc
)
return _run_meson(root, binary_dir, cross)

def test_build_msvc_is_the_nvcc_host_compiler(self) -> None:
"""The build compiles C++ with MSVC: nvcc gets that cl.exe and no search runs.

A second toolset compiles the host half of every .cu against another STL
(T-WINDOWS-NVCC-CCBIN-OLDEST-TOOLSET-2026-10-06).
"""
result = self.configure(vswhere="forbidden", path_compiler=True, build_msvc=BUILD_COMPILER)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)

def test_build_msvc_comes_from_the_cpp_compiler(self) -> None:
source = SOURCE.read_text(encoding="utf-8")
marker = source.index("# default C compiler. Use vswhere + powershell")
derivation = source[
marker : source.index(" if host_machine.system() == 'windows'", marker)
]
self.assertIn("if cxx.get_id() == 'msvc'", derivation)
self.assertIn("nvcc_build_msvc = find_program(cxx.cmd_array()[0]).full_path()", derivation)

def test_search_takes_the_newest_toolset(self) -> None:
"""The first cl.exe of a recursive walk was the oldest toolset (14.29 in VS 18)."""
block = discovery_block()
self.assertNotIn("-Recurse -Filter cl.exe", block)
self.assertIn("Sort-Object { [version]$_.Name } -Descending", block)
self.assertIn('Join-Path $_.FullName "bin/HostX64/x64/cl.exe"', block)

def test_vswhere_success(self) -> None:
result = self.configure(vswhere="found", path_compiler=False)
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
Expand Down
23 changes: 15 additions & 8 deletions docs/getting-started/building-on-windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,21 @@ The native MSVC CUDA build compiles, links and installs in CI; the hosted
Windows runner has no GPU, so it runs no GPU scoring tests.

CUDA needs Visual Studio Build Tools and the Windows SDK, even when the host
library is built with MinGW. Meson looks for `cl.exe` in two places:

1. through `vswhere`;
2. if that finds nothing, on `PATH`, as in an x64 Native Tools Command Prompt.

The compiler it finds is passed to NVCC as `-ccbin` and used for MSVC header
discovery. If neither place has a compiler, configuration stops and says that
Visual Studio Build Tools are required.
library is built with MinGW. NVCC's host compiler (`-ccbin`) is chosen in this
order:

1. when the build compiles C++ with MSVC (a Native Tools Command Prompt or
`vcvarsall.bat`), that same `cl.exe`, so the host half of every `.cu` file
uses the same toolset and standard library as the rest of the build;
2. otherwise the newest MSVC toolset of the latest Visual Studio install that
`vswhere` reports (an install can carry older toolsets beside the current
one, such as 14.29 in Visual Studio 2026, whose library cannot compile the
C++20 headers the CUDA kernels use);
3. if that finds nothing, `cl.exe` on `PATH`.

The compiler it picks is also used for MSVC header discovery; configuration
prints which one. If no place has a compiler, configuration stops and says
that Visual Studio Build Tools are required.

A regression test checks this lookup on any POSIX host, without a Windows SDK
or a GPU, using stubbed compiler responses:
Expand Down
9 changes: 9 additions & 0 deletions docs/rebase-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -62404,3 +62404,12 @@ persona sources under `.agents/` and their four projections
paths; a sync keeps the block where it is (the managed block must stay at the tail).
`scripts/ci/tests/test_praetor_hashed_files_lf.py` fails when a rule is missing. No score, public
API or FFmpeg patch impact.

## nvcc on Windows uses the build's MSVC (`fix/nvcc-ccbin-build-msvc`)

The Windows discovery block of `core/src/meson.build` (ported from the unmerged Netflix PR #1472, ADR-0150)
now gives nvcc the build's own `cl.exe` when `cxx` is MSVC (`nvcc_build_msvc`, assigned just before the
block), otherwise the newest toolset under the latest `vswhere` install, otherwise `cl` on `PATH`
(`T-WINDOWS-NVCC-CCBIN-OLDEST-TOOLSET-2026-10-06`). Upstream master has no such block; a re-port keeps
this order and `core/test/test_windows_cuda_compiler_discovery.py`. No score, public API or FFmpeg patch
impact.
Loading
Loading