Cross-compilation of `zstd-sys` for `x86_64-pc-windows-msvc` target fails starting at 1.0.77

[Hainish]

I am cross-compiling zstd-sys to x86_64-pc-windows-msvc in https://github.com/EFForg/apkeep/blob/5bffbf77cba71751884a46234dab66d83d5b0dbe/build-remote.sh#L75-L101. When cc is updated from 1.0.76 to 1.0.77, the build starts failing with the following:

  running: "clang-cl" "-nologo" "-MD" "-O2" "-Brepro" "-m64" "-Wno-unused-command-line-argument" "-fuse-ld=lld-link" "/imsvc/home/admin/xwin/crt/include" "/imsvc/home/admin/xwin/sdk/include/ucrt" "/imsvc/home/ad
min/xwin/sdk/include/um" "/imsvc/home/admin/xwin/sdk/include/shared" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB
_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-Fo/home/admin/apkeep/target/x86_64-pc-windows-msvc/release/build/zstd-sys-6caf8d3f6c2dc823/out/zstd/lib/legacy/zstd
_v06.o" "-c" "--" "zstd/lib/legacy/zstd_v06.c"
  cargo:warning=clang: warning: unknown argument ignored in clang-cl: '-fvisibility=hidden' [-Wunknown-argument]
  exit status: 0
  running: "clang-cl" "-nologo" "-MD" "-O2" "-Brepro" "-m64" "-Wno-unused-command-line-argument" "-fuse-ld=lld-link" "/imsvc/home/admin/xwin/crt/include" "/imsvc/home/admin/xwin/sdk/include/ucrt" "/imsvc/home/ad
min/xwin/sdk/include/um" "/imsvc/home/admin/xwin/sdk/include/shared" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB
_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-Fo/home/admin/apkeep/target/x86_64-pc-windows-msvc/release/build/zstd-sys-6caf8d3f6c2dc823/out/zstd/lib/legacy/zstd
_v02.o" "-c" "--" "zstd/lib/legacy/zstd_v02.c"
  cargo:warning=clang: warning: unknown argument ignored in clang-cl: '-fvisibility=hidden' [-Wunknown-argument]
  exit status: 0
  running: "clang-cl" "-nologo" "-MD" "-O2" "-Brepro" "-m64" "-Wno-unused-command-line-argument" "-fuse-ld=lld-link" "/imsvc/home/admin/xwin/crt/include" "/imsvc/home/admin/xwin/sdk/include/ucrt" "/imsvc/home/ad
min/xwin/sdk/include/um" "/imsvc/home/admin/xwin/sdk/include/shared" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB
_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-Fo/home/admin/apkeep/target/x86_64-pc-windows-msvc/release/build/zstd-sys-6caf8d3f6c2dc823/out/zstd/lib/legacy/zstd
_v03.o" "-c" "--" "zstd/lib/legacy/zstd_v03.c"
  cargo:warning=clang: warning: unknown argument ignored in clang-cl: '-fvisibility=hidden' [-Wunknown-argument]
  exit status: 0
  running: "clang-cl" "-nologo" "-MD" "-O2" "-Brepro" "-m64" "-Wno-unused-command-line-argument" "-fuse-ld=lld-link" "/imsvc/home/admin/xwin/crt/include" "/imsvc/home/admin/xwin/sdk/include/ucrt" "/imsvc/home/ad
min/xwin/sdk/include/um" "/imsvc/home/admin/xwin/sdk/include/shared" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB
_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-Fo/home/admin/apkeep/target/x86_64-pc-windows-msvc/release/build/zstd-sys-6caf8d3f6c2dc823/out/zstd/lib/legacy/zstd
_v05.o" "-c" "--" "zstd/lib/legacy/zstd_v05.c"
  cargo:warning=clang: warning: unknown argument ignored in clang-cl: '-fvisibility=hidden' [-Wunknown-argument]
  exit status: 0
  running: "clang-cl" "-nologo" "-MD" "-O2" "-Brepro" "-m64" "-Wno-unused-command-line-argument" "-fuse-ld=lld-link" "/imsvc/home/admin/xwin/crt/include" "/imsvc/home/admin/xwin/sdk/include/ucrt" "/imsvc/home/ad
min/xwin/sdk/include/um" "/imsvc/home/admin/xwin/sdk/include/shared" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB
_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-Fo/home/admin/apkeep/target/x86_64-pc-windows-msvc/release/build/zstd-sys-6caf8d3f6c2dc823/out/zstd/lib/legacy/zstd
_v04.o" "-c" "--" "zstd/lib/legacy/zstd_v04.c"
  cargo:warning=clang: warning: unknown argument ignored in clang-cl: '-fvisibility=hidden' [-Wunknown-argument]
  exit status: 0
  running: "ml64.exe" "-nologo" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISI
BILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-fvisibility=hidden" "-Fo/home/admin/apkeep/target/x86_64-pc-windows-msvc/release/build/zstd-sys-6caf8d3f6c2dc823/out/zstd/lib/decompress/huf_decompress_amd64.o" "-c" "zstd/li
b/decompress/huf_decompress_amd64.S"
  exit status: 0

  --- stderr


  error occurred: Failed to find tool. Is `ml64.exe` installed?

In 1.0.76 it succeeds.

[Hainish]

cc @gyscos

[thomcc]

In 1.0.76 it succeeds.

Huh weird. We landed some changes around this in 1.0.76 and 1.0.75, but nothing in 1.0.77, so that's very surprising.

[dot-asm]

nothing in 1.0.77

1.0.77 changes meaning of is_asm in compile_object. As result, ml64 is called to compile a file it can't handle and gets clang-cl's flags it doesn't understand.

[thomcc]

Very nice catch, thank you. I probably won't be able to fix this this week (although who knows, I have slightly more time than anticipated), but it does seem quite bad :(

[thomcc]

Oh, looking more closely, the way this changes is_asm is that now .s and .asm (case-insensitive) are considered to be asm files, whereas before it was just .asm.

I didn't think about that at the time, but I think we should keep that behavior...

[Hainish]

Is there an appropriate env var to specify that capitalized .S files should be handled by clang-cl as well?

[thomcc]

No, but reasonable suggestions are welcome.

[Hainish]

Probably should be incorporated within the env vars already existing: https://github.com/EFForg/apkeep/blob/5bffbf77cba71751884a46234dab66d83d5b0dbe/build-remote.sh#L91-L93

[thomcc]

Those impacting how we handle a specific file extension are pretty surprising though...

Aside from it failing in this specific project, do you have an argument for why .S files should be treated differently than other assembly files? My experience has certainly been that .s/.S are more common file extensions than .asm for assembly...

[Hainish]

I'm out of my element here so I don't have good arguments to make, I just have this compilation error to deal with.

[Hainish]

If my understanding of the issue is correct, it is being treated differently currently: it's calling out to ml64.exe rather than using clang-cl as specified in the env vars above. So getting past this means having .S files respect that env var and use clang-cl instead. But I may be misunderstanding the underlying problem.

[gyscos]

As mentioned in gyscos/zstd-rs#176, it appears we should not have included that file for compilation. Using ml64.exe is the correct choice for assembly files (and if you wanted to support asm files you'd have to install it), but in this case the assembly file does not actually support windows.
It was included by mistake because we were looking at the host OS rather than the target OS to decide whether to include this file.

[Hainish]

Thank you, looks like you fixed it in the downstream dependency!