Running autotools-based external project on Windows

[QuLogic]

I'm trying to add a wrap to a project for libidn (not libidn2), which uses autotools as a build system. I've put together a small example doing so. On Linux, this works out just fine. On Windows (using x64 native command prompt), I've run into two issues:

1. The shebang contains /bin/sh; if I let Meson run normally, it fails to find sh, which is expected. I've tried to set a native machine file with sh pointing to bash.exe

[binaries]
sh = 'c:/Windows/System32/bash.exe'

(or pointing to Git Bash, or using Windows slashes instead) but this always fails to find the interpreter:

libidn| Running command ['sh', 'C:/Users/quant/extproj/subprojects/libidn-1.41/configure', '--prefix=/', '--libdir=//lib'] in directory C:\Users\quant\extproj\build\subprojects\libidn-1.41\build

Traceback (most recent call last):
  File "mesonbuild\mesonmain.py", line 194, in run
  File "mesonbuild\msetup.py", line 358, in run
  File "mesonbuild\msetup.py", line 183, in generate
  File "mesonbuild\msetup.py", line 228, in _generate
  File "mesonbuild\interpreter\interpreter.py", line 2997, in run
  File "mesonbuild\interpreterbase\interpreterbase.py", line 164, in run
  File "mesonbuild\interpreterbase\interpreterbase.py", line 190, in evaluate_codeblock
  File "mesonbuild\interpreterbase\interpreterbase.py", line 182, in evaluate_codeblock
  File "mesonbuild\interpreterbase\interpreterbase.py", line 196, in evaluate_statement
  File "mesonbuild\interpreterbase\interpreterbase.py", line 521, in function_call
  File "mesonbuild\interpreterbase\decorators.py", line 277, in wrapper
  File "mesonbuild\interpreterbase\decorators.py", line 596, in wrapper
  File "mesonbuild\interpreter\interpreter.py", line 878, in func_subproject
  File "mesonbuild\interpreter\interpreter.py", line 959, in do_subproject
  File "mesonbuild\interpreter\interpreter.py", line 943, in do_subproject
  File "mesonbuild\interpreter\interpreter.py", line 986, in _do_subproject_meson
  File "mesonbuild\interpreter\interpreter.py", line 2997, in run
  File "mesonbuild\interpreterbase\interpreterbase.py", line 164, in run
  File "mesonbuild\interpreterbase\interpreterbase.py", line 190, in evaluate_codeblock
  File "mesonbuild\interpreterbase\interpreterbase.py", line 182, in evaluate_codeblock
  File "mesonbuild\interpreterbase\interpreterbase.py", line 198, in evaluate_statement
  File "mesonbuild\interpreterbase\interpreterbase.py", line 635, in assignment
  File "mesonbuild\interpreterbase\interpreterbase.py", line 200, in evaluate_statement
  File "mesonbuild\interpreterbase\interpreterbase.py", line 550, in method_call
  File "mesonbuild\interpreter\interpreterobjects.py", line 811, in method_call
  File "mesonbuild\interpreterbase\decorators.py", line 277, in wrapper
  File "mesonbuild\interpreterbase\decorators.py", line 596, in wrapper
  File "mesonbuild\modules\external_project.py", line 302, in add_project
  File "mesonbuild\modules\external_project.py", line 107, in __init__
  File "mesonbuild\modules\external_project.py", line 171, in _configure
  File "mesonbuild\modules\external_project.py", line 216, in _run
  File "mesonbuild\utils\universal.py", line 1483, in Popen_safe
  File "mesonbuild\utils\universal.py", line 1505, in Popen_safe_legacy
  File "subprocess.py", line 1026, in __init__
  File "subprocess.py", line 1538, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified

It seems that the interpreter from the shebang cannot be overridden by the machine file. Consequently, the above project contains a patch that changes the shebang to /bin/bash instead.
2. After applying the patch, the interpreter can be found on the PATH as usual. However, it fails to find the configure script:

libidn| Running command ['bash', 'C:/Users/quant/extproj/subprojects/libidn-1.41/configure', '--prefix=/', '--libdir=//lib'] in directory C:\Users\quant\extproj\build\subprojects\libidn-1.41\build

/bin/bash: C:/Users/quant/extproj/subprojects/libidn-1.41/configure: No such file or directory

If I then cd to the build directory and run configure with a relative path, then it works:

> cd build\subprojects\libidn-1.41\build
>bash ../../../../subprojects/libidn-1.41/configure
configure: GNU Libidn 1.41 libidn 18:4:6 12
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a race-free mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for gcc... gcc
# output trimmed

The full log and the internal log can be found on this gist.

Is there something wrong I'm doing here, or is Meson messing up?

Has anyone successfully built an autotools-based external project on Windows? Do I have to bite the bullet and write a Meson build (which I'd rather not do since there's so much configuration there)?

[eli-schwartz]

I think that the issue here may be because you cannot run C:\.... paths as arguments to the bash shell due to the POSIX emulation layer, which expects a cygpath.

I suspect it would be more reliable if you need to actually execute an autotools based subproject to build within msys2 targeting the UCRT, because in that case the autotools configure script is much more likely to run correctly.

However, one possible fix would be for mesonbuild/modules/external_project.py to run the configure command as ./configure which is indeed how most projects run a configure command, it feels like we don't really need abspaths here.

/cc @xclaesse