autotools m4 breakage from v2.0.12 to v2.0.13

[flatmax]

m4 package finding is breaking between the versions v2.0.12 to v2.0.13

If I build the package sox, the m4 code works for the earlier version and then breaks for the latter.
At the end of configure, for v2.0.12 I get : oggvorbis..................yes
At the end of configure, for v2.0.13 I get : oggvorbis..................no

To reproduce .... put the script at the end of this issue in the file /tmp/wasmBreakage.sh

I install/actiave v2.0.12 and remove build directories and then rebuild :

./emsdk install 2.0.12
./emsdk activate 2.0.12
# create a new terminal to get rid of residual env vars.
rm -rf /tmp/wasm; /tmp/wasmBreakage.sh

At the end of the execution, I see : oggvorbis..................yes

I install/actiave 2.0.13 and remove build directories and then rebuild :

./emsdk install 2.0.13
./emsdk activate 2.0.13
# create a new terminal to get rid of residual env vars.
rm -rf /tmp/wasm; /tmp/wasmBreakage.sh

At the end of the execution, I see : oggvorbis..................no

#########################################################################################
# File : wasmBreakage.sh
# Author Matt Flax <flatmax@>
# Date 20210324
# Demonstrates m4 breakage with emscripten

source /usr/local/emsdk/emsdk_env.sh

export BUILD_DIR=/tmp/wasm
export EMSCRIPTEN=$EMSDK/upstream/emscripten/system

if [ ! -d $BUILD_DIR ]; then
	mkdir $BUILD_DIR

	cd $BUILD_DIR

	OGG_DIR=libogg-1.3.4
	wget https://ftp.osuosl.org/pub/xiph/releases/ogg/$OGG_DIR.tar.gz
	tar zxpvf $OGG_DIR.tar.gz
	cd $OGG_DIR
	emconfigure ./configure --enable-shared --prefix=$EMSCRIPTEN
	emmake make -j8
	emmake make install

	cd $BUILD_DIR

	VORBIS_DIR=libvorbis-1.3.6
	wget https://ftp.osuosl.org/pub/xiph/releases/vorbis/$VORBIS_DIR.tar.gz
	tar zxpvf $VORBIS_DIR.tar.gz
	cd $VORBIS_DIR
	./autogen.sh
	emconfigure ./configure --enable-shared --prefix=$EMSCRIPTEN
	emmake make -j8
	emmake make install

	cd $BUILD_DIR

	export SOX_DIR=sox-git
	git clone --depth=1 git://git.code.sf.net/u/flatmax/sox $SOX_DIR
	cd $SOX_DIR
	autoreconf -i
	emconfigure ./configure --enable-shared --prefix=$EMSCRIPTEN
else
	echo ERROR $BUILD_DIR should be rm before running
fi
############################################################################################

wasmBreakage.txt

[flatmax]

I believe this is the file which stops working between versions :
https://sourceforge.net/u/flatmax/sox/ci/master/tree/m4/optional-fmt.m4

[kripken]

Bisection is probably the fastest way with an issue like this. The docs there have just been updated and that should be a lot simpler and clearer now, https://emscripten.org/docs/contributing/developers_guide.html#bisecting

[flatmax]

OK, I have stepped through releases as a finer level using your suggestion above.

The commit which breaks autotools and m4 is : 096b46a0d5765fa2952c24547c184059c336e2e4
I ran the above tests with the following commits :
bf79139b43a8d562e865bf7af3b47f7df0eae5b9 works
096b46a0d5765fa2952c24547c184059c336e2e4 breaks

Copying @sbc100 too here.

[kripken]

Thanks @flatmax !

The breaking commit you mention is https://chromium.googlesource.com/emscripten-releases/+/096b46a0d5765fa2952c24547c184059c336e2e4 which is a single emscripten commit,

Roll emscripten from ae6f40d7d145 to 298b2af3714e (1 revision)

https://chromium.googlesource.com/external/github.com/emscripten-core/emscripten.git/+log/ae6f40d7d145..298b2af3714e

2021-01-10 sbc@chromium.org Build a complete sysroot in the cache directory (#13090)

So #13090 is at fault (and @sbc100 already cc'd)

[sbc100]

Ah I see the problem.

You are setting

 EMSCRIPTEN=$EMSDK/upstream/emscripten/system

And then using:

--prefix=$EMSCRIPTEN

Which is telling autoconf to install stuff in upstream/emscripten/system/lib and upstream/emscripten/system/include which are no longer part of the search path so the following configure phases can't find the headers.

Instead of upstream/emscripten/system/lib I think you want to just specify --prefix=$EM_CACHE/sysroot. I am in the process of verifying that that works.

[sbc100]

Yup that works.

Ideally we would support installing to sysroot/usr and/or sysroot/usr/local too...

[flatmax]

I installed latest and did a test.
with the following :
export SYSPATH=$EM_CACHE/sysroot/usr
using --prefix=$SYSPATH

It doesn't work.

[flatmax]

I also tried the following :

export SYSPATH=$EMSDK/upstream/emscripten/system/usr
export SYSPATH=$EM_CACHE/sysroot

They don't work either.

[sbc100]

The only that I expect will work today is SYSPATH=$EM_CACHE/sysroot.

I just tested that locally with the script your provided and it works for me. Can you try again? What value does $EM_CACHE have for you?

[flatmax]

I couldn't get that to work. I used the following :

export SYSPATH=$EM_CACHE/sysroot
--prefix=$SYSPATH

I will copy the script again below. These are the commands I used :

./emsdk install latest
./emsdk activate latest
rm -rf /tmp/wasm; /tmp/wasmBreakage.sh

############### FILE wasmBreakage.sh  ##############################
source /usr/local/emsdk/emsdk_env.sh

export BUILD_DIR=/tmp/wasm
#export SYSPATH=$EMSDK/upstream/emscripten/system/usr
export SYSPATH=$EM_CACHE/sysroot

if [ ! -d $BUILD_DIR ]; then
mkdir $BUILD_DIR

cd $BUILD_DIR

OGG_DIR=libogg-1.3.4
wget https://ftp.osuosl.org/pub/xiph/releases/ogg/$OGG_DIR.tar.gz
tar zxpvf $OGG_DIR.tar.gz
cd $OGG_DIR
emconfigure ./configure --enable-shared CXXFLAGS="$EMSC_FLAGS" --prefix=$SYSPATH
emmake make -j8
emmake make install

cd $BUILD_DIR

VORBIS_DIR=libvorbis-1.3.6
wget https://ftp.osuosl.org/pub/xiph/releases/vorbis/$VORBIS_DIR.tar.gz
tar zxpvf $VORBIS_DIR.tar.gz
cd $VORBIS_DIR
./autogen.sh
emconfigure ./configure --enable-shared CXXFLAGS="$EMSC_FLAGS" --prefix=$SYSPATH
emmake make -j8
emmake make install

cd $BUILD_DIR

export SOX_DIR=sox-git
git clone --depth=1 git://git.code.sf.net/u/flatmax/sox $SOX_DIR
cd $SOX_DIR
autoreconf -i
emconfigure ./configure --enable-shared CXXFLAGS="$EMSC_FLAGS" --prefix=$SYSPATH


else
echo ERROR $BUILD_DIR should be rm before running
fi
#####################################################

[sbc100]

That works for me. How is it failing for you?

I'll attach the full log of running those steps on my machine.

[sbc100]

What emsdk version is latest resolving to for you? Maybe try emsdk update too?

[sbc100]

buildlog.txt

[flatmax]

Oh, not working for you. Your buildlog says :
oggvorbis..................no
It should say :
oggvorbis..................yes

Sox's configure script can't find the installed ogg vorbs dev files.

[sbc100]

Ah, I see the problem. The problem is that we include sysroot/lib/wasm32-emscripten in the library path by default but we don't add sysroot/lib.

I'm working on a real fix but for now you should be able work around this using export LDFLAGS=-L$EM_CACHE/sysroot/lib