TRUST CI: i686-unknown-linux-musl build hang

[posborne]

This might be a spurious failure or related to the build in general taking too long on travis. Creating this as a general tracking item.

Changes to fix would go into new-ci/master branch for now.

Instance: https://travis-ci.org/berkowski/nix/jobs/205534671

[posborne]

Running locally using cross test --target=i686-unknown-linux-musl, it appear that some of the AIO tests might be hanging...

$ cross test --target=i686-unknown-linux-musl
...
test sys::test_aio::test_aio_write_sigev_signal has been running for over 60 seconds
test sys::test_aio::test_lio_listio_signal has been running for over 60 seconds

CC @asomers. Note that this is prior to your most recent PR.

[asomers]

https://github.com/japaric/cross says that QEMU "gets upset when you spawn several threads". Those tests technically do spawn several threads, because glibc implements POSIX AIO using threads. It could be a QEMU bug. However, the two tests that hang are the only two that use signals, which suggests that something involving signal delivery is screwed up. In either case, it would be easier to debug if we can get Travis and QEMU out of the picture. Is there a musl-based VM image or installer somewhere that I can spin up in a different hypervisor?

[posborne]

Is there a musl-based VM image or installer somewhere that I can spin up in a different hypervisor?

MUSL binaries should be statically linked, so you should be able to run them in a standard linux environment.

[posborne]

Ok, I did a little more digging on this one and the problem with the new CI is one that we have had in the past relating to concurrency.

By default, cargo will run multiple tests in parallel. When I changed things locally to run with RUST_TEST_THREADS=1, I saw consistent passes with this target. Due to the type of testing we have to do in nix, I will make the change to define this environment variable in CI to avoid these spurious failures.

Here's the original issue where we ran into this before: #292