Wrapper compiler adding incorrect library search paths

[bwbarrett]

I'm guessing this is also in v4.0.x, v5.0.x, and master, but have not yet confirmed. On Ubuntu 18.04, when configuring Open MPI with:

--prefix=/opt/amazon/openmpi --with-hwloc=/usr --with-libevent=/usr

We end up with wrapper link line:

mpicc --show-me
gcc -I/opt/amazon/openmpi/include -pthread -L/usr/lib -Wl,-rpath -Wl,/usr/lib -Wl,-rpath -Wl,/opt/amazon/openmpi/lib -Wl,--enable-new-dtags -L/opt/amazon/openmpi/lib -lmpi

There are two problems with this link line:

1. It appears the code to avoid leaking system includes / library search paths into the CFLAGS/LDFLAGS is not working, given the -L/usr/lib -Wl,-rpath -Wl,/usr/lib.
2. We are placing dependency search paths before our -L<libdir> search path, which means that if any of the dependency directories have a libmpi.so installed in them, we will find the wrong libmpi.so and confusion will reign.

[hppritcha]

this may be in v4.0.x, @bwbarrett checking

[bwbarrett]

This is an issue in v4.0.x and v4.1.x, but not v5.0.x and master. So small favors.

[bwbarrett]

I was only partially right. In 5.0 and master, we don't include the -L/usr/lib (yay), but we do order the search paths we do add so that the location of libmpi.so is last (boo). We should fix that.