stack build --copy-bins --test only copies binaries after testing

[Fuuzetsu]

Currently we can not run tests which rely on a binary produced during build. Something like

stack build && stack build --test --bench

works but rebuilds twice. Ideally stack build --test --bench would make binaries produced available before the tests run so we can use the executable for the testing.

[mgsloan]

I also ran into this and reported with #701, and I'm not sure what the ideal solution is.

Maybe binaries should be copied to the local install root even when tests don't pass (and do it before running tests)?

[snoyberg]

Maybe binaries should be copied to the local install root even when tests don't pass (and do it before running tests)?

I did comment on that in the previous issue:

that's intentional and correct: we shouldn't be doing any installing of something which fails tests

I think the right approach is to modify the test suite to use the executable from the dist directory. You could use getExecutablePath to find the location of the test suite executable, and then ../exename/exename should give you the path to the executable just built.

There may be some issues though around not rebuilding the executable without explicitly requesting it. This is an optimization for the common case of testing a library. You may need to explicitly state that you want to build both the executable and test suite, e.g. stack build :exename :testname.

[snoyberg]

IMO, the real bug here actually is that doing stack build and then stack test forces the library to be recompiled, due to the changes to cabal_macros.h. I've opened up an issue about this against cabal:

haskell/cabal#2775

With the current cabal interface, I can't think of a good way to solve this.

[Fuuzetsu]

The question is whether we wait for cabal or do what you suggest and look into dist.

[snoyberg]

I wouldn't recommend waiting for cabal, getting a change made and released
will take much more than our usual turnaround time.

On Tue, Aug 18, 2015, 1:11 PM Mateusz Kowalczyk notifications@github.com
wrote:

The question is whether we wait for cabal or do what you suggest and look
into dist.

—
Reply to this email directly or view it on GitHub
#805 (comment)
.

[snoyberg]

This is quickly veering off course from the original issue description, but here's my plan of attack right now (inspired by an idea @gregwebs had a few months back):

• Check if all test suite dependencies are available
• If they are, configure with --enable-tests

Then: cabal_macros.h won't change between stack build and stack test, which should make the compile time much smaller, hopefully obviating the speed hit described here. This will also mean that we can resolve the comment at:

stack/src/Stack/Build/ConstructPlan.hs

Line 135 in 4159ec0

{-

Which I believe will also adversely affect you.

[mboes]

I think the right approach is to modify the test suite to use the executable from the dist directory. You could use getExecutablePath to find the location of the test suite executable, and then ../exename/exename should give you the path to the executable just built.

That sounds like quite a bit of gymnastics for something that is a very common use case: perform system-level testing of the binary itself, e.g. to check that piping scripts through stdin and parsing all the command line arguments yields the correct result. I agree that performing say --copy-bins before --test is the wrong thing to do. But why not make the binary that was just built before the testing locally available in the PATH immediately? That is, I would expect that stack build && stack test and stack build --test to always be equivalent, yet this is not the case here.

The fact that stack build && stack test takes longer than expected is a separate bug I think. Fantastic to see that you're addressing that in #815!

[snoyberg]

Actually, I didn't realize it when initially reading the issue, but I think the change I made did just that. Specifically, I think the culprit here was:

stack/src/Stack/Build/ConstructPlan.hs

Line 135 in 4159ec0

{-

In other words: before #815, we weren't rebuilding the executable at all when doing stack test, because the reconfigure overhead from cabal was too high. #815 gets rid of that overhead with the hack described there, so now the executables are being built and copied into the local installation directory.

Can you try out the #815 PR and see how that works for you?

[snoyberg]

The behavior on master for this command should now be:

• Library and executables are built
• Library and executables are copied to the .stack-work/install directory and library is registered
• Test suite is built
• Test suite is run with the install directory on the PATH
• If everything succeeds, the executables are copied to the install path

If this is not happening, please reopen.