testsuite: Run tests in temporary system directories #9717
Conversation
|
I like to be able to run $ cabal run cabal-testsuite:cabal-tests -- --with-cabal=./dist-newstyle/build/x86_64-linux/ghc-9.8.1/cabal-install-3.11.0.0/x/cabal/build/cabal/cabal cabal-testsuite/PackageTests/CopyHie/setup.test.hs
...
- setup.test.hs: /home/<USERNAME>/.../cabal/cabal-testsuite/PackageTests/CopyHie/setup.dist/usr/lib/x86_64-linux-ghc-9.8.1/hie-local-0.1.0.0/extra-compilation-artifacts/hie/HieLocal.hie should exist
+ setup.test.hs: /tmp/cabal-testsuite-156428/setup.dist/usr/lib/x86_64-linux-ghc-9.8.1/hie-local-0.1.0.0/extra-compilation-artifacts/hie/HieLocal.hie should existPutting the tests into a temporary directory is definitely an improvement, if only because it puts the tests into a clean room away from whatever junk may be lying around in my working directory. |
alt-romes
force-pushed
the
wip/system-testsuite
branch
2 times, most recently
from
891a8c6 to
8214596
Compare
| -- This requires the test repository to be a Git checkout, because | ||
| -- we use the Git metadata to figure out what files to copy into the | ||
| -- hermetic copy. | ||
| -- | ||
| -- Also see 'withSourceCopyDir'. | ||
| withSourceCopy :: TestM a -> TestM a | ||
| withSourceCopy m = do | ||
| env <- getTestEnv | ||
| initWorkDir | ||
| let curdir = testSourceDir env | ||
| dest = testSourceCopyDir env | ||
| r <- getSourceFiles |
There was a problem hiding this comment.
We should probably add a warning or error here if there are no tracked source files. Otherwise, it may be confusing to a user who forgot to track files that their tests do not get run as expected.
There was a problem hiding this comment.
Good idea, I have added the check.
There was a problem hiding this comment.
@mpickering this looks good to me, but it seems we should add an additional error or warning if there are no tracked files for a test (see above comment).
Thanks!
mpickering
force-pushed
the
wip/system-testsuite
branch
2 times, most recently
from
f410cdb to
016d60a
Compare
|
Does another Cabal developer wish to approve this before I merge it? It does slightly change the test workflow when adding new tests. |
mpickering
force-pushed
the
wip/system-testsuite
branch
2 times, most recently
from
552f817 to
a6b878d
Compare
mergify
bot
added
the
merge delay passed
1. Working directory is not contaminated with results of the testsuite. Tests can be written in a more liberal way (creating/writing files etc). 2. The tests are more hermetic.. for example it was basically impossible to write a test which run outside of a project context. 3. The overall path lengths will be much shorter, which has been a consistent issue on windows. We can remove hacks like `withShorterPathForNewBuildStore`. Fixes haskell#9711
Mikolaj
force-pushed
the
wip/system-testsuite
branch
from
a6b878d to
0ef4f44
Compare
withShorterPathForNewBuildStore.Fixes #9711
Please read Github PR Conventions and then fill in one of these two templates.
Template Β: This PR does not modify
cabalbehaviour (documentation, tests, refactoring, etc.)Include the following checklist in your PR: