cabal-testsuite/PackageTests/Get/OnlyDescription/cabal.test.hs isn't idempotent

[nlander]

Describe the bug
The test cabal-testsuite/PackageTests/Get/OnlyDescription/cabal.test.hs succeeds on first run and then fails on all subsequent runs

To Reproduce
In the cabal directory

cd cabal-testsuite/PackageTests/Get/OnlyDescription/
cab=$(cabal list-bin cabal-install:cabal)
cabtest=$($cab list-bin cabal-testsuite:cabal-tests)
$cabtest --quiet --with-cabal $cab  # this one should pass if you haven't already run this test
$cabtest --quiet --with-cabal $cab # this one should fail

Expected behavior
I would expect that running this test more than once produces the same result. If it passed once, it should pass again. If it failed once, it should fail again (for the same reason).

System information

• Operating system Pop!_OS 22.04 LTS
• cabal-install version 3.10.1.0 compiled using version 3.10.1.0 of the Cabal library
• ghc version 9.6.1

Additional context
This issue is being filed as a result of discussion in the hackage irc channel.

[ulysses4ever]

Thanks for the report! It shouldn't be hard to fix but there are two ways one can go (you can also do both): improve cleanup in the testsuite (this could fix some more issues like this) or make cabal-get less strict about overwrites.

[ffaf1]

The “improve cleanup” solution can be made easier if you download a specific criterion version, i.e.

import Test.Cabal.Prelude
main = cabalTest $ withRepo "repo" $ do
  cabal "update" []
  cabal
    "get"
    [ "criterion-1.2.2.0", "--only-package-description" ] -- Notice 1.2.2.0 here.

Then you can simply call removeFile.

[ulysses4ever]

That could work fine, i guess. But more generally I'd expect the cabalTest utility to make sure that tests run in a clean environment…

[fgaz]

https://github.com/haskell/cabal/tree/a191f0a2bf72894b3d830ca3ee4ea111d96b6f65/cabal-testsuite#hermetic-tests