`cabal new-haddock` regression (setup: internal error when calculating transitive package dependencies.)

[RyanGlScott]

I originally noticed this issue when trying to upgrade singletons' Travis setup to use cabal-install-head. That is:

$ /opt/cabal/head/bin/cabal --version
cabal-install version 2.3.0.0
compiled using version 2.3.0.0 of the Cabal library

I've prepared a more minimal reproduction at https://github.com/RyanGlScott/cabal-gh5423. To reproduce the issue, do the following:

$ git clone https://github.com/RyanGlScott/cabal-gh5423
$ cd cabal-gh5423/
$ /opt/cabal/head/bin/cabal new-haddock cabal-gh5423:tests
Resolving dependencies...
Build profile: -w ghc-8.4.3 -O1
In order, the following will be built (use -v for more details):
 - cabal-gh5423-0.1 (first run)
[1 of 1] Compiling Main             ( /home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/setup/setup.hs, /home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/setup/Main.o )
Linking /home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/setup/setup ...
Configuring cabal-gh5423-0.1...
Found dependency closure
Preprocessing library for cabal-gh5423-0.1..
Running Haddock on library for cabal-gh5423-0.1..
Haddock coverage:
   0% (  0 /  1) in 'CabalSandbox'
  Missing documentation for:
    Module header
Documentation created:
/home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/doc/html/cabal-gh5423/index.html
Preprocessing test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..
Running Haddock on test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..
setup: internal error when calculating transitive package dependencies.
Debug info: []

cabal: Failed to build documentation for cabal-gh5423-0.1.

cabal-gh5423 is a package with a library, test suite, and custom-setup script that looks like:

module Main (main) where

import Distribution.PackageDescription
import Distribution.Simple
import Distribution.Simple.LocalBuildInfo
import Distribution.Simple.PackageIndex

main :: IO ()
main = defaultMainWithHooks simpleUserHooks
  { haddockHook = \pkg lbi hooks flags -> do
      blah pkg lbi
      haddockHook simpleUserHooks pkg lbi hooks flags
  }

blah :: PackageDescription -> LocalBuildInfo -> IO ()
blah pkg lbi =
  withLibLBI pkg lbi $ \_ libCLBI -> do
    let libDeps = map fst $ componentPackageDeps libCLBI
    case dependencyClosure (installedPkgs lbi) libDeps of
      Left p  -> p `seq` putStrLn "Found dependency closure"
      Right _ -> error "Broken dependency closure"

(In reality, singletons actually uses the computed dependency closure for something useful, but I've elided this for the sake of keeping things simple.)

[RyanGlScott]

For comparison, it succeeds with cabal-install-2.2:

$ /opt/cabal/2.2/bin/cabal new-haddock cabal-gh5423:tests
Resolving dependencies...
Build profile: -w ghc-8.4.3 -O1
In order, the following will be built (use -v for more details):
 - cabal-gh5423-0.1 (first run)
[1 of 1] Compiling Main             ( /home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/setup/setup.hs, /home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/setup/Main.o )
Linking /home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/setup/setup ...
Configuring cabal-gh5423-0.1...
Found dependency closure
Preprocessing library for cabal-gh5423-0.1..
Running Haddock on library for cabal-gh5423-0.1..
Haddock coverage:
   0% (  0 /  1) in 'CabalSandbox'
  Missing documentation for:
    Module header
Documentation created:
/home/rgscott/Documents/Hacking/Haskell/cabal-gh5423/dist-newstyle/build/x86_64-linux/ghc-8.4.3/cabal-gh5423-0.1/doc/html/cabal-gh5423/index.html
Preprocessing test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..

[RyanGlScott]

I cannot appear to reproduce this on cabal-install-2.4.0.0/Cabal-2.4.0.1, so perhaps this is no longer an issue?

[hvr]

maybe this was fixed as a side-effect of #5253

[23Skidoo]

Optimistically closing, please reopen if the issue resurfaces.

[RyanGlScott]

Actually, this is still an issue, but it requires particular circumstances to trigger it. In particular, you need:

• The cabal v2-haddock all --haddock-all command,
• A custom Setup.hs script that calls dependencyClosure, and
• A version of cabal-install that links against a different major version of Cabal than what is bundled with the GHC version that you are using.

Here is a demonstration of the bug using modern versions of cabal-install and GHC. I can trigger the bug with:

• GHC 9.4.1 (which bundles Cabal-3.8.1.0) and cabal-install-3.6.2.0 (which links against Cabal-3.6.2.0):

  $ cabal-3.6.2.0.exe v2-haddock all --haddock-all -w ghc-9.4.1
  Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.6.2.0 supports
  'ghc' version < 9.4): C:\ghcup\bin\ghc-9.4.1.exe is version 9.4.1
  Resolving dependencies...
  Build profile: -w ghc-9.4.1 -O1
  In order, the following will be built (use -v for more details):
   - cabal-gh5423-0.1 (first run)
  [1 of 2] Compiling Main             ( C:\\Users\winferno\Documents\Hacking\Haskell\cabal-gh5423\dist-newstyle\build\x86_64-windows\ghc-9.4.1\cabal-gh5423-0.1\setup\setup.hs, C:\\Users\winferno\Documents\Hacking\Haskell\cabal-gh5423\dist-newstyle\build\x86_64-windows\ghc-9.4.1\cabal-gh5423-0.1\setup\Main.o )
  [2 of 2] Linking C:\\Users\\winferno\\Documents\\Hacking\\Haskell\\cabal-gh5423\\dist-newstyle\\build\\x86_64-windows\\ghc-9.4.1\\cabal-gh5423-0.1\\setup\\setup.exe
  Configuring cabal-gh5423-0.1...
  Found dependency closure
  Preprocessing library for cabal-gh5423-0.1..
  Running Haddock on library for cabal-gh5423-0.1..
  Warning: The documentation for the following packages are not installed. No
  links will be generated to these packages: base-4.17.0.0, ghc-bignum-1.3,
  ghc-prim-0.9.0
     0% (  0 /  1) in 'CabalSandbox'
    Missing documentation for:
      Module header
  Documentation created:
  C:\Users\winferno\Documents\Hacking\Haskell\cabal-gh5423\dist-newstyle\build\x86_64-windows\ghc-9.4.1\cabal-gh5423-0.1\doc\html\cabal-gh5423\index.html
  Preprocessing test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..
  Running Haddock on test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..
  setup.exe: internal error when calculating transitive package dependencies.
  Debug info: []
  
  cabal-3.6.2.0.exe: Failed to build documentation for cabal-gh5423-0.1.
  

• GHC 9.2.4 (which bundles Cabal-3.6.3.0) and cabal-install-3.8.1.0 (which links against Cabal-3.8.1.0):

  $ cabal-3.8.1.0.exe v2-haddock all --haddock-all -w ghc-9.2.4
  Resolving dependencies...
  Build profile: -w ghc-9.2.4 -O1
  In order, the following will be built (use -v for more details):
   - cabal-gh5423-0.1 (first run)
  [1 of 1] Compiling Main             ( C:\\Users\winferno\Documents\Hacking\Haskell\cabal-gh5423\dist-newstyle\build\x86_64-windows\ghc-9.2.4\cabal-gh5423-0.1\setup\setup.hs, C:\\Users\winferno\Documents\Hacking\Haskell\cabal-gh5423\dist-newstyle\build\x86_64-windows\ghc-9.2.4\cabal-gh5423-0.1\setup\Main.o )
  Linking C:\\Users\\winferno\\Documents\\Hacking\\Haskell\\cabal-gh5423\\dist-newstyle\\build\\x86_64-windows\\ghc-9.2.4\\cabal-gh5423-0.1\\setup\\setup.exe ...
  Configuring cabal-gh5423-0.1...
  Found dependency closure
  Preprocessing library for cabal-gh5423-0.1..
  Running Haddock on library for cabal-gh5423-0.1..
  Warning: The documentation for the following packages are not installed. No
  links will be generated to these packages: base-4.16.3.0, ghc-bignum-1.2,
  ghc-prim-0.8.0
     0% (  0 /  1) in 'CabalSandbox'
    Missing documentation for:
      Module header
  Documentation created:
  C:\Users\winferno\Documents\Hacking\Haskell\cabal-gh5423\dist-newstyle\build\x86_64-windows\ghc-9.2.4\cabal-gh5423-0.1\doc\html\cabal-gh5423\index.html
  Preprocessing test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..
  Running Haddock on test suite 'cabal-gh5423-tests' for cabal-gh5423-0.1..
  setup.exe: internal error when calculating transitive package dependencies.
  Debug info: []
  
  Error: cabal-3.8.1.0.exe: Failed to build documentation for cabal-gh5423-0.1.
  

However, note that the bug does not occur if I use:

• GHC 9.4.1 (which bundles Cabal-3.8.1.0) and cabal-install-3.8.1.0 (which links against Cabal-3.8.1.0)
• GHC 9.2.4 (which bundles Cabal-3.6.3.0) and cabal-install-3.6.2.0 (which links against Cabal-3.6.2.0)

This is likely why I didn't notice the bug in #5423 (comment), as I switched from using cabal-install-2.3.0.0 to cabal-install-2.4.0.0, the latter of links against the same major version of Cabal (2.4.0.1) as the version of Cabal that GHC 8.4.3 bundles.

[RyanGlScott]

This issue is, unfortunately, preventing me from building singletons-base on CI at the moment. singletons-base uses a custom Setup.hs script involving dependencyClosure, and if I try to combine GHC 9.4.1 with cabal-install-3.6.2.0, then things break as above when building the Haddocks using v2-haddock all --haddock-all.

The closest existing issue that I can find to this is #1919, which has been fixed upstream (see #7827). It's not clear to me if I need to update the custom Setup.hs script somehow in order to avoid #1919. (In fact, another custom Setup.hs script user was wondering the same thing.)