Packages missing documentation on GHC 9.4 & 9.6

[jchia]

Attempting to generate database for local GHC packages fails on GHC versions 9.4.8 & 9.6.4. 9.2.8 is fine.

All the GHC versions in my tests were installed using GHCup. The hoogle is the latest version installed using cabal install hoogle.

hoogle generate -l fails to find the documentation for built-in packages like base and binary under the problematic GHC versions. hoogle generate -h seems to find the documentation when given the html/libraries directory associated with GHC installation, but the resulting database is unsuitable for serving as the generated documentation URLs have wrong paths, so -h is not a suitable workaround.

$ which ghc
/home/jchia/.ghcup/bin/ghc
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.6.4
$ ll /home/jchia/.ghcup/bin/ghc
lrwxrwxrwx 1 jchia jchia 26 Jan 26 18:01 /home/jchia/.ghcup/bin/ghc -> ../ghc/9.6.4/bin/ghc-9.6.4*
$ rm /tmp/dm.hoo; hoogle generate -vvv -l --database /tmp/dm.hoo
Starting generate
Generating files to /tmp
Reading ghc-pkg... 0.03s

Packages missing documentation: array base binary bytestring Cabal Cabal-syntax containers deepseq directory exceptions filepath ghc ghc-bignum ghc-boot ghc-boot-th ghc-compact ghc-heap ghc-prim ghci haskeline hpc integer-gmp libiserv mtl parsec pretty process rts stm system-cxx-std-lib template-haskell terminfo text time transformers unix xhtml
No packages were found, aborting (use no arguments to index all of Stackage)
$ rm /tmp/dm.hoo; hoogle generate -vvv -h ~/.ghcup/ghc/9.6.4/share/doc/ghc-9.6.4/html/libraries/ --database /tmp/dm.hoo
Starting generate
Generating files to /tmp
Reading ghc-pkg... 0.03s
[4/37] base... 1.03s        
[12/37] ghc... 3.52s       
[31/37] transformers... 0.06s    
[33/37] xhtml... 0.03s
Packages missing documentation: ghc-compact integer-gmp rts system-cxx-std-lib
Found 208 warnings when processing items

Reordering items... 0.02s
Writing tags... 0.07s
Writing names... 0.07s
Writing types... 0.33s
Took 7.83s

[jchia]

I notice that in 9.4.8 & 9.6.4, the package database info uses pkgroot as a variable to describe directories like haddock-html, but not 9.2.8.

'pkgroot' does not appear in the hoogle codebase, so perhaps hoogle does not do the necessary string interpolation with pkgroot, resulting in the failure to find the documentation? If so, then there are 2 problems:

• The failure to do the necessary interpolation.
• The failure to log errors about invalid paths, causing hard-to-explain failure to properly generate the database.

I suppose a deeper problem is the lack of an API/library to extract package database information. (AFAIK there is none.)

19:17 ~$ ghc-pkg-9.2.8 --global -v2 describe base-4.16.4.0 | egrep -A1 'haddock-html|^pkgroot'
Timestamp 2023-05-28 17:36:43.842187821 UTC for /home/jchia/.ghcup/ghc/9.2.8/lib/ghc-9.2.8/package.conf.d/package.cache
haddock-html:
    /home/jchia/.ghcup/ghc/9.2.8/share/doc/ghc-9.2.8/html/libraries/base-4.16.4.0
pkgroot: "/home/jchia/.ghcup/ghc/9.2.8/lib/ghc-9.2.8"

19:17 ~$ ghc-pkg-9.6.4 --global -v2 describe base-4.18.2.0 | egrep -A1 'haddock-html|^pkgroot'
Timestamp 2024-01-23 05:17:12.998622263 UTC for /home/jchia/.ghcup/ghc/9.6.4/lib/ghc-9.6.4/lib/package.conf.d/package.cache
haddock-html:
    ${pkgroot}/../../../share/doc/ghc-9.6.4/html/libraries/base-4.18.2.0
pkgroot: "/home/jchia/.ghcup/ghc/9.6.4/lib/ghc-9.6.4/lib"

When I manually hacked the .conf files to interpolate pkgconf, the problem went away:

$ pwd
/home/jchia/.ghcup/ghc/9.6.4/lib/ghc-9.6.4/lib/package.conf.d
$ sed -i 's/${pkgroot}/\/home\/jchia\/.ghcup\/ghc\/9.6.4\/lib\/ghc-9.6.4\/lib/' *.conf

I suppose now one of the main questions is whose job it is to do the interpolation of pkgroot, the GHC installation procedure, ghc-pkg or tools that use ghc-pkg?

[hololeap]

I am also running into this and I have observed the same thing: it worked properly with ghc-9.2.8, but as of ghc-9.4.8, hoogle cannot find any of the bootstrap libraries that get installed alongside GHC.

[hololeap]

I notice that in 9.4.8 & 9.6.4, the package database info uses pkgroot as a variable to describe directories like haddock-html, but not 9.2.8.

---

I noticed the same thing as well. In ghc-9.4.8, ghc-pkg lists haddock-interfaces for bootstrap libraries as a relative path to ${pkgroot}. This wasn't the case in ghc-9.2.8:

• ghc-9.2.8

  •  haddock-interfaces: /usr/share/doc/ghc-9.2.8/html/libraries/base-4.16.4.0/base.haddock
    

• ghc-9.4.8

  •  haddock-interfaces: ${pkgroot}/../../../share/doc/ghc-9.4.8/html/libraries/base-4.17.2.1/base.haddock
    

---

Also, on my 9.4.8 system, I noticed that the haddock interfaces that hoogle can find are listed as absolute paths:

haddock-interfaces:   /usr/share/doc/crypton-0.34/html/crypton.haddock

This supports @jchia's theory that hoogle just isn't parsing the ${pkgroot} paths properly.

[mpickering]

See #405 for a patch which fixes this.