GHC 9.4, stack and multiple components causes hidden package errors

[TeofilC]

After upgrading to GHC-9.4.5, we've been noticing a lot more of the -package ... is hidden errors in HLS with a stack multi-cradle.
Initially I thought this was merely #366. But after further testing it's clear that this is caused by something else. #366 is caused by stack repl failing to pass the correct arguments when components aren't already built. But this error also happens when they have successfully built.

I then compared the dynflags that cabal and stack produce for the same project. I noticed that cabal passed -this-unit-id while stack did not. Adding -this-unit-id <package-name> to ghc-options for each package made the new problem go away.

So, my theory is that because stack repl doesn't pass -this-unit-id to GHC, something with GHC's new MHU breaks, causing only the dependencies of one package to be exposed in HLS's GHC API session.

My impression is that this issue will occur when using HLS, with stack, GHC 9.4, and any session with multiple components loaded.

I've created a somewhat minimal reproducer below.

Your environment

Which OS do you use? NixOS

Which version of GHC do you use and how did you install it? GHC 9.4.5 using nix

How is your project built (alternative: link to the project)? stack

Which LSP client (editor/plugin) do you use? emacs+lsp-mode

Which version of HLS do you use and how did you install it? 1.10.0 and 2.0.0 (built with nix)

Have you configured HLS in any way (especially: a hie.yaml file)? yes, hie.yaml generated by gen-hie

Steps to reproduce

I've created a somewhat minimal reproducer: https://github.com/TeofilC/hls-repro-1
It features two packages that have different dependencies.
When opening both in a single HLS session with stack, HLS fails with a -package is hidden error.

1. git clone https://github.com/TeofilC/hls-repro-1.git
2. cd hls-repro-1
3. nix develop (if you use nix or get stack/hls some other way)
4. stack build
5. haskell-language-server -d pack-a/src/Lib1.hs pack-b/src/Lib2.hs

If using nix make sure you have the following in your .stack/config:

nix:
  enable: false
system-ghc: true

Expected behaviour

HLS should be able to load these modules and can do if we use cabal instead.

Actual behaviour

It fails:

Message: 
  Could not load module ‘Data.These’
  It is a member of the hidden package ‘these-1.2’.
  You can run ‘:set -package these’ to expose it.
  (Note: this unloads all the modules in the current scope.)
2023-05-28T15:11:31.379037Z | Debug | Finished: User TypeCheck Took: 0.03s
2023-05-28T15:11:31.379893Z | Debug | Finished: GetHie Took: 0.00s
2023-05-28T15:11:31.380154Z | Debug | LOOKUP PERSISTENT FOR: GhcSessionDeps
2023-05-28T15:11:31.380246Z | Debug | Finished: GenerateCore Took: 0.00s
Files that failed:
 * ./pack-b/src/Lib2.hs

Completed (1 file worked, 1 file failed)

Debug information

[michaelpj]

cc @wz1000 ? I think it sounds like there may be a stack bug here, but possibly also we should try and be more robust somehow? not sure.

[fendor]

@michaelpj We have identified this issue before: #366 (comment) and #1822 (comment)

However, maybe @wz1000 has an idea for fixing this. Or we try to fix it upstream.

[TeofilC]

Nice! Feel free to close this as a duplicate if this is being tracked elsewhere

[cherryblossom000]

I have the same issue. My package.yaml looks something like this

name: foo
version: 0.0.1

dependencies:
- base

library:
  source-dirs: src
  dependencies:
  - megaparsec
  - parser-combinators
  - text

executable:
  source-dirs: app
  main: Main.hs

tests:
  foo-test:
    main: Spec.hs
    source-dirs: test
    dependencies:
    - foo
    - megaparsec
    - tasty
    - tasty-hunit
    - text

and I was getting the missing package for tasty in my test files.

A workaround I’m using is to add these missing dependencies to the top-level dependencies property:

dependencies:
- base
- tasty
- tasty-hunit

While this isn’t ideal as it adds those dependencies to all components, at least HLS works correctly with this config.