HLS seems confused about the source directory

[chris-martin]

I'm not sure how to describe the problem except that HLS doesn't seem to understand that haskell/typeclasses/src is the source directory.

See the popup error message indicating that the Typeclasses.Postgres.Connection.Concepts cannot be found, and the file tree on the left where you can see it's right there - and also the erroneous suggestion at the top to rename the open module to haskell.typeclasses.src.Typeclasses.Postgres.Connection.PoolSetup.

I'm a bit lost because I haven't found much documentation for using the "direct" configuration option. I'm not sure whether this is supported, or if the problem is with hie-bios, or what. I've always used nix and various scripts to launch ghci and to build this project, so I have no cabal file. I don't entirely understand, is using HLS without a cabal file meant to be supported?

$ haskell-language-server-wrapper --probe-tools
haskell-language-server version: 1.1.0.0 (GHC: 8.10.4) (PATH: /nix/store/6k9vz0jiivzdifmfjn4s6r3bb9q0y4ca-haskell-language-server-1.1.0.0/bin/haskell-language-server-wrapper)
Tool versions found on the $PATH
cabal:          Not found
stack:          Not found
ghc:            8.10.4

NixOS, VSCode.

hie.yaml:

cradle:
  multi:
    - path: "./haskell/typeclasses/src"
      config:
        cradle:
          direct:
            arguments:
              - "-XApplicativeDo"
              - "-XBlockArguments"
              - "-XDeriveFunctor"
              - "-XDeriveAnyClass"
              - "-XDeriveDataTypeable"
              - "-XDeriveFoldable"
              - "-XDeriveGeneric"
              - "-XDeriveLift"
              - "-XDerivingStrategies"
              - "-XDerivingVia"
              - "-XFunctionalDependencies"
              - "-XGeneralizedNewtypeDeriving"
              - "-XInstanceSigs"
              - "-XLambdaCase"
              - "-XNamedFieldPuns"
              - "-XNoImplicitPrelude"
              - "-XNumericUnderscores"
              - "-XOverloadedStrings"
              - "-XPartialTypeSignatures"
              - "-XPatternSynonyms"
              - "-XQuasiQuotes"
              - "-XRecordWildCards"
              - "-XScopedTypeVariables"
              - "-XStandaloneDeriving"
              - "-XTemplateHaskell"
              - "-XTypeApplications"
              - "-XViewPatterns"

I have also tried adding -ihaskell/typeclasses/src to the flag list, but it doesn't help.

What I see in the vscode output panel:

[client] run command: "/home/chris/typeclasses/haskell-language-server.hs --lsp"
[client] debug command: "/home/chris/typeclasses/haskell-language-server.hs --lsp"
[client] server cwd: undefined
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
haskell-language-server version: 1.1.0.0 (GHC: 8.10.4) (PATH: /nix/store/6k9vz0jiivzdifmfjn4s6r3bb9q0y4ca-haskell-language-server-1.1.0.0/bin/haskell-language-server)
Starting (haskell-language-server)LSP server...
  with arguments: GhcideArguments {argsCommand = LSP, argsCwd = Nothing, argsShakeProfiling = Nothing, argsTesting = False, argsExamplePlugin = False, argsDebugOn = False, argsLogFile = Nothing, argsThreads = 0, argsProjectGhcVersion = False}
  with plugins: [PluginId "pragmas",PluginId "floskell",PluginId "fourmolu",PluginId "tactics",PluginId "ormolu",PluginId "stylish-haskell",PluginId "retrie",PluginId "brittany",PluginId "class",PluginId "haddockComments",PluginId "eval",PluginId "importLens",PluginId "moduleName",PluginId "hlint",PluginId "splice",PluginId "ghcide-hover-and-symbols",PluginId "ghcide-code-actions",PluginId "ghcide-completions",PluginId "ghcide-type-lenses",PluginId "ghcide-core"]
  in directory: /home/chris/typeclasses
 Starting LSP server...
If you are seeing this in a terminal, you probably should have run WITHOUT the --lsp option!
Started LSP server in 0.00s
setInitialDynFlags cradle: Cradle {cradleRootDir = "/home/chris/typeclasses", cradleOptsProg = CradleAction: Multi}
2021-06-08 13:09:37.125141805 [ThreadId 5] INFO hls:	Registering ide configuration: IdeConfiguration {workspaceFolders = fromList [NormalizedUri (-435663548406963376) "file:///home/chris/typeclasses"], clientSettings = hashed Nothing}
2021-06-08 13:09:37.134612595 [ThreadId 99] INFO hls:	Consulting the cradle for "haskell/typeclasses/src/Typeclasses/Postgres/Connection/PoolSetup.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/home/chris/typeclasses", cradleOptsProg = CradleAction: Multi}
2021-06-08 13:09:37.225230392 [ThreadId 99] INFO hls:	Using interface files cache dir: /home/chris/.cache/ghcide/main-68a12c08e7c0786a7ac06c80be21f439766d1574
2021-06-08 13:09:37.225389311 [ThreadId 99] INFO hls:	Making new HscEnv[main]
2021-06-08 13:09:37.248532256 [ThreadId 190] INFO hls:	finish: splice.codeAction.GitHieAst (took 0.00s)
2021-06-08 13:09:37.248508279 [ThreadId 179] INFO hls:	finish: Pragmas.GetParsedModule (took 0.00s)
2021-06-08 13:09:37.248551184 [ThreadId 185] INFO hls:	finish: HaddockComments.GetAnnotatedParsedSource (took 0.00s)
2021-06-08 13:09:37.24852285 [ThreadId 188] INFO hls:	finish: importLens (took 0.00s)
2021-06-08 13:09:37.364360385 [ThreadId 203] INFO hls:	finish: retrie (took 0.12s)
2021-06-08 13:09:37.364436275 [ThreadId 208] INFO hls:	finish: MinimalImports (took 0.12s)
2021-06-08 13:09:37.365348008 [ThreadId 212] INFO hls:	finish: CodeAction (took 0.12s)
2021-06-08 13:09:37.372925271 [ThreadId 209] INFO hls:	finish: tactic (took 0.13s)
2021-06-08 13:09:37.389338927 [ThreadId 221] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:09:37.389445 [ThreadId 223] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:09:37.389494655 [ThreadId 225] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:09:37.38991386 [ThreadId 229] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:09:37.390014209 [ThreadId 227] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:09:37.390191341 [ThreadId 231] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:09:37.390718018 [ThreadId 233] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:09:37.620325503 [ThreadId 250] INFO hls:	finish: Pragmas.GetParsedModule (took 0.00s)
2021-06-08 13:09:37.620437196 [ThreadId 252] INFO hls:	finish: CodeAction (took 0.00s)
2021-06-08 13:09:37.620444715 [ThreadId 256] INFO hls:	finish: importLens (took 0.00s)
2021-06-08 13:09:37.620583609 [ThreadId 260] INFO hls:	finish: splice.codeAction.GitHieAst (took 0.00s)
2021-06-08 13:09:37.620612087 [ThreadId 258] INFO hls:	finish: tactic (took 0.00s)
2021-06-08 13:09:37.620628207 [ThreadId 261] INFO hls:	finish: retrie (took 0.00s)
2021-06-08 13:09:37.620632078 [ThreadId 259] INFO hls:	finish: MinimalImports (took 0.00s)
2021-06-08 13:09:37.620755962 [ThreadId 254] INFO hls:	finish: HaddockComments.GetAnnotatedParsedSource (took 0.00s)
2021-06-08 13:12:34.637356651 [ThreadId 273] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:12:34.63742108 [ThreadId 275] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:12:34.637640738 [ThreadId 279] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:12:34.637686957 [ThreadId 277] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:12:34.637821478 [ThreadId 281] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:12:34.638129364 [ThreadId 283] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:12:34.638150355 [ThreadId 285] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:12:35.824979388 [ThreadId 313] INFO hls:	finish: CodeAction (took 0.00s)
2021-06-08 13:12:35.825044643 [ThreadId 315] INFO hls:	finish: HaddockComments.GetAnnotatedParsedSource (took 0.00s)
2021-06-08 13:12:35.825075716 [ThreadId 317] INFO hls:	finish: tactic (took 0.00s)
2021-06-08 13:12:35.825128668 [ThreadId 319] INFO hls:	finish: splice.codeAction.GitHieAst (took 0.00s)
2021-06-08 13:12:35.825084297 [ThreadId 321] INFO hls:	finish: importLens (took 0.00s)
2021-06-08 13:12:35.825232596 [ThreadId 322] INFO hls:	finish: MinimalImports (took 0.00s)
2021-06-08 13:12:35.825899394 [ThreadId 324] INFO hls:	finish: retrie (took 0.00s)
2021-06-08 13:12:35.82590273 [ThreadId 311] INFO hls:	finish: Pragmas.GetParsedModule (took 0.00s)
2021-06-08 13:12:42.72449145 [ThreadId 343] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:12:42.724685889 [ThreadId 347] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:12:42.72486953 [ThreadId 351] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:12:42.724930835 [ThreadId 345] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:12:42.724842883 [ThreadId 349] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:12:42.725102292 [ThreadId 353] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:12:42.7257802 [ThreadId 355] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:12:44.474467357 [ThreadId 379] INFO hls:	finish: Pragmas.GetParsedModule (took 0.00s)
2021-06-08 13:12:44.474624175 [ThreadId 381] INFO hls:	finish: HaddockComments.GetAnnotatedParsedSource (took 0.00s)
2021-06-08 13:12:44.474789944 [ThreadId 385] INFO hls:	finish: retrie (took 0.00s)
2021-06-08 13:12:44.474598223 [ThreadId 383] INFO hls:	finish: CodeAction (took 0.00s)
2021-06-08 13:12:44.474850625 [ThreadId 387] INFO hls:	finish: splice.codeAction.GitHieAst (took 0.00s)
2021-06-08 13:12:44.474885645 [ThreadId 393] INFO hls:	finish: MinimalImports (took 0.00s)
2021-06-08 13:12:44.474877373 [ThreadId 389] INFO hls:	finish: importLens (took 0.00s)
2021-06-08 13:12:44.47497724 [ThreadId 391] INFO hls:	finish: tactic (took 0.00s)
2021-06-08 13:13:59.416889552 [ThreadId 421] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:13:59.417143544 [ThreadId 423] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:13:59.41732268 [ThreadId 427] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:13:59.417614681 [ThreadId 425] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:13:59.417555266 [ThreadId 429] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:13:59.417881505 [ThreadId 431] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:13:59.418438961 [ThreadId 433] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:14:21.528838663 [ThreadId 449] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:14:21.528838565 [ThreadId 447] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:14:21.528861878 [ThreadId 443] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:14:21.528926722 [ThreadId 445] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:14:21.52904513 [ThreadId 451] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:14:21.529092395 [ThreadId 453] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:14:21.529223638 [ThreadId 455] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:14:22.307373125 [ThreadId 474] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:14:22.307453526 [ThreadId 478] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:14:22.307613724 [ThreadId 476] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:14:22.307752568 [ThreadId 482] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:14:22.307872427 [ThreadId 480] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:14:22.308097066 [ThreadId 484] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:14:22.308634022 [ThreadId 486] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:14:42.968709422 [ThreadId 506] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:14:42.968780507 [ThreadId 510] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:14:42.968906758 [ThreadId 512] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:14:42.968994584 [ThreadId 508] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:14:42.96900364 [ThreadId 514] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:14:42.969109952 [ThreadId 516] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)
2021-06-08 13:14:42.969287051 [ThreadId 518] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:15:38.14166604 [ThreadId 528] INFO hls:	finish: codeLens.TypeCheck (took 0.00s)
2021-06-08 13:15:38.141777356 [ThreadId 530] INFO hls:	finish: parsed (took 0.00s)
2021-06-08 13:15:38.141969994 [ThreadId 534] INFO hls:	finish:  (took 0.00s)
2021-06-08 13:15:38.142167851 [ThreadId 532] INFO hls:	finish: ModuleName.ghcSession (took 0.00s)
2021-06-08 13:15:38.142585588 [ThreadId 536] INFO hls:	finish: codeLens.GetBindings (took 0.00s)
2021-06-08 13:15:38.142672226 [ThreadId 538] INFO hls:	finish: ModuleName.GetParsedModule (took 0.00s)
2021-06-08 13:15:38.142839446 [ThreadId 540] INFO hls:	finish: codeLens.GetGlobalBindingTypeSigs (took 0.00s)

Also, below is the working script that I normally use to launch ghci for this project.

#! /usr/bin/env runhaskell

import qualified Data.List as List
import qualified System.Directory as Dir
import qualified System.Process as Proc

main = findSourceFiles >>= \files -> Proc.callProcess "ghci" (flags ++ files)

flags =
    [ "-no-user-package-db"
    , "-ignore-dot-ghci"
    , "-Wall"
    , "-fdefer-typed-holes"
    , "-Werror=missing-fields"
    ]
    ++ hieFlags
    ++ extensionFlags

hieFlags =
    [ "-fwrite-ide-info"
    , "-hiedir=.hie"
    ]

extensionFlags = map ("-X" ++) extensions

extensions =
    [ "ApplicativeDo"
    , "BlockArguments"
    , "DeriveFunctor"
    , "DeriveAnyClass"
    , "DeriveDataTypeable"
    , "DeriveFoldable"
    , "DeriveGeneric"
    , "DeriveLift"
    , "DerivingStrategies"
    , "DerivingVia"
    , "FunctionalDependencies"
    , "GeneralizedNewtypeDeriving"
    , "InstanceSigs"
    , "LambdaCase"
    , "NamedFieldPuns"
    , "NoImplicitPrelude"
    , "NumericUnderscores"
    , "OverloadedStrings"
    , "PartialTypeSignatures"
    , "PatternSynonyms"
    , "QuasiQuotes"
    , "RecordWildCards"
    , "ScopedTypeVariables"
    , "StandaloneDeriving"
    , "TemplateHaskell"
    , "TypeApplications"
    , "ViewPatterns"
    ]

findSourceFiles = findSourceFilesFrom "haskell/typeclasses/src"

findSourceFilesFrom x =
  do
    isDir <- Dir.doesDirectoryExist x
    isFile <- Dir.doesFileExist x
    let isHsFile = isFile && (".hs" `List.isSuffixOf` x)
    case () of
        _ | isHsFile -> return [x]
        _ | isDir    -> Dir.listDirectory x >>= foldMap (\y -> findSourceFilesFrom (x ++ "/" ++ y))
        _            -> return []

[pepeiborra]

A direct cradle must enumerate all the source files - a -isrc flag is not enough. I'm sorry that you wasted time on this, please feel free to submit a documentation patch either to HLS or to hie-bios if this requirement is not indicated clearly.

Why are you using a direct cradle instead of letting HLS infer the default cradle for your project?

[chris-martin]

Why are you using a direct cradle instead of letting HLS infer the default cradle for your project?

I don't understand what that means. How do I specify my language flags without specifying a "direct" cradle? What is a "default cradle"? The documentation does not mention this.

[chris-martin]

I tried adapting my ghci script to turn it into an hie-bios program instead:

#! /usr/bin/env runhaskell

import qualified Data.List as List
import qualified System.Directory as Dir
import qualified System.Environment as Env
import qualified System.IO as IO
import qualified System.Process as Proc

main =
  do
    files <- findSourceFiles
    out <- Env.getEnv "HIE_BIOS_OUTPUT"
    IO.writeFile out (unlines (flags ++ files))

flags =
    [ "-no-user-package-db"
    , "-ignore-dot-ghci"
    , "-Wall"
    , "-fdefer-typed-holes"
    , "-Werror=missing-fields"
    , "-ihaskell/typeclasses/src"
    ]
    ++ hieFlags
    ++ extensionFlags

hieFlags =
    [ "-fwrite-ide-info"
    , "-hiedir=.hie"
    ]

extensionFlags = map ("-X" ++) extensions

extensions =
    [ "ApplicativeDo"
    , "BlockArguments"
    , "DeriveFunctor"
    , "DeriveAnyClass"
    , "DeriveDataTypeable"
    , "DeriveFoldable"
    , "DeriveGeneric"
    , "DeriveLift"
    , "DerivingStrategies"
    , "DerivingVia"
    , "FunctionalDependencies"
    , "GeneralizedNewtypeDeriving"
    , "InstanceSigs"
    , "LambdaCase"
    , "NamedFieldPuns"
    , "NoImplicitPrelude"
    , "NumericUnderscores"
    , "OverloadedStrings"
    , "PartialTypeSignatures"
    , "PatternSynonyms"
    , "QuasiQuotes"
    , "RecordWildCards"
    , "ScopedTypeVariables"
    , "StandaloneDeriving"
    , "TemplateHaskell"
    , "TypeApplications"
    , "ViewPatterns"
    ]

findSourceFiles = findSourceFilesFrom "haskell/typeclasses/src"

findSourceFilesFrom x =
  do
    isDir <- Dir.doesDirectoryExist x
    isFile <- Dir.doesFileExist x
    let isHsFile = isFile && (".hs" `List.isSuffixOf` x)
    case () of
        _ | isHsFile -> return [x]
        _ | isDir    -> Dir.listDirectory x >>= foldMap (\y -> findSourceFilesFrom (x ++ "/" ++ y))
        _            -> return []

hie.yaml now looks like this:

cradle:
  multi:
    - path: "./haskell/typeclasses/src"
      config:
        cradle:
          bios:
            program: ./hie-bios.hs

It seems that HLS can now find imported modules, but it still makes a bad recommendation about renaming the module.

[pepeiborra]

In the absence of a cradle, HLS resorts to implicit-hie to autogenerate one. Your language flags can be extracted from the Cabal descriptor, assuming you are using Cabal.

[chris-martin]

As I mentioned above, I have not packaged any of this code for Cabal. Is cabal meant to be a requirement?

[pepeiborra]

Cabal is not a requirement, but if you are not using Cabal like >90% of our users, be prepared to invest a little bit more time learning and working around rough edges.

[chris-martin]

My hie-bios program is now producing output that looks like this:

-ihaskell/typeclasses/src
-XApplicativeDo
-XBlockArguments
-XDeriveFunctor
[...]
Typeclasses.Postgres.Querying
Typeclasses.Postgres.Connection.PoolSetup
Typeclasses.Postgres.Connection.Secrets
[...]

which seems like what it ought to be? Still I'm getting prompted to change every module name.

[pepeiborra]

The prompt to change module names could be a bug in the hls-module-name-plugin, contributed by @Ailrun, but I'm not sure.

You are probably running into it because your bios script is producing module names instead of module paths. Which is fine and supported by ghcide, but both cabal and stack produce module paths so this plugin has probably never been tested with a cradle that outputs module names

[chris-martin]

Should be paths be relative to the top level directory, to the source directory, or absolute? If relative, should I start them with ./?

Is there some example of a result produced by a well-supported build tool, so that I may match my bios output as closely as possible?

hie-bios does mention this:

Note, the program is intended to produce the build flags to compile the whole component the given source file belongs to. This entails that the program lists all of the component's module- and file targets.

but does not give an example.

[jneira]

The main usage of a hie.yaml with a bios cradle i am aware of is ghc itself! It uses hadrian under the hood to produce the flags and modules:

• https://gitlab.haskell.org/ghc/ghc/-/blob/master/hadrian/hie-bios
• https://gitlab.haskell.org/ghc/ghc/-/tree/master/hadrian#integrating-hadrian-into-other-tooling
• https://gitlab.haskell.org/ghc/ghc/-/blob/master/hadrian/src/Rules/ToolArgs.hs

So it is a less used hie-bios cradle but an important one imo.

Maybe the documentation of the bios/direct cradle could be completed with the suggestions from this issue. (cc @fendor)

[chris-martin]

The prompt to change module names could be a bug in the hls-module-name-plugin, contributed by @Ailrun, but I'm not sure.

I haven't been able to find anything named hls-module-name-plugin, am I looking in the wrong place?