Issues around th warning (`This HLS binary does not support Template Haskell`) and build hls with dynamic linking

[jneira]

After adding the warning about TH a bunch of issues have come up:

• The instructions to build in the troubleshooting guide were incomplete, it did not include specific commands for stack or ghcup compile hls: i tried to fix that with Post 1.6.0.0 fixes and prepare 1.6.1.0 bug fix release #2647
• Users with env where th was working fine has been compelled to build hls from source only to take ride of the warning (but well, we should have expected this i guess).
• If you build hls with --enable-executable-dynamic (which links dynamically only the system libs) instead of --ghc-options="-dynamic" (which additionally links dynamically all haskell deps) the warning does not go away. However i thought the first option was sufficient to workaround th problems (maybe i am wrong but i infer that from comments like this)
• As reported in irc, stack install haskell-language-server --ghc-options="dynamic" seems to not work out of the box. It seems you have to build it both statically and dynamically (i am trying to reproduce)
• An user in macOS has got this error trying cabal install pkg:haskell-language-server --ghc-options="-dynamic":

src/Distribution/Extra/Doctest.hs:34:8: error:
    Could not find module ‘Prelude’
    Perhaps you haven't installed the "p_dyn" libraries for package ‘base-4.14.3.0’?
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
34 | module Distribution.Extra.Doctest (

[jneira]

As reported in irc, stack install haskell-language-server --ghc-options="dynamic" seems to not work out of the box. It seems you have to build it both statically and dynamically (i am trying to reproduce)

I've reproduced the error in wsl ubuntu with stack install haskell-language-server --ghc-options="-dynamic"

hiedb                             > [1 of 9] Compiling HieDb.Compat
hiedb                             >
hiedb                             > /tmp/stack-6de8f3e7cd60c56b/hiedb-0.4.1.0/src/HieDb/Compat.hs:84:1: error:
hiedb                             >     Bad interface file: /mnt/d/hls/.stack-work/install/x86_64-linux/c3ef80eb51e5bf52e59152c0adb03374e24f1447c1b9109c398861481f8ffd97/8.10.7/lib/x86_64-linux-ghc-8.10.7/hie-compat-0.2.1.1-6OvyfjjnZ4IJE2CvK3kpC3/Compat/HieTypes.hi
hiedb                             >         mismatched interface file ways (wanted "", got "dyn")
hiedb                             >    |
hiedb                             > 84 | import Compat.HieTypes
hiedb                             >    | ^^^^^^^^^^^^^^^^^^^^^^

[jneira]

• If you build hls with --enable-executable-dynamic (which links dynamically only the system libs) instead of --ghc-options="-dynamic" (which additionally links dynamically all haskell deps) the warning does not go away. However i thought the first option was sufficient to workaround th problems (maybe i am wrong but i infer that from comments like this)

@pepeiborra please, could confirm if --enable-executable-dynamic would be sufficient to make th evaluation reliable as commented in #1982 (comment)?
In that case, would be possible change the alert condition to skip it in that case. Build with --enable-executable-dynamic is better imo as you dont have to rebuild all dependencies, only the executable and it seems it causes less problems (like the Perhaps you haven't installed the "p_dyn" libraries mentioned above).

[pepeiborra]

If you build hls with --enable-executable-dynamic (which links dynamically only the system libs) instead of --ghc-options="-dynamic" (which additionally links dynamically all haskell deps)

What is the source for this information? It doesn't match my understanding, which is:

• System libs are always linked dynamically unless --enable-executable-static is used.
• Haskell libs are always linked statically unless --enable-executable-dynamic is used.

the warning does not go away.

But it does in my system! (Mac OS). Have you been able to validate this information yourself?

In that case, would be possible change the alert condition to skip it in that case

The condition is accurate (I think). So if the condition is firing, I think the risk of segfaults still exists.

Perhaps you haven't installed the "p_dyn" libraries for package ‘base-4.14.3.0’?

This is what happens if you try to build both with --enable-executable-dynamic and with profiling at the same time. Unfortunately GHC doesn't ship with compilation artifacts for this mode, and thus it can never work. We can add this information in the HLS user's guide.

[jneira]

What is the source for this information? It doesn't match my understanding, which is:

Firstly an user in discord built hls with --ghc-options="-dynamic" and then with --enable-executable-dynamic and applied ldd to both executables, the first one had all haskell libs and the second one only the system libs. Also the first one caused a full recompilation of all hls dependencies.

Then i got to reproduce locally in wsl ubuntu. Will double check, in ci if possible to get a shareable reproducer

But it does in my system! (Mac OS). Have you been able to validate this information yourself?

Second hand, the same discord user in manjaro checked the first executable made the alert go away and the second one did not and another user in macos checked the alert continued being showed building with --enable-executable-dynamic (it could not check the other case cause they could not build hls with --ghc-options="-dynamic" due to the mentioned error)

Will check myself locally in wsl ubuntu

This is what happens if you try to build both with --enable-executable-dynamic and with profiling at the same time.

It might be, the user is a beginner who probably dont know even how to enable profiling (but maybe it is activated and they were not aware). That was trying to build hls itself to get the wanted dynamically linked executable after checking it out the project. We dont have profiling enabled in the project but maybe it is enabled in the global cabal config 🤔

The entire discord talk is in https://discord.com/channels/280033776820813825/505370075402862594/937443543667056711 and posterior comments

[jneira]

What is the source for this information? It doesn't match my understanding, which is:

* System libs are always linked dynamically unless `--enable-executable-static` is used.

* Haskell libs are always linked statically unless `--enable-executable-dynamic` is used.

see also @hasufell comment about diff between --enable-executable-dynamic and --ghc-options="-dynamic" here: #1431 (comment)

[pepeiborra]

What is the source for this information? It doesn't match my understanding, which is:

Firstly an user in discord built hls with --ghc-options="-dynamic" and then with --enable-executable-dynamic and applied ldd to both executables, the first one had all haskell libs and the second one only the system libs. Also the first one caused a full recompilation of all hls dependencies.

Probably user error. --enable-executable-dynamic definitely links the Haskell libs dynamically.

Then i got to reproduce locally in wsl ubuntu. Will double check, in ci if possible to get a shareable reproducer

But it does in my system! (Mac OS). Have you been able to validate this information yourself?

Second hand, the same discord user in manjaro checked the first executable made the alert go away and the second one did not and another user in macos checked the alert continued being showed building with --enable-executable-dynamic (it could not check the other case cause they could not build hls with --ghc-options="-dynamic" due to the mentioned error)

Building with --ghc-options="-dynamic" is not advisable, as it will try to build every dependency with -dynamic which is unnecessary. This is also what @hasufell was trying to point out.

[hasufell]

Building with --ghc-options="-dynamic" is not advisable

The other way around. --enable-executable-dynamic is broken with cabal v2-install (which ghcup uses), similar to --enable-executable-stripping. There are countless bug reports on cabal about it:

• cabal install/build: static executables and stripping cabal#7236
• Cannot build a statically linked binary with cabal new-* commands cabal#5847

[jneira]

Building with --ghc-options="-dynamic" is not advisable, as it will try to build every dependency with -dynamic which is unnecessary. This is also what @hasufell was trying to point out.

That was also my understanding, however we are suggesting that (ghc-options variant) in the actual troubleshooting guide, precisely due to

The other way around. --enable-executable-dynamic is broken with cabal v2-install (which ghcup uses), similar to --enable-executable-stripping. There are countless bug reports on cabal about it:

And for stack it seems simply there is no --enable-executable-dynamic, although according to docs ghc-options only is applied to local targets by default

[pepeiborra]

I missed that change to the cabal instructions in the troubleshooting section. I don't think that cabal install --ghc-options=-dynamic works at all, so we should revert it and refresh the users guide.

If cabal install --enable-executable-dynamic doesn't work, then that's a Cabal bug and we can report it to the Cabal issue tracker. Has it been reported already @hasufell ?

[hasufell]

I missed that change to the cabal instructions in the troubleshooting section. I don't think that cabal install --ghc-options=-dynamic works at all, so we should revert it and refresh the users guide.

Why would it not work? I'm using that right now.

If cabal install --enable-executable-dynamic doesn't work, then that's a Cabal bug and we can report it to the Cabal issue tracker. Has it been reported already @hasufell ?

I linked two of them here #2659 (comment)

[pepeiborra]

I linked two of them here #2659 (comment)

I am confused: those tickets don't seem to mention --enable-executable-dynamic. So the problem is with cabal install then?

[hasufell]

I linked two of them here #2659 (comment)

I am confused: those tickets don't seem to mention --enable-executable-dynamic. So the problem is with cabal install then?

Those are all the same issue with similar flags.