evaluate plugin error hPutChar: invalid argument (cannot encode character '\8216')

[Delli]

Your environment

Which OS do you use?
windows 10

Which version of GHC do you use and how did you install it?
ghc 9.8.1 / tried other versions 9.2.2 same issue

How is your project built (alternative: link to the project)?
stack build

Which LSP client (editor/plugin) do you use?

vscode latest + extension Haskell v2.4.1

Which version of HLS do you use and how did you install it?
hls v 2.4.0.0
ghcup installed it

Have you configured HLS in any way (especially: a hie.yaml file)?
no

Steps to reproduce

stack new project

in any hs file

try add evaluation
{-| double

double 2
: hPutChar: invalid argument (cannot encode character '\8216')
-}

double :: Num a => a -> a
double x = (2*x)

Expected behaviour

{-| double

double 2
4
-}

Actual behaviour

{-| double

double 2
: hPutChar: invalid argument (cannot encode character '\8216')
-}

Debug information

[Bodigrim]

Year 2023, Unicode in Windows strikes again.

This is likely https://gitlab.haskell.org/ghc/ghc/-/commit/74132c2b0992849f83ef87c8a56ac3975738e767

[fendor]

Thank you for the context!

Does that mean, if we set the environment variable GHC_NO_UNICODE in HLS, these kinds of issues shouldn't happen in the eval plugin?
Or should we mirror ghc's logic in determining the unicode support and only then set the environment variable?

[Delli]

thx , confirm solved the issue on my machine
@ fendor what i did

set system variable GHC_CHARENC to value GHC_NO_UNICODE in windows os

[Delli]

@fendor

To set the environment variable GHC_CHARENC for GHC, you can use the following steps:

Windows:

Open the Control Panel.
Click on "System and Security".
Click on "System".
Click on "Advanced system settings".
In the "System Properties" window, click on the "Environment Variables" button.
In the "Environment Variables" window, under "System variables", find the variable GHC_CHARENC. If it does not exist, click on "New" and create it.
Set the value of the variable to the desired character encoding.
Click on "OK" to save the changes.

[Bodigrim]

@fendor yes, I suggest to set GHC_NO_UNICODE unconditionally, at least on Windows. I think fancy Unicode quotes \8216 in GHC output are ill-advised in the first place.

[lorenzschreyer]

Simple Fix on Windows:

1. go to Settings > Time & language > Language & region > Administrative language settings > Change system locale.
2. check the Beta: Use Unicode UTF-8 ...
3. reboot

[geraldus]

Faced with similar problem after migrating to macOS Sonoma 14.2.1.

Caused by LC_ALL not being set. Fixed by explicitly setting this environment variable in my ZSH profile

[unlocked2412]

Hi @geraldus. I am having the same error on my macOS Sonoma 14.2.1. Could you tell me the steps you took to solve it please ?

I added this line

export GHC_CHARENC=GHC_NO_UNICODE

to my ~/.zshrc profile but no luck.

[geraldus]

@unlocked2412 Hi! Sorry for the late reply. I checked my locale output first:

locale
LANG="ru_RU.UTF-8"
LC_COLLATE="ru_RU.UTF-8"
LC_CTYPE="ru_RU.UTF-8"
LC_MESSAGES="ru_RU.UTF-8"
LC_MONETARY="ru_RU.UTF-8"
LC_NUMERIC="ru_RU.UTF-8"
LC_TIME="ru_RU.UTF-8"
LC_ALL=

As you can see, no LC_ALL value was set. So I've added the following to my ~/.zshprofile:

if [[ -z "$LC_ALL" ]]; then
  export LC_ALL='en_US.UTF-8'
fi

And restarted the shell session.

[chansey97]

@Delli From https://ghc.gitlab.haskell.org/ghc/doc/users_guide/using.html?highlight=ghc_charenc#envvar-GHC_NO_UNICODE there are two environment variables.

GHC_NO_UNICODE

When non-empty, disables Unicode diagnostics output regardless of locale settings. GHC can usually determine that locale is not Unicode-capable and fallback to ASCII automatically, but in some corner cases (e. g., when GHC output is redirected) you might hit invalid argument (cannot encode character '\8216'), in which case do set GHC_NO_UNICODE.

GHC_CHARENC

When set to UTF-8 the compiler will always print UTF-8-encoded output, regardless of the current locale.

If I understand correctly, if some one want to disable Unicode diagnostics output, they should set something like export GHC_NO_UNICODE=true and don't set GHC_CHARENC.

What is the meaning of GHC_CHARENC=GHC_NO_UNICODE? (I guess you might set the wrong variable ?)

[akilmd0]

I have tried everything still getting the same error
my locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

[Sofviic]

I'm using WSL2 (Ubuntu), and setting the locale in the in there (eg in the .bashrc file) didn't work.

@lorenzschreyer 's solution worked for me: (#3841 (comment))