IO.getEnv returns invalid UTF-8 on windows

[digama0]

Minimized from this Zulip thread. It appears that std::getenv on windows returns data in the current code page, so putting it directly into a lean String is not legal (because it assumes UTF-8 encoding). Other languages have faced the same issue:

• Nim: getEnv doesn't return UTF-8 on Windows nim-lang/Nim#20083
• Rust: implementation

Both of these languages take the approach of using the widestring version of the function (which is reliably UTF-16 (with potentially unpaired surrogates)) and manually converting it to UTF-8 (or WTF-8). This is a bit sad, I thought the A version of the function was supposed to be UTF-8. (Maybe it is, I can't test it.)

[Kha]

Is this #2554?

[digama0]

Unclear. (The issue description is a bit vague.) But I don't think so; from what I can tell #668 is about a build system issue, not runtime usage of getEnv and related functions.

[Kha]

No, it is not a build system issue. We set the code page of lean to UTF-8 but not yet for other Lean executables, so seems very relevant here. So your

I thought the A version of the function was supposed to be UTF-8

is indeed correct, depending on which executable that code is linked into.

[Kha]

Assuming duplicate of #2554