Some libc functions e.g. fopen accept or return char*. However, on Windows, they don't understand utf-8, instead they speak ANSI.
If the function returns char* which contains non-ascii, utf-8 assertion error rises. See #9418 and #9618 for actual cases. (They may be fixed by #9812)
If the function accepts char* parameter but we put non-ascii utf8, then the api fails to understand it thus returns error.
To fix it, we must avoid them and use UTF-16 alternatives e.g. _wfopen on Windows.
This requires converting utf8 into utf16 before calling libc apis.
Similarly, WinAPI with A postfix uses ANSI. We must use W functions instead.
Here is list of problematic points I know (far from complete):
std::io uses fopen. (I don't think we should fix it now since it will be replaced by std::rt::io; libuv accepts utf8 and internally calls W-apis so it is safe)std::os::env uses GetEnvironmentStringsA and FreeEnvironmentStringsA.std::os::rename_file uses libc::rename. This will fail to rename ☆.txt to ★.txt on Windows.
Some libc functions e.g.
fopenaccept or returnchar*. However, on Windows, they don't understand utf-8, instead they speak ANSI.If the function returns
char*which contains non-ascii, utf-8 assertion error rises. See #9418 and #9618 for actual cases. (They may be fixed by #9812)If the function accepts
char*parameter but we put non-ascii utf8, then the api fails to understand it thus returns error.To fix it, we must avoid them and use UTF-16 alternatives e.g.
_wfopenon Windows.This requires converting utf8 into utf16 before calling libc apis.
Similarly, WinAPI with
Apostfix uses ANSI. We must useWfunctions instead.Here is list of problematic points I know (far from complete):
std::iousesfopen. (I don't think we should fix it now since it will be replaced bystd::rt::io; libuv accepts utf8 and internally callsW-apis so it is safe)std::os::envusesGetEnvironmentStringsAandFreeEnvironmentStringsA.std::os::rename_fileuseslibc::rename. This will fail to rename☆.txtto★.txton Windows.