Erlang/OTP 26 hangs when stdio is set utf-8 encoding without a terminal on Windows

[josevalim]

Describe the bug

Take this module:

-module(foo).
-compile(export_all).

run() ->
  io:put_chars("START\n"),

  open_port(
    {spawn_executable, os:find_executable("erl")},
    [use_stdio, exit_status, binary, hide,
     {args, ["-eval", "io:setopts(standard_io, [{encoding, utf8}]), io:write(hello), erlang:halt()."]}]
  ),

  loop().

loop() ->
  receive
    {_, {data, Data}} ->
      io:put_chars(Data),
      loop();

    {_, {exit_status, _}} ->
      erlang:halt()
  end.

If you call erl -s foo run on Windows, it will hang because the open_port never completes. The io:write(hello) operation in the subprocess hangs once we set the encoding to io:setopts(standard_io, [{encoding, utf8}]). This only happens on Windows and only if we set the device to [{encoding, utf8}].

Affected versions
Erlang/OTP 26.

[josevalim]

For completeness, this issue also manifests when invoking rebar3 without an attached terminal on Windows.