[wasm] Test failing because of not being able to output utf8 characters on windows - `?????` instead of `こんにちは`

[radical]

Seen on #58578
Wasm.Build.Tests.SatelliteAssembliesTests.ResourcesFromMainAssembly, and Wasm.Build.Tests.SatelliteAssembliesTests.ResourcesFromProjectReference:

Assert.Contains() Failure
Not found: got: こんにちは

Actual output: For 'ja-JP' got: ?????

https://dev.azure.com/dnceng/public/_build/results?buildId=1344204&view=ms.vss-test-web.build-test-results-tab&runId=39357506&resultId=100020&paneView=debug

[radical]

cc @radekdoulik

[radical]

These are still failing, tried to enable these in #61725 , build. The console log isn't accessible due to helix issues, but the results xml is. And that shows:

Test: Wasm.Build.Tests.SatelliteAssembliesTests.ResourcesFromMainAssembly(buildArgs: BuildArgs { ProjectName = placeholder, Config = Release, AOT = False, ProjectFileContents = placeholder, ExtraBuildArgs = }, nativeRelink: False, argCulture: \"ja-JP\", expectedOutput: \"got: こんにちは\", host: V8, id: \"qbxa2uv0.w22\"

Failure message:

Assert.Contains() Failure
Not found: got: こんにちは
In value:  [wasm test] [1.0.0-prerelease.22310.1+43e9fe312ac5513edf763877ce9ebf5d57ca9f88] XHarness command issued: wasm test --app=. --output-directory=C:\helix\work\workitem\uploads\xharness-output\logs\qbxa2uv0.w22\V8 --expected-exit-code=42 --js-file=test-main.js --engine=V8 -v trace -- --fetch-random-delay=200 --setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true --setenv=DOTNET_INSTALL_DIR=C:\helix\work\workitem\e\dotnet-workload --setenv=DOTNET_MULTILEVEL_LOOKUP=0 --setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 --setenv=_WasmStrictVersionMatch=true --setenv=MSBuildSDKsPath= --setenv=PATH=C:\helix\work\workitem\e\dotnet-workload;C:\helix\work\workitem\e\dotnet-workload;C:\helix\work\correlation\dotnet-workload;C:\helix\work\correlation\xharness-cli;C:\helix\work\correlation\dotnet-cli;C:\helix\work\correlation\build\emsdk-node/14.18.2_64bit/bin;C:\helix\work\correlation\chrome-win;C:\helix\work\correlation\chromedriver_win32;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Users\ContainerAdministrator\AppData\Local\Microsoft\WindowsApps;C:\Python;C:\python\scripts;C:\Program Files\nodejs\;C:\7z;C:\git\cmd;C:\Users\ContainerAdministrator\AppData\Roaming\npm;C:\Users\ContainerAdministrator\.jsvu --setenv=EM_WORKAROUND_PYTHON_BUG_34780=1 --setenv=WasmNativeStrip=false --setenv=WasmCachePath=C:\Users\ContainerAdministrator\.emscripten-cache --run sat_asm_from_main_asm.dll ja-JP
           [wasm test] [22:19:07] info: 
           [wasm test] 
           [wasm test] [22:19:07] info: Running \"C:\Users\ContainerAdministrator\.jsvu\v8.cmd\" --expose_wasm test-main.js -- --fetch-random-delay=200 --setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true \"--setenv=DOTNET_INSTALL_DIR=C:\helix\work\workitem\e\dotnet-workload\" --setenv=DOTNET_MULTILEVEL_LOOKUP=0 --setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 --setenv=_WasmStrictVersionMatch=true --setenv=MSBuildSDKsPath= \"--setenv=PATH=C:\helix\work\workitem\e\dotnet-workload;C:\helix\work\workitem\e\dotnet-workload;C:\helix\work\correlation\dotnet-workload;C:\helix\work\correlation\xharness-cli;C:\helix\work\correlation\dotnet-cli;C:\helix\work\correlation\build\emsdk-node/14.18.2_64bit/bin;C:\helix\work\correlation\chrome-win;C:\helix\work\correlation\chromedriver_win32;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Users\ContainerAdministrator\AppData\Local\Microsoft\WindowsApps;C:\Python;C:\python\scripts;C:\Program Files\nodejs\;C:\7z;C:\git\cmd;C:\Users\ContainerAdministrator\AppData\Roaming\npm;C:\Users\ContainerAdministrator\.jsvu\" --setenv=EM_WORKAROUND_PYTHON_BUG_34780=1 --setenv=WasmNativeStrip=false \"--setenv=WasmCachePath=C:\Users\ContainerAdministrator\.emscripten-cache\" --run sat_asm_from_main_asm.dll ja-JP
           [wasm test] 
           [wasm test] [22:19:08] info: Incoming arguments: --fetch-random-delay=200 --setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true --setenv=DOTNET_INSTALL_DIR=C:\helix\work\workitem\e\dotnet-workload --setenv=DOTNET_MULTILEVEL_LOOKUP=0 --setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 --setenv=_WasmStrictVersionMatch=true --setenv=MSBuildSDKsPath= --setenv=PATH=C:\helix\work\workitem\e\dotnet-workload;C:\helix\work\workitem\e\dotnet-workload;C:\helix\work\correlation\dotnet-workload;C:\helix\work\correlation\xharness-cli;C:\helix\work\correlation\dotnet-cli;C:\helix\work\correlation\build\emsdk-node/14.18.2_64bit/bin;C:\helix\work\correlation\chrome-win;C:\helix\work\correlation\chromedriver_win32;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Users\ContainerAdministrator\AppData\Local\Microsoft\WindowsApps;C:\Python;C:\python\scripts;C:\Program Files\nodejs\;C:\7z;C:\git\cmd;C:\Users\ContainerAdministrator\AppData\Roaming\npm;C:\Users\ContainerAdministrator\.jsvu --setenv=EM_WORKAROUND_PYTHON_BUG_34780=1 --setenv=WasmNativeStrip=false --setenv=WasmCachePath=C:\Users\ContainerAdministrator\.emscripten-cache --run sat_asm_from_main_asm.dll ja-JP
           [wasm test] [22:19:08] info: console.warn: --fetch-random-delay only works on browser
           [wasm test] [22:19:08] info: Application arguments: --run sat_asm_from_main_asm.dll ja-JP
           [wasm test] [22:19:08] info: console.debug: mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
           [wasm test] [22:19:08] info: console.debug: Could not find symbols file dotnet.js.symbols. Ignoring.
           [wasm test] [22:19:08] info: console.info: Initializing.....
           [wasm test] [22:19:08] info: For 'ja-JP' got: ?????
           [wasm test] [22:19:11] info: Process v8.cmd exited with 42
           [wasm test] 
           [wasm test] [22:19:11] info: Application has finished with exit code: 42
           [wasm test] [22:19:11] dbug: Saving diagnostics data to 'C:\helix\work\workitem\e\diagnostics.json'
           [wasm test] XHarness exit code: 0
           [wasm test-stderr] 
           [wasm test]

Trace:
        at Wasm.Build.Tests.SatelliteAssembliesTests.ResourcesFromMainAssembly(BuildArgs buildArgs, Boolean nativeRelink, String argCulture, String expectedOutput, RunHost host, String id) in /_/src/tests/BuildWasmApps/Wasm.Build.Tests/SatelliteAssembliesTests.cs:line 71
        at InvokeStub_SatelliteAssembliesTests.ResourcesFromMainAssembly(Object, Object, IntPtr*)
        at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)]]>

============== wasm test =============

---------- Running with wasm test ---------
Running C:\helix\work\workitem\e\\dotnet-workload\dotnet.exe C:\helix\work\correlation\microsoft.dotnet.xharness.cli\1.0.0-prerelease.22310.1\tools\net7.0\any\Microsoft.DotNet.XHarness.CLI.dll wasm test --app=. --output-directory=C:\helix\work\workitem\uploads\xharness-output\logs\qbxa2uv0.w22\V8 --expected-exit-code=42 --js-file=test-main.js --engine=V8 -v trace --  --fetch-random-delay=200 "--setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true" "--setenv=DOTNET_INSTALL_DIR=C:\helix\work\workitem\e\\dotnet-workload" "--setenv=DOTNET_MULTILEVEL_LOOKUP=0" "--setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1" "--setenv=_WasmStrictVersionMatch=true" "--setenv=MSBuildSDKsPath=" "--setenv=PATH=C:\helix\work\workitem\e\\dotnet-workload;C:\helix\work\workitem\e\\dotnet-workload;C:\helix\work\correlation\dotnet-workload;C:\helix\work\correlation\xharness-cli;C:\helix\work\correlation\dotnet-cli;C:\helix\work\correlation\build\emsdk-node/14.18.2_64bit/bin;C:\helix\work\correlation\chrome-win;C:\helix\work\correlation\chromedriver_win32;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Users\ContainerAdministrator\AppData\Local\Microsoft\WindowsApps;C:\Python;C:\python\scripts;C:\Program Files\nodejs\;C:\7z;C:\git\cmd;C:\Users\ContainerAdministrator\AppData\Roaming\npm;C:\Users\ContainerAdministrator\.jsvu" "--setenv=EM_WORKAROUND_PYTHON_BUG_34780=1" "--setenv=WasmNativeStrip=false" "--setenv=WasmCachePath=C:\Users\ContainerAdministrator\.emscripten-cache" --run sat_asm_from_main_asm.dll ja-JP
WorkingDirectory: C:\helix\work\workitem\e\yrx1j0ua.iak\bin\Release\net7.0\browser-wasm\AppBundle
Setting environment variables for execution:
        XHARNESS_DISABLE_COLORED_OUTPUT = true
        DOTNET_INSTALL_DIR = C:\helix\work\workitem\e\\dotnet-workload
        DOTNET_MULTILEVEL_LOOKUP = 0
        DOTNET_SKIP_FIRST_TIME_EXPERIENCE = 1
        _WasmStrictVersionMatch = true
        MSBuildSDKsPath =
        PATH = C:\helix\work\workitem\e\\dotnet-workload;C:\helix\work\workitem\e\\dotnet-workload;C:\helix\work\correlation\dotnet-workload;C:\helix\work\correlation\xharness-cli;C:\helix\work\correlation\dotnet-cli;C:\helix\work\correlation\build\emsdk-node/14.18.2_64bit/bin;C:\helix\work\correlation\chrome-win;C:\helix\work\correlation\chromedriver_win32;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Users\ContainerAdministrator\AppData\Local\Microsoft\WindowsApps;C:\Python;C:\python\scripts;C:\Program Files\nodejs\;C:\7z;C:\git\cmd;C:\Users\ContainerAdministrator\AppData\Roaming\npm;C:\Users\ContainerAdministrator\.jsvu
        EM_WORKAROUND_PYTHON_BUG_34780 = 1
        WasmNativeStrip = false
        WasmCachePath = C:\Users\ContainerAdministrator\.emscripten-cache
[wasm test] [1.0.0-prerelease.22310.1+43e9fe312ac5513edf763877ce9ebf5d57ca9f88] XHarness command issued: wasm test --app=. --output-directory=C:\helix\work\workitem\uploads\xharness-output\logs\qbxa2uv0.w22\V8 --expected-exit-code=42 --js-file=test-main.js --engine=V8 -v trace -- --fetch-random-delay=200 --setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true --setenv=DOTNET_INSTALL_DIR=C:\helix\work\workitem\e\\dotnet-workload --setenv=DOTNET_MULTILEVEL_LOOKUP=0 --setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 --setenv=_WasmStrictVersionMatch=true --setenv=MSBuildSDKsPath= --setenv=PATH=C:\helix\work\workitem\e\\dotnet-workload;C:\helix\work\workitem\e\\dotnet-workload;C:\helix\work\correlation\dotnet-workload;C:\helix\work\correlation\xharness-cli;C:\helix\work\correlation\dotnet-cli;C:\helix\work\correlation\build\emsdk-node/14.18.2_64bit/bin;C:\helix\work\correlation\chrome-win;C:\helix\work\correlation\chromedriver_win32;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Users\ContainerAdministrator\AppData\Local\Microsoft\WindowsApps;C:\Python;C:\python\scripts;C:\Program Files\nodejs\;C:\7z;C:\git\cmd;C:\Users\ContainerAdministrator\AppData\Roaming\npm;C:\Users\ContainerAdministrator\.jsvu --setenv=EM_WORKAROUND_PYTHON_BUG_34780=1 --setenv=WasmNativeStrip=false --setenv=WasmCachePath=C:\Users\ContainerAdministrator\.emscripten-cache --run sat_asm_from_main_asm.dll ja-JP
[wasm test] [22:19:07] info:
[wasm test]
[wasm test] [22:19:07] info: Running "C:\\Users\\ContainerAdministrator\\.jsvu\\v8.cmd" --expose_wasm test-main.js -- --fetch-random-delay=200 --setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true "--setenv=DOTNET_INSTALL_DIR=C:\\helix\\work\\workitem\\e\\\\dotnet-workload" --setenv=DOTNET_MULTILEVEL_LOOKUP=0 --setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 --setenv=_WasmStrictVersionMatch=true --setenv=MSBuildSDKsPath= "--setenv=PATH=C:\\helix\\work\\workitem\\e\\\\dotnet-workload;C:\\helix\\work\\workitem\\e\\\\dotnet-workload;C:\\helix\\work\\correlation\\dotnet-workload;C:\\helix\\work\\correlation\\xharness-cli;C:\\helix\\work\\correlation\\dotnet-cli;C:\\helix\\work\\correlation\\build\\emsdk-node/14.18.2_64bit/bin;C:\\helix\\work\\correlation\\chrome-win;C:\\helix\\work\\correlation\\chromedriver_win32;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Users\\ContainerAdministrator\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Python;C:\\python\\scripts;C:\\Program Files\\nodejs\\;C:\\7z;C:\\git\\cmd;C:\\Users\\ContainerAdministrator\\AppData\\Roaming\\npm;C:\\Users\\ContainerAdministrator\\.jsvu" --setenv=EM_WORKAROUND_PYTHON_BUG_34780=1 --setenv=WasmNativeStrip=false "--setenv=WasmCachePath=C:\\Users\\ContainerAdministrator\\.emscripten-cache" --run sat_asm_from_main_asm.dll ja-JP
[wasm test]
[wasm test] [22:19:08] info: Incoming arguments: --fetch-random-delay=200 --setenv=XHARNESS_DISABLE_COLORED_OUTPUT=true --setenv=DOTNET_INSTALL_DIR=C:\\helix\\work\\workitem\\e\\\\dotnet-workload --setenv=DOTNET_MULTILEVEL_LOOKUP=0 --setenv=DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 --setenv=_WasmStrictVersionMatch=true --setenv=MSBuildSDKsPath= --setenv=PATH=C:\\helix\\work\\workitem\\e\\\\dotnet-workload;C:\\helix\\work\\workitem\\e\\\\dotnet-workload;C:\\helix\\work\\correlation\\dotnet-workload;C:\\helix\\work\\correlation\\xharness-cli;C:\\helix\\work\\correlation\\dotnet-cli;C:\\helix\\work\\correlation\\build\\emsdk-node/14.18.2_64bit/bin;C:\\helix\\work\\correlation\\chrome-win;C:\\helix\\work\\correlation\\chromedriver_win32;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Users\\ContainerAdministrator\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Python;C:\\python\\scripts;C:\\Program Files\\nodejs\\;C:\\7z;C:\\git\\cmd;C:\\Users\\ContainerAdministrator\\AppData\\Roaming\\npm;C:\\Users\\ContainerAdministrator\\.jsvu --setenv=EM_WORKAROUND_PYTHON_BUG_34780=1 --setenv=WasmNativeStrip=false --setenv=WasmCachePath=C:\\Users\\ContainerAdministrator\\.emscripten-cache --run sat_asm_from_main_asm.dll ja-JP
[wasm test] [22:19:08] info: console.warn: --fetch-random-delay only works on browser
[wasm test] [22:19:08] info: Application arguments: --run sat_asm_from_main_asm.dll ja-JP
[wasm test] [22:19:08] info: console.debug: mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
[wasm test] [22:19:08] info: console.debug: Could not find symbols file dotnet.js.symbols. Ignoring.
[wasm test] [22:19:08] info: console.info: Initializing.....
[wasm test] [22:19:08] info: For 'ja-JP' got: ?????
[wasm test] [22:19:11] info: Process v8.cmd exited with 42
[wasm test]
[wasm test] [22:19:11] info: Application has finished with exit code: 42
[wasm test] [22:19:11] dbug: Saving diagnostics data to 'C:\helix\work\workitem\e\diagnostics.json'
[wasm test] XHarness exit code: 0

These are failing with, and without AOT.

@radekdoulik @pavelsavara @lewing would it make sense to use chcp 65001 for this?

[radical]

Actually, chcp shouldn't be needed, since we are invoking dotnet.exe directly to run the test app. So, encoding needs to be set on the Process class used for this. I think dotnet/runtime@f7c018d (#70595) should work.