Regression from .NET 8: `dotnet run` on .NET 9 emits ANSI terminal codes when redirected

[danpere]

Describe the bug

dotnet run > some_file should output only the output of the program being run to some_file. That is the behavior of .NET SDK 8. .NET SDK 9 instead prefixes that output with .]9;4;3;.\.]9;4;0;.\ (hex 1b5d 393b 343b 333b 1b5c 1b5d 393b 343b 303b 1b5c), the Windows Terminal ANSI escape sequence for "set progress indicator to indeterminate, then clear progress indicator". This breaks essentially any script that redirects the output of dotnet run and performs any kind of analysis on it.

To Reproduce

dotnet new console # Generate the default C# console project that prints "Hello, World!".
dotnet run > dotnet-redirect # Run, redirecting output to file.
script -q /dev/null -c 'dotnet run' > dotnet-script # Run with terminal output, but actually send that to a file.

Then examine those files with xxd (required because cating ANSI escape sequences to the console won't display them):

xxd dotnet-redirect
xxd dotnet-script

While the second one is expected to have escape sequences, observe that the first one does as well. Here's the output on both .NET 8 and .NET 9:

.NET 9 using > redirect (bad)

00000000: 1b5d 393b 343b 333b 1b5c 1b5d 393b 343b  .]9;4;3;.\.]9;4;
00000010: 303b 1b5c 4865 6c6c 6f2c 2057 6f72 6c64  0;.\Hello, World
00000020: 210a                                     !.

.NET 8 using > redirect (expected)

00000000: 4865 6c6c 6f2c 2057 6f72 6c64 210a       Hello, World!.

And for reference, here's the output using script. Note that the output is not identical, so clearly dotnet run is detecting that its output is being redirected, but maybe that isn't getting to all of the code outputting to the console?

.NET 9 using script

00000000: 1b5b 3f31 681b 3d1b 5d39 3b34 3b33 3b1b  .[?1h.=.]9;4;3;.
00000010: 5c1b 5b3f 3235 6c1b 5b31 460d 0a1b 5b3f  \.[?25l.[1F...[?
00000020: 3235 681b 5b3f 3235 6c1b 5b31 460d 0a1b  25h.[?25l.[1F...
00000030: 5b3f 3235 681b 5b3f 3235 6c1b 5b31 460d  [?25h.[?25l.[1F.
00000040: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b31  ..[?25h.[?25l.[1
00000050: 460d 0a1b 5b3f 3235 681b 5b3f 3235 6c1b  F...[?25h.[?25l.
00000060: 5b31 460d 0a1b 5b3f 3235 681b 5b3f 3235  [1F...[?25h.[?25
00000070: 6c1b 5b31 460d 0a1b 5b3f 3235 681b 5b3f  l.[1F...[?25h.[?
00000080: 3235 6c1b 5b31 460d 0a1b 5b3f 3235 681b  25l.[1F...[?25h.
00000090: 5b3f 3235 6c1b 5b31 460d 0a1b 5b3f 3235  [?25l.[1F...[?25
000000a0: 681b 5b3f 3235 6c1b 5b31 460d 0a1b 5b3f  h.[?25l.[1F...[?
000000b0: 3235 681b 5b3f 3235 6c1b 5b31 460d 0a1b  25h.[?25l.[1F...
000000c0: 5b3f 3235 681b 5b3f 3235 6c1b 5b31 460d  [?25h.[?25l.[1F.
000000d0: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b31  ..[?25h.[?25l.[1
000000e0: 460d 0a1b 5b3f 3235 681b 5b3f 3235 6c1b  F...[?25h.[?25l.
000000f0: 5b31 460d 0a20 202f 686f 6d65 2f64 616e  [1F..  /home/dan
00000100: 7065 7265 2f48 656c 6c6f 576f 726c 642f  pere/HelloWorld/
00000110: 4865 6c6c 6f57 6f72 6c64 2e63 7370 726f  HelloWorld.cspro
00000120: 6a20 1b5b 3132 3047 1b5b 3134 4452 6573  j .[120G.[14DRes
00000130: 746f 7265 2028 302e 3073 290d 0a1b 5b3f  tore (0.0s)...[?
00000140: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
00000150: 5b31 3230 471b 5b36 4428 302e 3173 290d  [120G.[6D(0.1s).
00000160: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
00000170: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
00000180: 3173 290d 0a1b 5b3f 3235 681b 5b3f 3235  1s)...[?25h.[?25
00000190: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
000001a0: 4428 302e 3173 290d 0a1b 5b3f 3235 681b  D(0.1s)...[?25h.
000001b0: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
000001c0: 471b 5b36 4428 302e 3273 290d 0a1b 5b3f  G.[6D(0.2s)...[?
000001d0: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
000001e0: 5b31 3230 471b 5b36 4428 302e 3273 290d  [120G.[6D(0.2s).
000001f0: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
00000200: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
00000210: 3273 290d 0a1b 5b3f 3235 681b 5b3f 3235  2s)...[?25h.[?25
00000220: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
00000230: 4428 302e 3373 290d 0a1b 5b3f 3235 681b  D(0.3s)...[?25h.
00000240: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
00000250: 471b 5b36 4428 302e 3373 290d 0a1b 5b3f  G.[6D(0.3s)...[?
00000260: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
00000270: 5b31 3230 471b 5b36 4428 302e 3373 290d  [120G.[6D(0.3s).
00000280: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
00000290: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
000002a0: 3473 290d 0a1b 5b3f 3235 681b 5b3f 3235  4s)...[?25h.[?25
000002b0: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
000002c0: 4428 302e 3473 290d 0a1b 5b3f 3235 681b  D(0.4s)...[?25h.
000002d0: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
000002e0: 471b 5b36 4428 302e 3473 290d 0a1b 5b3f  G.[6D(0.4s)...[?
000002f0: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
00000300: 5b31 3230 471b 5b36 4428 302e 3573 290d  [120G.[6D(0.5s).
00000310: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
00000320: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
00000330: 3573 290d 0a1b 5b3f 3235 681b 5b3f 3235  5s)...[?25h.[?25
00000340: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
00000350: 4428 302e 3573 290d 0a1b 5b3f 3235 681b  D(0.5s)...[?25h.
00000360: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
00000370: 471b 5b36 4428 302e 3673 290d 0a1b 5b3f  G.[6D(0.6s)...[?
00000380: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
00000390: 5b31 3230 471b 5b36 4428 302e 3673 290d  [120G.[6D(0.6s).
000003a0: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
000003b0: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
000003c0: 3673 290d 0a1b 5b3f 3235 681b 5b3f 3235  6s)...[?25h.[?25
000003d0: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
000003e0: 4428 302e 3773 290d 0a1b 5b3f 3235 681b  D(0.7s)...[?25h.
000003f0: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
00000400: 471b 5b36 4428 302e 3773 290d 0a1b 5b3f  G.[6D(0.7s)...[?
00000410: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
00000420: 5b31 3230 471b 5b36 4428 302e 3773 290d  [120G.[6D(0.7s).
00000430: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
00000440: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
00000450: 3873 290d 0a1b 5b3f 3235 681b 5b3f 3235  8s)...[?25h.[?25
00000460: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
00000470: 4428 302e 3873 290d 0a1b 5b3f 3235 681b  D(0.8s)...[?25h.
00000480: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
00000490: 471b 5b36 4428 302e 3873 290d 0a1b 5b3f  G.[6D(0.8s)...[?
000004a0: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
000004b0: 5b31 3230 471b 5b36 4428 302e 3973 290d  [120G.[6D(0.9s).
000004c0: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
000004d0: 460d 0a1b 5b31 3230 471b 5b36 4428 302e  F...[120G.[6D(0.
000004e0: 3973 290d 0a1b 5b3f 3235 681b 5b3f 3235  9s)...[?25h.[?25
000004f0: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
00000500: 4428 302e 3973 290d 0a1b 5b3f 3235 681b  D(0.9s)...[?25h.
00000510: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
00000520: 471b 5b36 4428 312e 3073 290d 0a1b 5b3f  G.[6D(1.0s)...[?
00000530: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
00000540: 5b31 3230 471b 5b36 4428 312e 3073 290d  [120G.[6D(1.0s).
00000550: 0a1b 5b3f 3235 681b 5b3f 3235 6c1b 5b32  ..[?25h.[?25l.[2
00000560: 460d 0a1b 5b31 3230 471b 5b36 4428 312e  F...[120G.[6D(1.
00000570: 3173 290d 0a1b 5b3f 3235 681b 5b3f 3235  1s)...[?25h.[?25
00000580: 6c1b 5b32 460d 0a1b 5b31 3230 471b 5b36  l.[2F...[120G.[6
00000590: 4428 312e 3173 290d 0a1b 5b3f 3235 681b  D(1.1s)...[?25h.
000005a0: 5b3f 3235 6c1b 5b32 460d 0a1b 5b31 3230  [?25l.[2F...[120
000005b0: 471b 5b36 4428 312e 3173 290d 0a1b 5b3f  G.[6D(1.1s)...[?
000005c0: 3235 681b 5b3f 3235 6c1b 5b32 460d 0a1b  25h.[?25l.[2F...
000005d0: 5b31 3230 471b 5b36 4428 312e 3273 290d  [120G.[6D(1.2s).
000005e0: 0a1b 5b3f 3235 681b 5b32 460d 0a1b 5b4a  ..[?25h.[2F...[J
000005f0: 1b5d 393b 343b 303b 1b5c 1b5d 393b 343b  .]9;4;0;.\.]9;4;
00000600: 333b 1b5c 1b5d 393b 343b 303b 1b5c 1b5b  3;.\.]9;4;0;.\.[
00000610: 3f31 681b 3d48 656c 6c6f 2c20 576f 726c  ?1h.=Hello, Worl
00000620: 6421 0d0a 1b5b 3f31 681b 3d              d!...[?1h.=

.NET 8 using script

00000000: 1b5b 3f31 681b 3d1b 5b3f 3168 1b3d 4865  .[?1h.=.[?1h.=He
00000010: 6c6c 6f2c 2057 6f72 6c64 210d 0a1b 5b3f  llo, World!...[?
00000020: 3168 1b3d                                1h.=

Other bugs referencing this issue

Searching for existing issues about this, I found #44442 which appears to possibly be a workaround for this same bug. There's also dotnet/msbuild#10678, which appears to be a fix for this bug... but that PR merged a while ago, and the bug is still happening.

Exceptions (if any)

N/A

Further technical details

• Include the output of dotnet --info

$ dotnet --info
.NET SDK:
 Version:           9.0.100
 Commit:            a2bc464e40
 Workload version:  9.0.100-manifests.6bf02610
 MSBuild version:   17.12.7+a2bc464e4

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  22.04
 OS Platform: Linux
 RID:         ubuntu.22.04-x64
 Base Path:   /usr/lib/dotnet/sdk/9.0.100/

.NET workloads installed:
There are no installed workloads to display.
Configured to use loose manifests when installing new manifests.

Host:
  Version:      9.0.0
  Architecture: x64
  Commit:       a2bc464e40

.NET SDKs installed:
  8.0.110 [/usr/lib/dotnet/sdk]
  9.0.100 [/usr/lib/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.10 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 9.0.0 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.10 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 9.0.0 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

• The IDE (VS / VS Code/ VS4Mac) you're running on, and its version

While the actual bug was originally encountered on Windows, this repro was done under Linux as it was easier to deal with the terminal manipulation details there. Specifically, all of this was done from the WSL command line, running Ubuntu 22.04:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[KalleOlaviNiemitalo]

This is being fixed as dotnet/msbuild#10998.

.NET SDK 9.0.1xx (I guess 9.0.101) is changed not to use TerminalLogger in that situation:

• [release/9.0.1xx] Using ConsoleLogger to prevent unexpected behavior #45015

For .NET SDK 9.0.2xx, there is a more complex fix pending, which lets MSBuild choose ConsoleLogger or TerminalLogger, but it needs a Visual Studio update too:

• [17.12] Adding a static factory for the TerminalLogger msbuild#11016
• [release/9.0.2xx] Use the static factory for creating a logger #45009

[marcpopMSFT]

Let's track this using the msbuild issue.