proposal: x/term: add detection of VT sequences processing support

[perillo]

Proposal Details

VT sequences processing support was added in Windows 10 build 14361.

I propose to add a new API in x/term package for detect if VT sequences are available.
Adding support for VT sequences for Windows is not trivial, so an official implementation may be useful.

As an example, see the implementation in the Zig standard library:

• https://github.com/ziglang/zig/blob/0.14.0/lib/std/fs/File.zig#L249
• https://github.com/ziglang/zig/blob/0.14.0/lib/std/fs/File.zig#L287

See also the official Windows documentation:

• https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences

Currently there is a change request in https://go-review.googlesource.com/557295.

[apparentlymart]

From the proposed changeset I guess this proposal is primarily about detecting whether vt emulation support is already enabled in the console, rather than for explicitly enabling it (using SetConsoleMode).

That seems to me like a reasonable complement to the existing IsTerminal, but I notice in the proposed changeset a small inconsistency that emerges due to the implementation choices: on Windows, SupportsEscapeSequences returns false if the given file descriptor does not refer to a console, but on Unix SupportsEscapeSequences returns true regardless of whether the file descriptor is associated with a terminal.

Might it be reasonable for SupportsEscapeSequences to be effectively just an alias for IsTerminal on Unix platforms? That way programs that wish to dynamically decide whether or not to emit escape sequences can just call SupportsEscapeSequences and get a reasonable answer on all platforms, rather than having to call both IsTerminal and SupportsEscapeSequences to get a correct answer on Unix, which would then cause a redundant extra call to GetConsoleMode on Windows.

[apparentlymart]

Separately from my previous comment, the section Example of Enabling Virtual Terminal Processing implies for me that the recommended approach for an application that wants to use VT sequences is to actively try to enable them, rather than just to check whether they are currently enabled.

Given that, I wonder about instead offering a function func EnableEscapeSequences() error (or similar) which would:

• On Windows, implement a Go equivalent of the C code illustrated in that documentation, returning an error if the given fd isn't a console or if the modesetting fails.
• On Unix, just call IsTerminal and return a predefined error value if it returns false, or nil if it returns true.

A program that prefers to use escape sequences but is able to degrade to not using them could then call EnableEscapeSequences to see if it succeeds, while a program that requires escape sequences (e.g. because it's a fullscreen TUI application) can report the returned error to its user and exit.

---

However, this does have at least one one notable hazard: when I implemented a version of this advice inline in an application I work on I heard a complaint that my program does not restore the previous VT state when it exits, and so if someone subsequently runs legacy software in the same console then it may misbehave due to the console not behaving as expected for the console in legacy Windows versions. Interestingly, I didn't yet find any official advice from Microsoft about whether and how to deal with that situation, and so the software I referred to still leaves the console in the modern vt mode when it exits.

I'm not sure if such behavior is acceptable for a portable standard library function though, since folks accustomed to other platforms might use it without understanding that consequence.

[perillo]

From the proposed changeset I guess this proposal is primarily about detecting whether vt emulation support is already enabled in the console, rather than for explicitly enabling it (using SetConsoleMode).

Yes. I avoided enabling VT emulation because it may cause problems with legacy console on Windows; it seems that you confirmed this problem.

I found this possible problem from ziglang/zig#15206 (comment).

That seems to me like a reasonable complement to the existing IsTerminal, but I notice in the proposed changeset a small inconsistency that emerges due to the implementation choices: on Windows, SupportsEscapeSequences returns false if the given file descriptor does not refer to a console, but on Unix SupportsEscapeSequences returns true regardless of whether the file descriptor is associated with a terminal.

This is a bug, sorry.

Might it be reasonable for SupportsEscapeSequences to be effectively just an alias for IsTerminal on Unix platforms? That way programs that wish to dynamically decide whether or not to emit escape sequences can just call SupportsEscapeSequences and get a reasonable answer on all platforms, rather than having to call both IsTerminal and SupportsEscapeSequences to get a correct answer on Unix, which would then cause a redundant extra call to GetConsoleMode on Windows.

Yes, this should be the correct behavior.

[perillo]

Separately from my previous comment, the section Example of Enabling Virtual Terminal Processing implies for me that the recommended approach for an application that wants to use VT sequences is to actively try to enable them, rather than just to check whether they are currently enabled.

Given that, I wonder about instead offering a function func EnableEscapeSequences() error (or similar) which would:

Zig standard library have both variants, but the getOrEnableAnsiEscapeSupport function returns a bool, not an error.

This may be added later, if there is interest.

[apparentlymart]

Interestingly, according to ziglang/zig#20172 (comment) there is now something resetting the VT enabled state before running each new program, and so it can apparently be safe to enable it as long as all other processes triggered by a single command are compatible with the vt mode.

That did not seem to be true when I originally tested this, but that was some time ago and so I wouldn't be surprised if the situation has changed to help fix compatibility problems.

I don't have a Windows installation to test with at the moment so I can't confirm the assertions in the comment I linked, but if that is true then I think that makes the hypothetical EnableEscapeSequences less troublesome.

FWIW I think an error result from that function would be more "Go-like" than a bool because it can then potentially describe different reasons why VT mode could not be enabled, such as distinguishing between the fd not representing a console at all vs. it being a console on an older version of Windows that didn't support this mode yet.

I think the "enable" function would be considerably more useful than just a function to test if it's already enabled, but indeed I suppose supporting both is also an option, in which case the enable function could be a separate proposal.