Bug Report: Control+Space not sent to terminal emulator.

[christianparpart]

Environment

Windows build number: Version 10.0.18362.356
Windows Terminal version 0.4.2382.0

Any other software? yes, **mine** (also a terminal emulator)

Steps to reproduce

Best way to see this (if you don't have key tracing on in our own terminal app),
fire up a TMUX session, have it configured with at least one more line, like this:

bind ^space next-window

which basically states, that Ctrl+Space will cause the terminal to switch to the next window.

Except, that on Windows Terminal (or my terminal emulator, using ConPTY), it doesn't. :-)

Expected behavior

TMUX windows switching (i.e.: NUL-byte sent to PTY slave)

Actual behavior

Action Ignored.

[zadjii-msft]

I'm going to guess this is a combo of:

• Shift+Enter always submits, breaking PSReadline features [Windows.Terminal] #530
• Some keybindings used by PSReadLine aren't getting through #879

There might be some discussion here if conpty should interpret a single NUL byte as Ctrl+Space. Isn't Ctrl+@ also NUL though? There might just be no way to handle that, if we can't be sure what it was to begin with.

[christianparpart]

So I tested the following terminals:

• xterm
• Linux console
• Konsole (KDE)
• gnome-terminal (VTE)
• urxvt
• OS/X's standard terminal

And all of them interpret Ctrl+Space as NUL character.

After long digging I finally found where I was first reading about it.

he NULL character (code 0) is represented by Ctrl-@, "@" being the code immediately
before "A" in the ASCII character set.
For convenience, a lot of terminals accept Ctrl-Space as an alias for Ctrl-@.

(Reference: Wikipedia)

In the VT520 docs I sadly didn't find it with crossreading, but in VT220 docs (link found via Chapter 3.2.5) it looks like it is what I was looking for - even though it's not talking about Ctrl+@.

But in case of doubt, I'd go with what most (see list above) terminal emulators do. Mapping Ctrl+Space to emit a NUL byte. :)

EDIT: added OS/X terminal to the list.

[zadjii-msft]

Ah but see the trick isn't in the Ctrl+Space -> NUL conversion. That's easy. The problem comes from conpty - the layer that's responsible for translating *nix-isms from the terminal into Windows-isms. We need to be able to build an INPUT_RECORD from the input that's sent from the terminal to conpty, so commandline applications can read that INPUT_RECORD. When commandline applications are attached to conpty, they still assume that they're able to read full INPUT_RECORDs from the input, not just a stream of characters. The problem comes from the fact that we can't be sure if NUL meant Ctrl+Space or Ctrl+@. If we also added the appropriate Ctrl+@ -> NUL mapping to the terminal, then conpty would be unable to differentiate between those keypresses.

I guess we could just presume one of them, with a note that it's by design, and hope for more robust input encoding to be able to support the other keystroke.

[christianparpart]

Hey @zadjii-msft, thx for the quick response. I am not a Windows-isms-dev, so I can't talk much about Windows internals, but how I'd see that, is, that Ctrl+space, A..Z, [, , ], ^, _ is being translated to C0 control codes.

I'm not sure you can simply construct an INPUT_RECORD that contains a C0 byte.

typedef struct _KEY_EVENT_RECORD {
  BOOL  bKeyDown;
  WORD  wRepeatCount;
  WORD  wVirtualKeyCode;
  WORD  wVirtualScanCode;
  union {
    WCHAR UnicodeChar;
    CHAR  AsciiChar;
  } uChar;
  DWORD dwControlKeyState;
} KEY_EVENT_RECORD;

Maybe KEY_EVENT_RECORD.AsciiChar could do the trick, since C0 is part of ASCII, that's how I'd understand that. p.s.: Please don't shoot at me, because I'm referencing the MSDN's KEY_EVENT_RECORD struct. :-)

The problem comes from the fact that we can't be sure if NUL meant Ctrl+Space or Ctrl+@. If we also added the appropriate Ctrl+@

WRT that, I don't think you really have to (from a console application's point of view, as - on the above terminal emulators, you can't either).

What part could I be missing?

[zadjii-msft]

So you're definitely on the right track here, but I'll point you straight at the meat of the code we're talking about: https://github.com/microsoft/terminal/blob/master/src/terminal/parser/InputStateMachineEngine.cpp

The InputStateMachineEngine is the part of conpty that's responsible for translating a stream of characters into a stream of INPUT_RECORDs. You can see here us attempting to manually translate a bunch of other control keys already. Or here, where we're building the INPUT_RECORDs.

We absolutely could use the UnicodeChar/AcsiiChar part of the struct to hold the C0 char. That's true. However, the job of conpty is to try and maintain compatibility for Windows-like programs. So for something like powershell.exe, vim.exe, far manager, etc, these are all programs that only know the world through the Win32 Console API. When they're running in a conpty session, they still need to be able to read input as if they were running in a real console window. That's where the InputStateMachineEngine comes in. It takes the other C0 characters and translates them into full-fledged INPUT_RECORDs, similar to the records that would be generated by typing input directly into the console window.
So, we'll translate them into not into a singular INPUT_RECORD holding only that char, but a series of records, indicating a full sequence of keypresses. For something like Ctrl+H, we'll generate sequences akin to "ctrl down", "h down (with char=0x8)", "h up", "ctrl up". If a win32 console application wants to use these Win32 style keys, they can without any changes to their code. For something like WSL that doesn't actually care about these keys, and only cares about the chars, we'll actually do another translation from INPUT_RECORD back to a sequence of chars, but that's a entirely separate discussion 😉.

So we can do this translation, and we can just pick one of Ctrl+Space or Ctrl+@ to be the translation for NUL, but we'd have to document it as such, because Windows client applications would no longer be able to differentiate between the two when running in conpty. I suppose that's not such a big deal, considering that neither works at all right now, but we'd still have to document it clearly, otherwise we'd inevitably get bug reports that say "when I press Ctrl+@ in the Terminal powershell thinks I pressed Ctrl+Space".

[christianparpart]

Thanks for the detailed explanation. I keep forgetting it's Windows OS, nah. When it comes to documentation, I'd go with what most common terminal emulators to at the time of this writing (I try hard being objectively here :-D).

But OTOH, I think maybe the #1173 could probably mitigate this issue?

[zadjii-msft]

You're definitely right. #1173 will also apply to input to the client application, as well as output.