[Linux] Keyboard not working in several games

[Sidefix]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

• This issue is present in the latest pre-release

Describe the Bug

I have configured Sunshine on a Linux Ubuntu 2310 installation, working perfectly as intended in most games.

For example, Cyberpunk 2077, Armello, Rimworld, Deep Rock Galactic, Dying Light 2, all work fine.

However there is an issue in a very small handful of games that I have not been able to identify a link for. In Against the Storm, Death Must Die and Songs of Conquest, the keyboard input simply does not work.

The keyboard is fine and interacts outside of Sunshine with no issues, or in other games as stated above. It does not matter if I start game X first or last or any combination. In some games the keyboard input simply refuses to work.

Expected Behavior

The keyboard should work as intended with any application.

Additional Context

Ubuntu 23.10
Sunshine 0.23.1 (issue occurred with older versions as well)
Proton-GE 9-5 (issue occurred with older versions as well)
X11 capture
Nvidia driver 555.42.02 (issue occurred with older versions as well)

Host Operating System

Linux

Operating System Version

23.10

Architecture

64 bit

Sunshine commit or version

0.23.1

Package

Linux - deb

GPU Type

Nvidia

GPU Model

RTXA2000

GPU Driver/Mesa Version

555.42.02

Capture Method (Linux Only)

X11

Config

log_path = /home/u/sunshine/configs/sunshine.log
nv_preset = p1
sunshine_name = ReVision
channels = 2
nvenc_preset = 7
fps = [60]
file_apps = /home/u/sunshine/configs/apps.json
resolutions = [
    1440x900
]
file_state = /home/u/sunshine/configs/sunshine_state.json
nv_rc = vbr
ping_timeout = 30000
min_log_level = 5
output_name = 0
global_prep_cmd = [{"do":"","undo":""}]
origin_web_ui_allowed = pc
credentials_file = /home/u/sunshine/configs/sunshine_state.json
nvenc_twopass = full_res
nvenc_spatial_aq = enabled

Apps

No response

Relevant log output

n/a

[Sidefix]

I should add that only the keyboard is affected, and so far only the games mentioned above have been identified to be affected.

Mouse works perfectly fine in everything.

[gschintgen]

Please post the regular logs (at level info) nonetheless.

At loglevel "debug" sunshine logs every mouse movement and every keypress. Please inspect those logs too and post an excerpt. (Annotated with what you pressed and if it worked or not.)

If I remember correctly there are some X-specific codepaths, so you could also try changing the capture method to KMS. As a second test you could also switch (temporarily) to Wayland instead of X (again with KMS).

[ReenigneArcher]

There is a new input library that we're getting ready to migrate to. It might be worth testing that PR.

[Sidefix]

@gschintgen

[2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000003] keyCode [804A] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8048] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8043] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8045] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8049] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [804E] modifiers [00] flags [00] --end keyboard packet--

Looks like the keyboard packets are sent... hmm.

@ReenigneArcher do you mean the latest nightly/dev build?

[gschintgen]

@gschintgen

[2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000003] keyCode [804A] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8048] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8043] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8045] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [8049] modifiers [00] flags [00] --end keyboard packet-- [2024:06:03:12:30:27]: Debug: --begin keyboard packet-- keyAction [00000004] keyCode [804E] modifiers [00] flags [00] --end keyboard packet--

So the issue is on the receiving end, which is not really surprising. Did you check with a different display server or capture method? (BTW are there any free games that are affected?)

Looks like the keyboard packets are sent... hmm.

@ReenigneArcher do you mean the latest nightly/dev build?

The new input library should be the one from #2606 . Test builds can be retrieved from github actions. Here's the appimage for example: https://github.com/LizardByte/Sunshine/actions/runs/9378855742/artifacts/1569733361 (AppImages should work fine on Ubuntu.)

[Sidefix]

Reaching back here, I was not able to get it to work with newer builds (including latest Dev v2024.730.191523).

I'll try to change capture methods, although I'm not sure if I can switch the display server. I'm currently using an X headless setup, and I have zero experience with wayland.

I'll post updated results soon.

Edit with results:

Absolutely all capture methods, including Automatic, have the same issue with the same games.
My current display server is X on ubuntu 24.04, all packages up to date and upgraded.

I've also tried playing around with ibus (uninstalling/stopping) and with PROTON XIM (enabling/disabling) with no changes in behavior.

Any additional help would be greatly appreciated.

[gschintgen]

Maybe this? https://steamcommunity.com/discussions/forum/1/1843566500527286941/ In particular if you have Moonlight set to always connect a virtual controller. If it's working with most games it seems unlikely to be an issue with Sunshine itself.

…

On August 1, 2024 8:51:20 PM GMT+02:00, Sidefix ***@***.***> wrote: Reaching back here, I was not able to get it to work with newer builds (including latest Dev v2024.730.191523). I'll try to change capture methods, although I'm not sure if I can switch the display server. I'm currently using an X headless setup, and I have zero experience with wayland. I'll post updated results soon. -- Reply to this email directly or view it on GitHub: #2614 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

[Sidefix]

Thanks for the thought.

I have a laptop and tried to unplug everything, and the machine running sunshine has nothing plugged in either other than power. Same issue.

I also tried toggling various input settings from sunshine, still nothing.

I agree that it's more likely something and sunshine is just stuck in the middle. I'll try to reach out to the proton community, maybe they have more ideas. Thanks for the help so far.

[Sidefix]

Further investigations:

The issue does not occur when playing the games locally. The issue only occurs when streaming through Sunshine.

So the issue is somewhere in the flow.

My flow is: Sunshine (which we've confirmed is registering the keyboard events) >>> X server display on :2 >>> Proton-GE >>> game.exe

No further conclusions. Will report back here with additional findings.

[gschintgen]

Did you connect a controller during those local tests?

…

On August 2, 2024 2:31:55 PM GMT+02:00, Sidefix ***@***.***> wrote: Further investigations: The issue _**does not**_ occur when playing the games locally. The issue only occurs when streaming through Sunshine. So the issue is somewhere in the flow. My flow is: Sunshine (which we've confirmed is registering the keyboard events) >>> X server display on :2 >>> Proton-GE >>> game.exe No further conclusions. Will report back here with additional findings. -- Reply to this email directly or view it on GitHub: #2614 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

[Sidefix]

No, no accessories were connected during the tests.

Edit: obviously the mouse + keyboard were connected to the device during the local test; but no other accessories were.

[gschintgen]

It would be nice to know though, in light of the comment on Reddit that some games seem to ignore keyboard input if a controller is connected. In that case it could be sufficient to ensure that Sunshine is not adding an unwanted virtual controller.

…

On August 2, 2024 2:49:45 PM GMT+02:00, Sidefix ***@***.***> wrote: No, no accessories were connected during the tests. -- Reply to this email directly or view it on GitHub: #2614 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

[Sidefix]

@gschintgen I'm not sure I understand your suggestion; are you saying I should try with a controller connected on the host machine or