Unable to use keyboard on Unreal Engine game

[aneutron]

Hey !

First of all thanks for the amazing work. I noticed the keyboard is usable in a normal manner, and it works for apps such as Messenger, etc., and it works with Hearthstone too. However, it does not work with PUBG Mobile.

I just noticed this, and thought that maybe it's worth reporting, because on the one hand it could be something with Unreal Engine, but it could also be a bug in scrcpy.

Anyway, you're doing amazing work, thanks a lot !

[rom1v]

However, it does not work with PUBG Mobile.

I just installed the game on some device, it works for me. I can type text for example in the "team chat".

[aneutron]

Yes but the controls don't work. I've seen people using OTG keyboard and mice on YouTube to play.

[pete1414]

Any progress on this? Is there a way for ScrCpy to say an external OTG keyboard is connected? Some Apps don't recognize the keyboard because it's looking for an OTG or bluetooth keyboard.

[rom1v]

Is there a way for ScrCpy to say an external OTG keyboard is connected?

One possibility would be using HID over AOA to generate HID events.

I already implemented HID events forwarding (i.e. forward raw events from an USB keyboard or mouse) in the past (here is a sample project), but generating the HID events from SDL events requires some work (including reading the USB HID spec).

And like audio support in scrcpy, this would only work on Linux (which is still great).

This could also be interesting as a separate project: this would make the computer keyboard act like an OTG keyboard.

[pete1414]

Please see screenshot below. Top half is key events from Scrcpy, bottom half is keyboard events from bluetooth keyboard. In both situations I held down the 'w' key'. It seems Scrcpy sends Key up before it is released.

[pete1414]

I have a work around for this, not fully sure of the implications but it at least works for me in PUBG.
I am not too familiar with github, so I'll just explain what I did.
in scrcpy.c
I removed SDL_TEXTINPUT from the case statement.
I copied the call in KEYUP to KEYDOWN.
Then in Convert.c I added more key mappings to include alpha keys and number keys and a few others..

It now works in PUBG, I can also still type normally and paste text.. things that were handled in text input will not work like volume up etc, they could easily be moved to the key handling event, but I haven't bothered.

I have not thoroughly tested this.. really just wanted it to work for games and now it does.. no longer have to connect a bluetooth keyboard.

This applies to #122 and #127 as well I believe.

[rom1v]

Cool 👍

I am not too familiar with github, so I'll just explain what I did.

You should click on "fork", then push your branch to your github fork and post the link here :)

Then in Convert.c I added more key mappings to include alpha keys and number keys and a few others..

What is your keyboard layout? What if you press Shift+, (for example, on a French keyboard)?

I initially implemented key events using keycodes only (before scrcpy was published), but after reading documentation and discussions on irc #sdl, I think there is no way to work independently of your keyboard layout. Maybe I am wrong though.

things that were handled in text input will not work like volume up etc, they could easily be moved to the key handling event, but I haven't bothered.

On dev branch, volume shortcuts have been changed to Ctrl+up and Ctrl+down without text input.

[pete1414]

I don't see a mapping for "<" in the android keycodes.h, which is what my Shift + "," would be. But I think as long as a mapping is created you could use any layout.

Also, thanks to you for creating this! The app is amazing and I can't say that enough. I am an original kickstarter backer for the 'superbook' and this might replace my need for it... all though I am still looking forward to it.

[rom1v]

But I think as long as a mapping is created you could use any layout.

I agree, but it may not work with all layout without remapping (while with text input events it should work).

You said:

Would it be easier to have the user switch between modes? ie, press Ctrl G to enter 'game' mode. Press again to use text mode.

This is a possibility. 👍

[rom1v]

I added a command-line option: (see next comment)

scrcpy --raw-key-events

or:

scrcpy -k

Please try the rawkeyevents branch. If it works as expected, I will merge it into dev.

press Ctrl G to enter 'game' mode

I decided to use a command-line option, because there is no clear feedback about the current mode, which may be confusing if someone enables it without noticing.

EDIT: not sure I will keep it that way. Maybe I will remove the option and bind it to a shortcut. What do you think?