Overhaul Keyboard Functionality (For multiple keyboard layouts, and other functionality) #116

williamjohnstone · 2025-01-30T00:34:24Z

This PR is to introduce functionality for different keyboard mappings.

If you would like your mapping to be included in the initial PR, let me know and I will try my best.

Fixes #30

Store is functioning as expected, adding new layouts should be trivial and easily scalable. Implementation is different for each function that uses the keyboard (PasteModal vs Typing in the WebRTC window) these will all require their own testing.

Alters ActionBar.tsx to make the CTRL+ALT+DEL button in the Action Bar PR work again with layered with jetkvm#116 - not pulling into the PR until jetkvm#116 has been integrated, but the my PR will probably just be dropped anyway.

PrinceJunkie · 2025-02-01T10:40:32Z

Please include German Keyboard Layout

…ted settings.tsx to set the keyboard layout properly.

williamjohnstone · 2025-02-02T16:48:14Z

I have found further issues with the way that keyboard functionality is implemented, therefor the scope of work on this PR has been expanded.

I am working on this when I have time, thank you for everyone's continued patience.

williamjohnstone · 2025-02-02T16:53:06Z

This PR will now also resolve #125

…teModal to add more clarity to error message. Begin working on key remapping in WebRTC (working to a reasonable degree).

stylobille · 2025-02-03T13:05:28Z

Thank you for your help on this, I can test French keyboard if / when needed :)

… my changes (YAY!). Also spent 4 hours troubleshooting to find out I didn't realise how useCallback works... :/ Anway, not much longer before work on just the mappings can begin.

Rhaedyr · 2025-02-05T02:56:49Z

I know this may have been referenced, but I have a number of Hyprland systems I would love to use the JetKVM for but being able to use win/hyper key without the OS taking over would be immensely helpful, I would be happy to help test if you get to a testing phase, I might also be able to pitch in if I can find the time as well, Good Luck.

Adding an escape sequence in full screen to exit and send all keystrokes in fullscreen mode to the JetKVM might be a solution. say ctrl+alt+esc to exit fullscreen or the overlay exit button and use current system in windowed mode.

williamjohnstone · 2025-02-06T02:15:04Z

I know this may have been referenced, but I have a number of Hyprland systems I would love to use the JetKVM for but being able to use win/hyper key without the OS taking over would be immensely helpful, I would be happy to help test if you get to a testing phase, I might also be able to pitch in if I can find the time as well, Good Luck.

Adding an escape sequence in full screen to exit and send all keystrokes in fullscreen mode to the JetKVM might be a solution. say ctrl+alt+esc to exit fullscreen or the overlay exit button and use current system in windowed mode.

Yes, working on implementing the Keyboard Web API which should resolve this.

the problem is actually getting the browser to capture the keys, by default they are passed straight to the OS and the browser never sees them, which is why currently they don't function

craxo · 2025-02-06T11:06:12Z

Please add support for Norwegian Bokmål:
https://kbdlayout.info/KBDNO/

Alters ActionBar.tsx to make the CTRL+ALT+DEL button in the Action Bar PR work again with layered with jetkvm#116 - not pulling into the PR until jetkvm#116 has been integrated, but the my PR will probably just be dropped anyway.

CLAassistant · 2025-02-13T13:29:36Z

All committers have signed the CLA.

williamjohnstone · 2025-02-22T22:04:47Z

Just wanted to post an update, to let everyone know I haven't abandoned this, just difficult to find the time to work on it currently.

Nevexo · 2025-02-24T08:01:21Z

Just wanted to post an update, to let everyone know I haven't abandoned this, just difficult to find the time to work on it currently.

Never worry about this, we're all incredibly thankful for the time you do spend on it.
Thanks!

…dded functionality to disable keyboard mapping.

timothystewart6 · 2025-03-14T18:12:33Z

I agree. Once this is done it will provide a good pattern for adding more!

…gs to new settings page, update PasteModal to use new keyboard implemention, dropped spanish mappings.

williamjohnstone · 2025-04-12T16:14:38Z

A very short window of time opened up for me this weekend, so I have completed the following:

Dropped spanish mappings as they were time consuming to test
Dropped Modifier key holding (virtual keyboard) from this PR
Dropped addding keyboard mapping setup to initial JetKVM setup.
Merged changes with the latest dev branch

I have tested my changes as much as I can today, and briefly reviewed all my code. I believe the PR is now ready for reviewing/merging.

Any issues, let me know.

Thank you :)

Novido · 2025-04-14T09:31:01Z

Please add swedish support.
Thank you.

williamjohnstone · 2025-04-14T14:01:56Z

Please add swedish support. Thank you. @Novido

I will not be including any additional layouts in this PR.

williamjohnstone · 2025-04-16T20:34:29Z

@adamshiervani This is now ready for review

TerraD · 2025-04-30T15:35:22Z

I purchased 3 JetKVM, because they seemed the ideal way to access servers from everywhere – I even intended to purchase some more. However non working keyboard mappings are a dealbreaker!

The keyboard of my notebook is Swissgerman – it's layout is pretty far away from a US keyboard. In fact so far away that any real world usage is nearly impossible. And I can simply not carry a US English keyboard along every day only for that usage... In particular any process where passwords are needed will be practically impossible anyway: as I understand it, this would not even be solved if an US English keyboard would be plugged in. Who would be able to type in a strong password like 'xT<þRÒ%UºÜ|<HtDlïuÖ§Ò5K'`¹çY6º' by means of an US keyboard and the Alt-key?

I would gladly help to supply needed information and testing. However as this problem seems to exist for any non US keyboard I do think there needs to be a global solution that allows for adaptation of foreign keyboards without manual adaptation for each keyboard!

In my opinion this issue is not solved as long as we do not have a solution for all keyboards!

williamjohnstone · 2025-04-30T15:39:35Z

@TerraD As far as I know a global solution is not possible. PiKVM also uses keyboard mappings like in this PR. Adding keyboard layouts is trivial and somebody could easily spend time adding most used layouts, I just don't have the time.

toxic0berliner · 2025-04-30T15:48:05Z

Keep calm dude you bought into a Kickstarter and open source projet, comments like that feel entitled and are neither productive nor well received I believe.
Thanks William for the work I hope this gets reviewed quickly so other less talented people like myself can start scratching at it to add new and more layouts like FR for my use case 😉

stylobille · 2025-04-30T16:28:14Z

By the way could you explain somewhere how to add a mapping (if this hasn't been done already)? I would gladly help for french keyboards but I have no idea what to do and I'm not a dev, just an Ops. Le mer. 30 avr. 2025, 17:48, toxic0berliner ***@***.***> a écrit :

…

*toxic0berliner* left a comment (jetkvm/kvm#116) <#116 (comment)> Keep calm dude you bought into a Kickstarter and open source projet, comments like that feel entitled and are neither productive nor well received I believe. Thanks William for the work I hope this gets reviewed quickly so other less talented people like myself can start scratching at it to add new and more layouts like FR for my use case 😉 — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEWMGLJC65IF6IOLUJWKKC324DWEZAVCNFSM6AAAAABWEC5Y4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNBSGQ2DQNJXGU> . You are receiving this because you commented.Message ID: ***@***.***>

dlorch · 2025-04-30T16:37:52Z

@TerraD As far as I know a global solution is not possible. PiKVM also uses keyboard mappings like in this PR. Adding keyboard layouts is trivial and somebody could easily spend time adding most used layouts, I just don't have the time.

Hi @williamjohnstone, I am happy to volunteer as the contributor, maintainer and tester for the de-CH (Swiss German) keyboard layout. I was able to make frontend and make build_dev your code in my environment.

What's the best way to support? Should I create and provide a ui/src/keyboardMappings/layouts/de_CH.ts to you?

dlorch · 2025-04-30T16:40:10Z

@williamjohnstone apologies, I just saw your earlier note on:

I will not be including any additional layouts in this PR.

Then I will wait for your PR to be merged in an create a new PR for de-CH. Thanks for all your great work on this!

dlorch · 2025-04-30T20:25:09Z

ui/src/routes/devices.$id.settings.mouse.tsx

+                { value: "en-US", label: "US" },
+                { value: "en-GB", label: "UK" },
+                { value: "en-GB_apple", label: "UK (Apple)" },
+                { value: "de_DE", label: "German (T1)" },


I think this should be value: "de-DE" to match the value in ui/src/keyboardMappings/KeyboardLayouts.ts

dlorch · 2025-04-30T23:56:38Z

ui/src/keyboardMappings/KeyboardLayouts.ts

@@ -0,0 +1,218 @@
+import { keysUKApple, charsUKApple, modifiersUKApple } from './layouts/uk_apple';


Keys and modifiers correspond to definitions in the Linux USB HID gadget driver and are not language-dependent. These definitions could probably be factored out for easier maintenance.

IDisposable

This is really huge and high-risk. Can we split this up into smaller more mergeable increments?

IDisposable · 2025-05-09T00:21:01Z

ui/src/components/WebRTCVideo.tsx

+          var removeCurrentKey = false;
+
+          // Shift Handling
+          if (mappedModifiers.shift && (e.code === "ShiftLeft" || e.code === "ShiftRight")) {


Doesn't this conflate left and right shift keys such that if I press left,, hold, press right, release (left or right), then we still have a physical shift key down, but the shift state says we are not shifted... so we're wrong... and then we release the other (right or left) and now we have no physical shift keys down, but the shift state is now shifted... so we're even wronger...

dlorch · 2025-05-09T05:36:13Z

This is really huge and high-risk. Can we split this up into smaller more mergeable increments?

It might be pretentious of me to make recommendations, but as I had the ambition of providing a keyboard layout on top of this PR, I had a deeper look at the code and will go ahead to suggest a split into:

paste box
virtual keyboard
key code remapping
miscellaneous code changes

Running the branch on my machine, I've had the high-level impression that there was something counter-intuitive around the key code remapping, where key strokes I entered on my physical keyboard ended up being remapped. But I couldn't quite pin-point on where these changes occurred in the code. This might be an unintended side effect from the interplay between the modules and something to review.

My setup and result:

Swiss German Physical keyboard
Enable Keyboard Mapping (this PR)
Keyboard Layout: German T1 (this PR) - both German and Swiss German are QWERTZ layouts
The target operating system has Swiss German keyboard layout configured
I press the button Z on my physical keyboard
Operating system sees Y

Expected value: Z

The virtual keyboard in JetKVM still needs a lot of improvement and that remains an open topic. There needs to be changes in terms of:

handling of meta keys
supporting different physical layouts (US keyboards feature a horizontal enter key, while European keyboards have a larger, vertical Enter key, shaped like an inverted L)
multi lingual keyboard layout support

Supporting multiple keyboard layouts in the virtual keyboard is completely different code from supporting multiple keyboard layouts in the paste text box (#405), i.e. this can be factored out and built on top of #116 and would be beneficial to the community. Great work on this so far.

toxic0berliner · 2025-05-11T18:23:28Z

Guess those things are subjective but it didn't look that huge to me. Maybe balance the risk against the current total absence of this feature that is highly requested...
I know several of us are eagerly awaiting a merge to be able to contribute other layouts, so maybe things of lower importance can be left to iron out at a later point.
I think focusing on making sure the current layout doesn't break, and the format for future layouts is sensible, anything else I think can be iterated many times later.
Just my 2 cents, I have no merge rights here anyway 😅

dlorch · 2025-05-11T23:50:15Z

Guess those things are subjective but it didn't look that huge to me. Maybe balance the risk against the current total absence of this feature that is highly requested...

I was going back and forth, but ultimately decided to reply and hope I can objectively add some value to the conversation, on a topic that moves many people. I hope also the JetKVM community managers and everyone involved see it as value add.

I'm just a user myself without any super powers and would like to see better keyboard layout support in JetKVM. But from a code point of view, we need to differentiate a few areas:

Physical keyboard handling: this is about typing on the keyboard that is in front of you. JetKVM does not have any notion of a keyboard layout for physical keyboards. It is "blind" about which label the buttons have. Think of keyboard layouts like removable stickers. JetKVM simply transmits the button’s associated key code to the target machine through an USB emulation of a keyboard. The same button that has the label A on an US English keyboard has the label Q on a French keyboard. Pushing this button on either keyboard generates the same key code. The target operating system (the one that you are controlling through JetKVM) must have the keyboard layout configured that matches your physical keyboard layout, or else there will be a mismatch and you'll see the wrong characters showing up. There's just one area where JetKVM intervenes in keyboard handling and that is to keep track which meta keys (Shift, Alt, Ctrl) are being pressed at any given time. This particular code interferes with Alt Gr handling, as this modifier oddly does not represent itself as an Alt key, and there needs to be special casing around it. Addressed in chore(ui): fix Alt Gr not recognized #399.
Paste text box: pasting text to the target machine is handled by taking character-by-character the text you entered into the text area, trying to figure out which is the appropriate key code for that character, and finally sending that key code to the target machine. As you can imagine, this must be done with keeping the target keyboard layout in mind. This mapping was based on an US English keyboard until now, but multiple keyboard layouts are supported with feat(ui): enable multiple keyboard layouts for "paste text" to remote host #405.
Virtual keyboard handling: the user interface is responsible for drawing a keyboard and handling the mapping to key codes, where again the current implementation is based on an US English keyboard. There are some differences in button sizes and arrangements (e.g. US keyboard feature a horizontal Enter key, but European keyboards have a larger, vertical Enter key, shaped like an inverted L) that still need to be accommodated in the future. Also meta key handling (Alt, Shift, Ctrl) could be improved. Overall, this is still an area that still requires a big amount of work.

IDisposable · 2025-05-12T00:27:29Z

Guess those things are subjective but it didn't look that huge to me. Maybe balance the risk against the current total absence of this feature that is highly requested...

I in no way meant any offense, so sorry. My comment was more about teasing this into bite-sized parts that can more easily reviewed and merged.

IDisposable · 2025-05-12T00:30:24Z

But from a code point of view, we need to differentiate a few areas:

This is hugely important, we should solve these individually to minimize untoward risk.

As for me, I am far more concerned with proper paste than anything else, but support fixing each issue...

puigru · 2025-05-12T12:40:45Z

In my opinion, the JetKVM should not have been released without this. It is such an essential feature that it should be part of the minimum viable product. I also purchased 3 JetKVMs on Kickstarter, the lack of keyboard layout support has made them unusable for me and as a result they have been sitting in a box. The fact that this feature is still missing months after release is making me lose confidence over the future of this product.

I'm very glad to all the contributors who are volunteering to this feature. However, this is really something that should have been present from the beggining or at least heavily prioritized by the JetKVM team.

cicorias · 2025-05-12T13:28:27Z

Ideally the PR can be safely flighted perhaps to canary users. Stability is key now that’s folks are using this

I’ll be turning off auto updates as I agree with the pushback on the breadth of this change

williamjohnstone · 2025-05-12T15:36:37Z

Hi all,

Thanks for all of the input, just wanted to add some thoughts:

I agree, this PR is probably too big now, however as I was working through my changes, everything kept breaking everything else. The original keyboard mappings were quite intertwined with the rest of the codebase.
I was told this would be quite an easy PR and it was even listed as a good first issue! However it's clear that this is quite complex.
I no longer have any time at all to dedicate to this, I knew that when I started but the changes took longer than expected, I've also had no input from the Jet team at all, so didn't have much direction in what way they wanted to go.
It is unfortunate that months after shipping this basic issue is still present, and I agree it can make these devices almost unusable.

Hopefully someone can build on what I've started but for the time being, I won't be working on this anymore, especially not for it just to be ignored by the maintainers.

williamjohnstone changed the title ~~[DRAFT] Early implementation of different keyboard layouts.~~ [DRAFT] Add functionality for multiple keyboard layouts Jan 30, 2025

Nevexo mentioned this pull request Jan 30, 2025

Cannot type characters with AltGr key #118

Closed

Nevexo added a commit to Nevexo/jetkvm-kvm that referenced this pull request Jan 30, 2025

NEXT-ONLY: Update README to introduce jetkvm#116

a3c9084

Nevexo mentioned this pull request Jan 30, 2025

JetKVM doesn't wake PC from sleep #120

Open

Move keyboardmapping store to stores.ts, simplified some things, upda…

7c40e2e

…ted settings.tsx to set the keyboard layout properly.

This was referenced Feb 2, 2025

multiKVM hotkey support #125

Open

Full screen should fully capture keyboard #127

Open

williamjohnstone changed the title ~~[DRAFT] Add functionality for multiple keyboard layouts~~ [DRAFT] Overhaul Keyboard Functionality (For multiple keyboard layouts, and other functionality) Feb 2, 2025

Fully implemented layout setting and switching in the UI. Updated Pas…

5f1e53f

…teModal to add more clarity to error message. Begin working on key remapping in WebRTC (working to a reasonable degree).

williamjohnstone added 2 commits February 4, 2025 01:45

Continued work on keyboard overhaul, found many more issues caused by…

f1de663

… my changes (YAY!). Also spent 4 hours troubleshooting to find out I didn't realise how useCallback works... :/ Anway, not much longer before work on just the mappings can begin.

Fix ghost keys issue, properly implemented modifer mapping

8732a6a

williamjohnstone mentioned this pull request Feb 11, 2025

Can't write/paste certain special characters using key combinations #30

Open

Nevexo added a commit to Nevexo/jetkvm-kvm that referenced this pull request Feb 11, 2025

NEXT-ONLY: Update README to introduce jetkvm#116

e847151

Nevexo added a commit to Nevexo/jetkvm-kvm that referenced this pull request Feb 11, 2025

NEXT-ONLY: Update README to introduce jetkvm#116

8c84441

Nevexo mentioned this pull request Feb 19, 2025

Spacebar pauses video stream in fullscreen mode #35

Open

Added German (T1) mappings, UK mappings, updated UK apple mappings. A…

40b1c70

…dded functionality to disable keyboard mapping.

williamjohnstone added 3 commits April 11, 2025 13:26

Updated keyboard character codes, added half working spanish mappings

940612a

Merge remote-tracking branch 'upstream/dev' into keyboard-layouts

95c1410

Update macros to use new keyboard implementation, add keyboard settin…

e534450

…gs to new settings page, update PasteModal to use new keyboard implemention, dropped spanish mappings.

williamjohnstone changed the title ~~[DRAFT] Overhaul Keyboard Functionality (For multiple keyboard layouts, and other functionality)~~ Overhaul Keyboard Functionality (For multiple keyboard layouts, and other functionality) Apr 12, 2025

Merge remote-tracking branch 'upstream/dev' into keyboard-layouts

f753a7a

williamjohnstone marked this pull request as ready for review April 12, 2025 16:11

williamjohnstone mentioned this pull request Apr 14, 2025

Not able to send "@" or "\" to the client #313

Open

dlorch reviewed Apr 30, 2025

View reviewed changes

dlorch mentioned this pull request May 2, 2025

feat(ui): enable multiple keyboard layouts for "paste text" to remote host #405

Merged

IDisposable reviewed May 9, 2025

View reviewed changes

		@@ -0,0 +1,218 @@
		import { keysUKApple, charsUKApple, modifiersUKApple } from './layouts/uk_apple';

Overhaul Keyboard Functionality (For multiple keyboard layouts, and other functionality) #116

Are you sure you want to change the base?

Overhaul Keyboard Functionality (For multiple keyboard layouts, and other functionality) #116

Conversation

williamjohnstone commented Jan 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PrinceJunkie commented Feb 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

williamjohnstone commented Feb 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

williamjohnstone commented Feb 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stylobille commented Feb 3, 2025

Uh oh!

Rhaedyr commented Feb 5, 2025

Uh oh!

williamjohnstone commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

craxo commented Feb 6, 2025

Uh oh!

CLAassistant commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

williamjohnstone commented Feb 22, 2025

Uh oh!

Nevexo commented Feb 24, 2025

Uh oh!

timothystewart6 commented Mar 14, 2025

Uh oh!

williamjohnstone commented Apr 12, 2025

Uh oh!

Novido commented Apr 14, 2025

Uh oh!

williamjohnstone commented Apr 14, 2025

Uh oh!

williamjohnstone commented Apr 16, 2025

Uh oh!

TerraD commented Apr 30, 2025

Uh oh!

williamjohnstone commented Apr 30, 2025

Uh oh!

toxic0berliner commented Apr 30, 2025

Uh oh!

stylobille commented Apr 30, 2025 via email

Uh oh!

dlorch commented Apr 30, 2025

Uh oh!

dlorch commented Apr 30, 2025

Uh oh!

dlorch Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

dlorch Apr 30, 2025

Choose a reason for hiding this comment

Uh oh!

IDisposable left a comment

Choose a reason for hiding this comment

Uh oh!

IDisposable May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dlorch commented May 9, 2025

Uh oh!

toxic0berliner commented May 11, 2025

Uh oh!

dlorch commented May 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

IDisposable commented May 12, 2025

Uh oh!

IDisposable commented May 12, 2025

Uh oh!

puigru commented May 12, 2025

Uh oh!

williamjohnstone commented Jan 30, 2025 •

edited

Loading

PrinceJunkie commented Feb 1, 2025 •

edited

Loading

williamjohnstone commented Feb 2, 2025 •

edited

Loading

williamjohnstone commented Feb 2, 2025 •

edited

Loading

williamjohnstone commented Feb 6, 2025 •

edited

Loading

CLAassistant commented Feb 13, 2025 •

edited

Loading

IDisposable May 9, 2025 •

edited

Loading

dlorch commented May 11, 2025 •

edited

Loading