[Web] Add IME input support #79362

bruvzg · 2023-07-12T07:22:01Z

Implements IME event handling for Web platform.

Fixes Cannot type dead keys (accented characters) or use Input Method Editor on a LineEdit exported to HTML #45904

Bugsquad edit:

Addresses Web part of Inputting Korean characters on any GUI control node breaks the characters on macOS #85597

2023-07-19.11-23-57.mp4

Faless

This looks really nice 🥳

I wonder if this can replace the experimental virtual keyboard support...

I remember I tested the composition events when doing that work, and the browser support was all over the place (missing events, repeated ones, etc) depending on both platform and vendor.

Which OS/browsers did you test?

platform/web/js/libs/library_godot_input.js

akien-mga · 2023-10-03T13:32:03Z

Would be nice to finalize this for 4.2 :)

jaekong · 2023-12-01T18:58:01Z

I have tested current build of your fork, and these are my findings:

Currently does not work on Safari and Chromium-based browsers (I have tested it on Arc) It is essentially the same behavior with 4.2.stable, but here's a bit of a detailed explanation video.

2023-12-02.3.33.38-H.264.MP4.mp4

On Firefox, it is better than stable (which does not use IME at all so it just comes out as English alphabet), but it has some weird behavior.

2023-12-02.3.39.03.mov

Vietnamese and French characters are duplicated.
When inputting Korean characters, cursor does not advance as it should - it sometimes does advance, but it seems to be random.

I am running it on macOS 14.0 Sonoma in case that is relevant.

bruvzg · 2023-12-01T19:42:13Z

Which OS/browsers did you test?

Retested it, and it seems to be working only in Firefox. Until Chromium have IME API implemented, we probably won't be able to do anything with it.

We should probably keep the reference to this instead of assigning it an ID and using selector to retrieve it (which must be page-unique, so ime is also likely to collide with user custom HTML).

Done.

Vietnamese and French characters are duplicated.
When inputting Korean characters, cursor does not advance as it should - it sometimes does advance, but it seems to be random.

Both should be fixed in the last commit (by applying the same logic as #85458 use).

jaekong · 2023-12-01T20:10:42Z

Retested it, and it seems to be working only in Firefox. Until Chromium have IME API implemented, we probably won't be able to do anything with it.

Out of curiosity, what APIs are missing from Chromium and Safari? compositionstart/update/end are all available in both engines and it seems like they are the only DOM events used by your PR? I am not familiar enough with Godot's source to dig into C++ code changes you've made (I tried 😭) so I might have missed a lot of things.

+ Edit: I have tested compositionstart/update/end events on the three browsers with this MDN example. At least with Korean, it seems like all three browsers are outputting the same event logs.

jaekong · 2023-12-01T20:23:48Z

I think I am onto something, but I am not sure.

Apparently, Safari just doesn't render any DOM element inside <canvas> so it was acting as if nothing was different.

2023-12-02.5.17.54.mov

After I just moved the element out of the canvas, it started to behave a bit better. It still is acting weird, but it might be because I am using the older build with the issues I talked above. Same behavior on Arc (Chromium) too.

It does not recognize backspace when I did that, so that's new.

bruvzg · 2023-12-01T20:51:05Z

I think I am onto something, but I am not sure.

Apparently, Safari just doesn't render any DOM element inside so it was acting as if nothing was different.

Interesting find, with the key_up / key_down event handlers added to the ime div it seems to be almost fully working in Chromium (the only issue I see is missing caret, which probably rely on canvas being focused).

jaekong · 2023-12-01T21:15:47Z

Interesting find, with the key_up / key_down event handlers added to the ime div it seems to be almost fully working in Chromium (the only issue I see is missing caret, which probably rely on canvas being focused).

How are you so fast? 😂 I was figuring out keyup keydown part and it just got fixed

It seems to work perfect so far, except I think it will be better if display mode is fixed instead of absolute. Absolute makes viewport overflow.

const ime = document.createElement("div");
      ime.className = "ime";
      ime.style.background = "none";
      ime.style.opacity = 0.0;
      ime.style.position = "fixed";

I'd submit PR if it were not just a single word change 😂 but I think it'd be more of a hassle for you to merge PR than just replacing a word.

Faless

This is looking great 🚀 .
We are missing some cleanup for the added node, this patch adds that.

diff --git a/platform/web/js/libs/library_godot_input.js b/platform/web/js/libs/library_godot_input.js
index ec73f3bdb3..7c070f029d 100644
--- a/platform/web/js/libs/library_godot_input.js
+++ b/platform/web/js/libs/library_godot_input.js
@@ -34,6 +34,7 @@
 
 const GodotIME = {
        $GodotIME__deps: ['$GodotRuntime', '$GodotEventListeners'],
+       $GodotIME__postset: 'GodotOS.atexit(function(resolve, reject) { GodotIME.clear(); resolve(); });',
        $GodotIME: {
                ime: null,
                active: false,
@@ -121,6 +122,13 @@ const GodotIME = {
                        GodotConfig.canvas.parentElement.appendChild(ime);
                        GodotIME.ime = ime;
                },
+
+               clear: function () {
+                       if (GodotIME.ime) {
+                               GodotIME.ime.remove();
+                               GodotIME.ime = null;
+                       }
+               },
        },
 };
 mergeInto(LibraryManager.library, GodotIME);

I've also tested this on Android (would be great if someone could test on iOS).

Chrome raise the virtual keyboard, but input doesn't work. This seems to be a problem with chrome on android not emitting composition events with some(?) virtual keyboards.

With Firefox on Android on the other end, some composition events are emitted, and input is received thought it seems to produce unreliable results when auto-completing and auto-correcting.

I don't want virtual keyboards to block this PR since that's not officially supported anyway.
I think this PR still needs some work to at least not break the optional experimental vk support we introduced some time ago which uses a similar approach.
I think we might want to disable IME completely when that option is enabled.
I'll see if I can come up with another patch.

platform/web/js/libs/library_godot_input.js

Faless · 2023-12-05T11:40:38Z

Here's the other patch to only enable IME when experimental VK support is disabled (default) or not available (enabled but does not detect a touchscreen).

diff --git a/platform/web/display_server_web.cpp b/platform/web/display_server_web.cpp
index e9f5577fbe..1b63f28b08 100644
--- a/platform/web/display_server_web.cpp
+++ b/platform/web/display_server_web.cpp
@@ -1137,7 +1137,6 @@ bool DisplayServerWeb::has_feature(Feature p_feature) const {
 	switch (p_feature) {
 		//case FEATURE_GLOBAL_MENU:
 		//case FEATURE_HIDPI:
-		case FEATURE_IME:
 		case FEATURE_ICON:
 		case FEATURE_CLIPBOARD:
 		case FEATURE_CURSOR_SHAPE:
@@ -1151,6 +1150,9 @@ bool DisplayServerWeb::has_feature(Feature p_feature) const {
 		//case FEATURE_WINDOW_TRANSPARENCY:
 		//case FEATURE_KEEP_SCREEN_ON:
 		//case FEATURE_ORIENTATION:
+		case FEATURE_IME:
+			// IME does not work with experimental VK support.
+			return godot_js_display_vk_available() == 0;
 		case FEATURE_VIRTUAL_KEYBOARD:
 			return godot_js_display_vk_available() != 0;
 		case FEATURE_TEXT_TO_SPEECH:

Faless

Amazing work 🚀 !
Thank you very much 🏆 !

platform/web/display_server_web.cpp

YuriSizov · 2023-12-16T16:59:03Z

Thanks!

zerosum-0 · 2024-01-18T23:54:47Z

Hi
Can you check this bugs.
I made one LineEdit and one Text box .
And When I change the focus while typing.

The last letter dissapears in mac.
The last letter move to another control in web (I used chrome)

KoreanIMEBugReproduction 2.zip

2024-01-19.8.37.56.mov

2024-01-19.8.41.04.mov

Zireael07 · 2024-01-19T09:22:35Z

@zerosum-0 please make a separate issue report

JunShiozawa · 2024-04-10T22:28:40Z

I am Japanese.
Prease add milestone Godot 3.x
Godot 4.x's web export don't run on iOS and Mac Safari.

MASoftware-es · 2024-04-16T06:32:58Z

Good morning.
I am currently using the official stable version v4.2.1, and this bug still occurs. When I use a "TextEdit" or an "InputLine" in the application, I can introduce accents if I compile natively in desktop application mode, both for Windows, Linux, Mac, but however, when I generate an HTML project, it only works from Windows, since neither from OSX nor from Linux can I get those fields to accept accents.
It seems that the case was closed, but it continues to happen.
Can you reconsider continuing the investigation and look for an alternative?
Thank you so much.

akien-mga · 2024-04-16T08:27:10Z

@MASoftware-es That issue was fixed in 4.3, you should test 4.3-dev5 or later.

MASoftware-es · 2024-04-18T18:53:25Z

Thanks so much!!El 16 abr 2024 10:27, Rémi Verschelde ***@***.***> escribió: @MASoftware-es That issue was fixed in 4.3, you should test 4.3-dev5 or later. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

JunShiozawa · 2024-04-25T23:57:06Z

@akien-mga
Why do not you add this Godot 3.x?
I need this on Godot 3.x

akien-mga · 2024-04-26T07:46:07Z

@akien-mga Why do not you add this Godot 3.x? I need this on Godot 3.x

It could be backported to 3.x if a contributor is interested. The 3.x branch is supported on a best-effort basis, and we already have our hands full with working on improvements for 4.3 (like this PR for example), and bug fixes for 4.2. So I can't make any promises because this is a resourcing question, not a technical issue.

JunShiozawa · 2024-05-06T04:05:44Z

@bruvzg

I am Japanese.

Could you please implement [Web] Add IME input support in Godot 3.x ?

Web export of Godot 4.x cannot be used for Safari for iPhone.

This is a political issue for Japanese people.

JunShiozawa · 2024-06-25T03:26:50Z

This is
Chrome for Android Japanese input OK.
Firefox for Android Japanese input OK.
Safari for iPhone Japanese input OK.
Firefox for Linux Japanese input OK.
Firefox for Windows Japanese input OK.

bruvzg added enhancement platform:web topic:porting topic:input labels Jul 12, 2023

bruvzg added this to the 4.x milestone Jul 12, 2023

bruvzg force-pushed the web_ime branch 3 times, most recently from 6b3e923 to f2a2101 Compare July 12, 2023 13:28

bruvzg force-pushed the web_ime branch 3 times, most recently from ef4d8b3 to ced2224 Compare July 26, 2023 07:14

bruvzg marked this pull request as ready for review July 26, 2023 07:14

YuriSizov requested a review from Faless July 26, 2023 14:31

Faless reviewed Jul 26, 2023

View reviewed changes

platform/web/js/libs/library_godot_input.js Outdated Show resolved Hide resolved

YuriSizov modified the milestones: 4.x, 4.2 Jul 26, 2023

Faless mentioned this pull request Oct 18, 2023

Cannot type dead keys (accented characters) or use Input Method Editor on a LineEdit exported to HTML #45904

Closed

AThousandShips modified the milestones: 4.2, 4.3 Oct 26, 2023

bruvzg mentioned this pull request Dec 1, 2023

Inputting Korean characters on any GUI control node breaks the characters on macOS #85597

Open

bruvzg force-pushed the web_ime branch 2 times, most recently from b2bb53d to 941ffe0 Compare December 1, 2023 10:49

akien-mga requested a review from Faless December 1, 2023 11:12

bruvzg force-pushed the web_ime branch from 941ffe0 to de98bf9 Compare December 1, 2023 19:29

bruvzg force-pushed the web_ime branch 2 times, most recently from 415cc74 to 7d7bd6e Compare December 1, 2023 21:07

bruvzg force-pushed the web_ime branch from 7d7bd6e to fc8eb5d Compare December 4, 2023 06:18

Faless reviewed Dec 4, 2023

View reviewed changes

platform/web/js/libs/library_godot_input.js Outdated Show resolved Hide resolved

platform/web/js/libs/library_godot_input.js Outdated Show resolved Hide resolved

bruvzg force-pushed the web_ime branch from fc8eb5d to 5f2c6fd Compare December 5, 2023 05:52

bruvzg requested a review from a team as a code owner December 5, 2023 05:52

bruvzg force-pushed the web_ime branch from 5f2c6fd to 382cf08 Compare December 5, 2023 11:42

Faless approved these changes Dec 6, 2023

View reviewed changes

YuriSizov changed the title ~~[Web] Add IME input support.~~ [Web] Add IME input support Dec 15, 2023

YuriSizov reviewed Dec 15, 2023

View reviewed changes

platform/web/display_server_web.cpp Outdated Show resolved Hide resolved

[Web] Add IME input support.

f9486a2

bruvzg force-pushed the web_ime branch from 382cf08 to f9486a2 Compare December 15, 2023 17:11

YuriSizov merged commit 86a4115 into godotengine:master Dec 16, 2023
15 checks passed

akien-mga mentioned this pull request Apr 26, 2024

TextEdit or LineEdit cannot input Chinese characters on the web #91204

Closed

bruvzg mentioned this pull request May 6, 2024

[Web, 3.x] Add IME input support. #91610

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Web] Add IME input support #79362

[Web] Add IME input support #79362

bruvzg commented Jul 12, 2023 •

edited by akien-mga

Loading

Faless left a comment •

edited

Loading

akien-mga commented Oct 3, 2023

jaekong commented Dec 1, 2023 •

edited

Loading

bruvzg commented Dec 1, 2023

jaekong commented Dec 1, 2023 •

edited

Loading

jaekong commented Dec 1, 2023 •

edited

Loading

bruvzg commented Dec 1, 2023

jaekong commented Dec 1, 2023 •

edited

Loading

Faless left a comment

Faless commented Dec 5, 2023

Faless left a comment

YuriSizov commented Dec 16, 2023

zerosum-0 commented Jan 18, 2024 •

edited

Loading

Zireael07 commented Jan 19, 2024

JunShiozawa commented Apr 10, 2024

MASoftware-es commented Apr 16, 2024

akien-mga commented Apr 16, 2024

MASoftware-es commented Apr 18, 2024 via email

JunShiozawa commented Apr 25, 2024

akien-mga commented Apr 26, 2024

JunShiozawa commented May 6, 2024

JunShiozawa commented Jun 25, 2024 •

edited

Loading

[Web] Add IME input support #79362

[Web] Add IME input support #79362

Conversation

bruvzg commented Jul 12, 2023 • edited by akien-mga Loading

Faless left a comment • edited Loading

Choose a reason for hiding this comment

akien-mga commented Oct 3, 2023

jaekong commented Dec 1, 2023 • edited Loading

bruvzg commented Dec 1, 2023

jaekong commented Dec 1, 2023 • edited Loading

jaekong commented Dec 1, 2023 • edited Loading

bruvzg commented Dec 1, 2023

jaekong commented Dec 1, 2023 • edited Loading

Faless left a comment

Choose a reason for hiding this comment

Faless commented Dec 5, 2023

Faless left a comment

Choose a reason for hiding this comment

YuriSizov commented Dec 16, 2023

zerosum-0 commented Jan 18, 2024 • edited Loading

Zireael07 commented Jan 19, 2024

JunShiozawa commented Apr 10, 2024

MASoftware-es commented Apr 16, 2024

akien-mga commented Apr 16, 2024

MASoftware-es commented Apr 18, 2024 via email

JunShiozawa commented Apr 25, 2024

akien-mga commented Apr 26, 2024

JunShiozawa commented May 6, 2024

JunShiozawa commented Jun 25, 2024 • edited Loading

bruvzg commented Jul 12, 2023 •

edited by akien-mga

Loading

Faless left a comment •

edited

Loading

jaekong commented Dec 1, 2023 •

edited

Loading

jaekong commented Dec 1, 2023 •

edited

Loading

jaekong commented Dec 1, 2023 •

edited

Loading

jaekong commented Dec 1, 2023 •

edited

Loading

zerosum-0 commented Jan 18, 2024 •

edited

Loading

JunShiozawa commented Jun 25, 2024 •

edited

Loading