Android: Show keyboard with WindowInsetsController. #3787
Conversation
xorgy
force-pushed
the
basic-keyboard-show-hide
branch
from
3ad944f
to
94d85fe
Compare
|
I'm a little unsatisfied that this is activated with |
Well, enabling IME doesn't mean that you'll have IME input, because you still enable it for |
It produces key events, and you can type with it (including with some alternate layouts). It only works for keyboards that produce these and are not IME-only (e.g. US, US Dvorak, MX-es Gboard, etc.). Here's what it looks like in a Xilem/Masonry demo app: screen-20240711-092914.2.mp4 |
|
@mwcampbell is going to get an |
xorgy
force-pushed
the
basic-keyboard-show-hide
branch
from
94d85fe
to
a242222
Compare
xorgy
changed the title
xorgy
force-pushed
the
basic-keyboard-show-hide
branch
2 times, most recently
from
01cf7c5
to
593e41a
Compare
xorgy
force-pushed
the
basic-keyboard-show-hide
branch
from
593e41a
to
69f11e7
Compare
I'll backport this to 0.30.x branch, because it's an addition and doesn't break anything, so if you can test with only 0.30.x in your setup it's fine, though you can test with glutin's android example and see the typing in the console or something or whether it pops when calling. |
|
Does hiding the keyboard work as expected for you? I have ported it to 0.29 to test it with my winit/egui stack and it crashes with: Also, typing in egui does not work, because egui expects Anyway, great work on pushing the keyboard input forward. |
|
@podusowski seems threading related, by the message you copied here. Cheers |
|
As for the |
|
Probably can just feel the text with the |
Do you by any chance somehow create the @MarijnS95 do you happen to know whether android is required to use the main thread? And in general, I think we should have a way to ensure that it's running on the main thread. We can just use the same linux code with |
@kchibisov No, I don't think so. It's a pretty typical eframe/egui app. I'll chop out some demo once I find some spare time, because currently it's on a private repo.
Yeah, it works nicely.
@xorgy It's 12. Running on Mi Note 10 Lite. |
|
Great work! Does this work with NativeActivity and GameActivity? |
I've tested it with |
|
I've created a demo based on winit 0.29 and egui, showing a crash here: https://github.com/podusowski/winit/tree/pr3787-with-egui-crash it's in I haven't got time to dig into it so no idea from me yet why this is happening. @lucasmerlin It's |
You figure the same check should be added? Just panic when you start the event loop off the main thread? I think this is still an issue if |
I'm not familiar with how android does things, like it probably won't hurt, but if it always spawns some thread for you which is not the same as |
|
@podusowski just in case, you don't have anything duplicated from Android side of things, like I can't really help with android stuff myself, just checking common mistakes. |
Yeah the thing I wonder is if the thread that the |
|
I have checked the process/thread calls the function and they are the same in both: No idea how it relates to "thread that created a view hierarchy", maybe the whole thing runs on a different, at least in Anyway, I did a test it with I don't even know how to start moving this to jni though. |
I think it would involve injecting Dalvik bytecode, or need to be built into the It is fascinating that showing it succeeds on your end, but hiding it fails. I'll have a think on how something similar to what you're suggesting could be done with plain JNI. |
|
I wonder how C++ stuff solves it, like in QT, since they likely need to workaround it and have a way to run things on the UI thread. I'm not really surprised though that something like that was needed, since macOS/ios/windows do require the exact same thing... |
For UI stuff it's been a long time ago, but it seems The thread is spawned off the "main thread for that At some point I've been wondering if it makes more sense to expose these lower level details to apps and |
|
@MarijnS95 You mean like exposing looper attached to the main thread (assuming I understand this stuff correctly) or something in |
|
I've made some experiments and came up with something like this: In android_app->main_thread_looper = ALooper_forThread();Then, I used this looper to run a Rust callback on the main thread like this: fn show_hide_keyboard_fallible(app: AndroidApp, show: bool) -> Result<(), jni::errors::Error> {
let looper = unsafe { ForeignLooper::from_ptr(app.main_thread_looper()) };
let (mut tx, mut rx) = pipe_channel::channel::<String>();
looper.add_fd_with_callback(
unsafe { BorrowedFd::borrow_raw(rx.as_raw_fd()) },
ndk::looper::FdEvent::INPUT,
|fd, _| {
let s = rx.recv().unwrap();
log::info!(
"got keyboard show/hide event in pid: {:?} tid: {}: {s}",
std::thread::current().id(),
std::process::id()
);
// After Android R, it is no longer possible to show the soft keyboard
// with `showSoftInput` alone.
// Here we use `WindowInsetsController`, which is the other way.
let vm = unsafe { JavaVM::from_raw(app.vm_as_ptr() as _).unwrap() };
let activity = unsafe { JObject::from_raw(app.activity_as_ptr() as _) };
let mut env = vm.attach_current_thread().unwrap();
let window = env
.call_method(&activity, "getWindow", "()Landroid/view/Window;", &[])
.unwrap()
.l()
.unwrap();
let wic = env
.call_method(
window,
"getInsetsController",
"()Landroid/view/WindowInsetsController;",
&[],
)
.unwrap()
.l()
.unwrap();
let window_insets_types = env.find_class("android/view/WindowInsets$Type").unwrap();
let ime_type = env
.call_static_method(&window_insets_types, "ime", "()I", &[])
.unwrap()
.i()
.unwrap();
env.call_method(&wic, &s, "(I)V", &[ime_type.into()])
.unwrap()
.v();
false
},
);
std::thread::sleep(Duration::from_secs(1));
log::info!(
"sending keyboard show/hide event from pid: {:?} tid: {}",
std::thread::current().id(),
std::process::id()
);
if show {
tx.send("show".to_string());
} else {
tx.send("hide".to_string());
}
std::thread::sleep(Duration::from_secs(5));
Ok(())
}Now, obviously this has these |
|
Why does this not use |
On NativeActivity at least, it's a no-op because of a platform change in Android 11. That's why #2993 petered out: it wasn't able to open the keyboard. If there's a way to make that clearer than I make it in the comment then I'd love to hear it. 😅 |
|
Still that seems like something that should be fixed with |
Should be fixed with [...] as in, this should be in I'm glad to do the equivalent in |
I think either way, some changes are needed in In:
|
Well, that's true in some sense, but it's also not that hard to initiate the call from the main thread. The way we do this in Masonry is to have things like this activated by a queue that is processed in the main thread. I am working on a new prebuilt activity (kinda like the proposed |
Yes. If there's any hack required to get this working that's specific to That should also more cleanly allow you to attach a callback to the "main UI thread"
This is going to be contributed to |
My intention is to contribute it to android-activity, and for it to reuse as much as possible there. The other thing is I want it to work before there's any interesting build infrastructure added, so while I'm excited for your work on that, I'm starting with the basics (as I described above). |
Ah I missed that but am glad to hear it; I've got some ideas and things in-progress and I think we can just meet in the middle in the end 🎉 |
changelogmodule if knowledge of this change could be valuable to usersShould resolve #1823 but it's not real IME.