[Bug]: swipe does not work on non-qwerty layouts

[tobiasBora]

CleverKeys Version

v1.1.71

Device Information

Android Phone

Android Version

11

Bug Category

Neural Prediction (ONNX models, swipe typing)

Bug Description

When swiping with a non-qwerty layout (bepo-fr) it produces completely unrelated results. For instance if I swipe "hello" I get completely unrelated suggestions like "met", "key"... Maybe it is using a different layout internally?

Also the prediction is fairly slow, maybe 2 seconds.

Steps to Reproduce

1. Enable the latin belo fr layout
2. swipe "hello"

Neural Configuration (if applicable)

- enabled
- beam width : 6
- max word lenght : 15
- confidence threshold : 0.010
- no idea

Logs

Troubleshooting Steps

• I have restarted the CleverKeys app
• I have tried disabling and re-enabling the keyboard
• I have checked if this happens with neural predictions disabled
• I have tried clearing the app cache/data (if safe to do so)

Additional Context

No response

[github-actions[bot]]

👋 Welcome to CleverKeys! Thank you for your first issue.

Getting Started

• 📖 Check our Contributing Guide for development setup
• 🧪 Our automated tests will help validate any fixes
• 💬 Join GitHub Discussions for questions

Neural Keyboard Context

CleverKeys is an AI-powered keyboard with:

• 🧠 On-device neural predictions (no cloud)
• 🔒 Complete privacy (100% local processing)
• ⚡ Hardware acceleration for modern Android devices
• 🧪 Comprehensive automated testing

We appreciate your contribution to making typing smarter and more private! 🚀

[tribixbite]

The neural network requires a dataset mapping swipe traces to words. Even for english and qwerty this is hard to find // options are limited. I'd love to add support for more languages and layouts but that requires finding or creating the needed datasets. Please re-create if you find one to use!

[tribixbite]

I don't feel any outrage, just some disappointment that this is not mentioned at all in the README and only find it here after looking through the settings for an hour.
What would I (or anybody willing to help) have to do to train a model in a given language? A short howto would be nice.

Sorry for that experience, my goal in publishing was to share what what I've been building for any others who might benefit, not to mislead.

Updating and running training code for a new layout or language will be rather trivial (a handful of lines), the tricky part is the reliance on a quality dataset.

To train the included qwerty/english model I filtered the FUTO dataset based on metadata attributes (eg omitting self-described non-native and 'unfamiliar' english speakers) and combined it with the only other available swipe dataset I've been able to find ('how we swipe'). Each alone yielded subpar results, so there's some optimization work to be done on the [FUTO] collection code/process.

[MXC48]

Would it be possible to fork https://swipe.futo.org/keyboard to allow the collection of other languages? It probably won't take a week to do, maybe even more than a month, but it could be worth it if, in the long term, it allows the creation of free databases accessible to all that could be useful in this project and others.
The work for the data collection tool has already been done, it just needs to be adapted for other languages with different layouts and other sources for the sentences to be swiped (there are freely available book databases easily accessible on the web).
What do you think?

Absolutely. Much better use of the (currently broken demo) linked website. Aiming to get this live by next week.

Great! Let us know when the project goes live! It would also be good to create an account on Mastodon or another social network to promote the data collection project a little :)
Looking forward to contributing!

[RaphaelPorcherot]

First of all thanks for this great app !
Second i'd be happy to help set up a french swipe dataset if needed, as i cant help with the coding.
Also correct me please if i m wrong but since the approach is hybrid as you mentioned would that not mean that with

A qwerty layout
Another language user dictionnary completely replacing the english one in my case a french one

Then we would have a short term solution allowing to use the nn for non english language swipe recognition?

But does it also mean thas it is not possible to add custom words without manually retraining the model?

No. The model handles the spatial decoding, but the vocabulary is dynamic.

The neural network is a Transformer-based encoder-decoder that translates raw touch coordinates into character probabilities (beams). We use a hybrid approach where these probabilities are constrained by a dictionary look-up (Trie/FST) during the decoding process.

When you add a word to the User Dictionary, it is inserted into the decoding graph immediately. The neural net provides the confidence score for the path, and the dictionary validates that the sequence is a user-approved word. No weight updates or retraining are required.

[tribixbite]

Romance languages especially should be somewhat interchangeable, yes. I don't know if the quality will be high enough to be practical but after some preliminary testing it seems worth investigating further. So I will do that soon. It would be extremely helpful if someone could provide a vocab file of 30-100k words, ideally with frequency scores (how common each word is).

Assembling (or finding and evaluating) one myself will probably take me longer than adding the swap capability to the kb.

[MXC48]

Romance languages especially should be somewhat interchangeable, yes. I don't know if the quality will be high enough to be practical but after some preliminary testing it seems worth investigating further. So I will do that soon. It would be extremely helpful if someone could provide a vocab file of 30-100k words, ideally with frequency scores (how common each word is).

Assembling (or finding and evaluating) one myself will probably take me longer than adding the swap capability to the kb.

Would that be sufficient for you? The first is a list of dictionaries used in applications such as HeliBoard, including frequency of use. The second is the sources used to build these dictionaries, with frequency specified directly next to each word:

https://codeberg.org/Helium314/aosp-dictionaries#dictionaries

https://github.com/hermitdave/FrequencyWords/tree/master/content%2F2016

[tribixbite]

Inference speed varies greatly with beam width try lowering to 2 or 4

[tribixbite]

Yes, and I'd be happy to do this

…

On Tue, Dec 30, 2025, 4:50 AM MXC48 ***@***.***> wrote: Yes, and any outrage you feel in reading this is exactly (in part) what led me to build this app in the first place- it's crazy that there aren't more open source gesture typing models (or datasets). It would only take about a week's worth of regular phone use by about 10 people to generate a dataset for a given language and/or layout, assuming they were using a keyboard that could share the swipe data. Which afaik doesn't exist. I was fairly impressed with the template based implementation used in AnySoftKb, so am reconsidering building similar functionality as a short term solution. Would it be possible to fork https://swipe.futo.org/keyboard to allow the collection of other languages? It probably won't take a week to do, maybe even more than a month, but it could be worth it if, in the long term, it allows the creation of free databases accessible to all that could be useful in this project and others. The work for the data collection tool has already been done, it just needs to be adapted for other languages with different layouts and other sources for the sentences to be swiped (there are freely available book databases easily accessible on the web). What do you think? — Reply to this email directly, view it on GitHub <#9 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC5DIOUSPBTOQ37S6H7AV34EJDGNAVCNFSM6AAAAACP2UEQ2WVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKMZXGM4TCMY> . You are receiving this because you modified the open/close state.Message ID: ***@***.*** com>

[tribixbite]

fyi i'm working on other languages now and initial results are very promising. french and italian are mostly done and in theory the existing nn should also be able to run:
portugeuse, indonesian, malay, german, swahili, dutch, and filipino

[tobiasBora]

Great, but on which layout? E.g. I use the bépo layout (not very standard) + french & english on a daily basis.

[tribixbite]

Great, but on which layout? E.g. I use the bépo layout (not very standard) + french & english on a daily basis.

On qwerty - different layouts require retraining. However this is very good news for your use case too as it means only one dataset and round of training will be needed for your setup.

Exploring these paths will help inform the website data collection specifics // implementation details

[tribixbite]

FYI the efficiency of the bépo layout may make training very difficult, due to so many near identical swipe paths for so many different words.