Skip to content

fix(config): support Unicode keymap aliases - #1004

Open
tianrking wants to merge 1 commit into
rmk-rs:mainfrom
tianrking:codex/keymap-unicode-aliases
Open

fix(config): support Unicode keymap aliases#1004
tianrking wants to merge 1 commit into
rmk-rs:mainfrom
tianrking:codex/keymap-unicode-aliases

Conversation

@tianrking

@tianrking tianrking commented Jul 26, 2026

Copy link
Copy Markdown

Fixes #569.

Summary

  • Parse alias identifiers on UTF-8 character boundaries instead of byte offsets, preventing non-English aliases from panicking during slicing.
  • Keep the existing alias grammar intact: names remain case-sensitive and end at Unicode whitespace.
  • Add regression coverage for direct Unicode aliases, recursive expansion into an action, and undefined Unicode aliases.
  • Document the supported behavior and add a changelog entry.

Validation

  • cargo test --manifest-path rmk-config/Cargo.toml --lib (28 tests passed)
  • cargo clippy --manifest-path rmk-config/Cargo.toml --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check

Design scope

This intentionally keeps the existing [aliases] format and resolution semantics; it only makes identifier scanning Unicode-safe.

Resolve keyboard.toml aliases on UTF-8 character boundaries so non-English names no longer panic. Add regression coverage, documentation, and a changelog entry for rmk-rs#569.
@github-actions

Copy link
Copy Markdown

Size Report

Example main PR Diff .text .data .bss
use_config/nrf52832_ble 395.0 KiB 395.0 KiB +0.00% 0 0 0
use_config/nrf52840_ble 440.6 KiB 440.6 KiB +0.00% ⬇️ -4 0 0
use_config/nrf52840_ble_split (central) 497.5 KiB 497.5 KiB +0.00% ⬇️ -8 0 0
use_config/nrf52840_ble_split (peripheral) 320.5 KiB 320.5 KiB +0.00% ⬆️ +4 0 0
use_config/pi_pico_w_ble 668.5 KiB 668.5 KiB +0.00% 0 0 0
use_config/rp2040 146.9 KiB 146.9 KiB +0.00% ⬇️ -4 0 0
use_config/rp2040_split (central) 160.4 KiB 160.4 KiB +0.00% 0 0 0
use_config/rp2040_split (peripheral) 27.8 KiB 27.8 KiB +0.00% 0 0 0
use_config/stm32f1 62.2 KiB 62.2 KiB +0.00% 0 0 0
use_config/stm32h7 99.9 KiB 99.9 KiB +0.00% 0 0 0
use_rust/nrf52832_ble 383.6 KiB 383.6 KiB +0.00% ⬆️ +8 0 0
use_rust/nrf52840_ble 437.1 KiB 437.1 KiB +0.00% 0 0 0
use_rust/nrf52840_ble_split (central) 503.7 KiB 503.7 KiB +0.00% ⬇️ -8 0 0
use_rust/nrf52840_ble_split (peripheral) 318.2 KiB 318.2 KiB +0.00% ⬆️ +4 0 0
use_rust/pi_pico_w_ble 668.9 KiB 668.9 KiB +0.00% 0 0 0
use_rust/rp2040 146.6 KiB 146.6 KiB +0.00% ⬇️ -4 0 0
use_rust/rp2040_split (central) 159.3 KiB 159.3 KiB +0.00% 0 0 0
use_rust/rp2040_split (peripheral) 28.2 KiB 28.2 KiB +0.00% 0 0 0
use_rust/stm32f1 61.6 KiB 61.6 KiB +0.00% 0 0 0
use_rust/stm32h7 119.5 KiB 119.5 KiB +0.00% ⬇️ -8 0 0
use_config/nrf52832_ble — 395.0 KiB → 395.0 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 361768	   8620	  34104	 404492	  62c0c	rmk-nrf52832

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 361768	   8620	  34104	 404492	  62c0c	rmk-nrf52832

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL
use_config/nrf52840_ble — 440.6 KiB → 440.6 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 394892	   8628	  47672	 451192	  6e278	rmk-nrf52840

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 394896	   8628	  47672	 451196	  6e27c	rmk-nrf52840

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +137  [ = ]       0    .debug_info
  +0.0%    +110  [ = ]       0    .debug_str
  +0.0%     +84  [ = ]       0    .debug_line
  +0.5%     +39  [ = ]       0    .debug_abbrev
  +0.0%     +30  [ = ]       0    .strtab
  +8.9%      +4  [ = ]       0    [Unmapped]
  -0.0%      -4  -0.0%      -4    .text
  -0.0%     -12  [ = ]       0    .debug_loc
  +0.0%    +388  -0.0%      -4    TOTAL
use_config/nrf52840_ble_split (central) — 497.5 KiB → 497.5 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 451720	   8684	  49064	 509468	  7c61c	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 451728	   8684	  49064	 509476	  7c624	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +16%      +8  [ = ]       0    [Unmapped]
  -0.0%      -8  -0.0%      -8    .text
  [ = ]       0  -0.0%      -8    TOTAL
use_config/nrf52840_ble_split (peripheral) — 320.5 KiB → 320.5 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 292216	   8500	  27448	 328164	  501e4	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 292212	   8500	  27448	 328160	  501e0	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +82%     +32  [ = ]       0    [Unmapped]
  +0.0%     +13  [ = ]       0    .strtab
  +0.0%      +4  +0.0%      +4    .text
  -0.0%     -97  [ = ]       0    .debug_line
  -0.0%    -122  [ = ]       0    .debug_str
  -0.0%    -234  [ = ]       0    .debug_info
  -0.0%    -404  +0.0%      +4    TOTAL
use_config/pi_pico_w_ble — 668.5 KiB → 668.5 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 631180	      0	  53360	 684540	  a71fc	rmk-pi-pico-w

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 631180	      0	  53360	 684540	  a71fc	rmk-pi-pico-w

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +10  [ = ]       0    .debug_loc
  -2.9%      -2  [ = ]       0    [Unmapped]
  +0.0%      +8  [ = ]       0    TOTAL
use_config/rp2040 — 146.9 KiB → 146.9 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 135616	      0	  14772	 150388	  24b74	rmk-rp2040

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 135620	      0	  14772	 150392	  24b78	rmk-rp2040

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +72  [ = ]       0    .debug_str
  +0.0%     +12  [ = ]       0    .debug_loc
  +0.0%      +8  [ = ]       0    .strtab
  +8.3%      +4  [ = ]       0    [Unmapped]
  -0.0%      -4  -0.0%      -4    .text
  -0.0%      -9  [ = ]       0    .debug_info
  -0.0%     -51  [ = ]       0    .debug_line
  +0.0%     +32  -0.0%      -4    TOTAL
use_config/rp2040_split (central) — 160.4 KiB → 160.4 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 148592	      0	  15636	 164228	  28184	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 148592	      0	  15636	 164228	  28184	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%     +35  [ = ]       0    .debug_str
  +0.0%      +3  [ = ]       0    .strtab
  -1.5%      -1  [ = ]       0    [Unmapped]
  -0.0%      -9  [ = ]       0    .debug_info
  -0.0%     -28  [ = ]       0    .debug_line
  [ = ]       0  [ = ]       0    TOTAL
use_config/rp2040_split (peripheral) — 27.8 KiB → 27.8 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  25616	     60	   2796	  28472	   6f38	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  25616	     60	   2796	  28472	   6f38	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%      +1  [ = ]       0    .debug_str
  +0.0%      +1  [ = ]       0    .strtab
  -5.3%      -2  [ = ]       0    [Unmapped]
  [ = ]       0  [ = ]       0    TOTAL
use_config/stm32f1 — 62.2 KiB → 62.2 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  56892	     28	   6792	  63712	   f8e0	rmk-stm32f1

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  56892	     28	   6792	  63712	   f8e0	rmk-stm32f1

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL
use_config/stm32h7 — 99.9 KiB → 99.9 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  93076	    264	   8976	 102316	  18fac	rmk-stm32h7

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  93076	    264	   8976	 102316	  18fac	rmk-stm32h7

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +12%      +6  [ = ]       0    [Unmapped]
  -0.3%    -187  [ = ]       0    .strtab
  -0.1% -1.23Ki  [ = ]       0    .debug_str
  -0.0% -1.40Ki  [ = ]       0    TOTAL
use_rust/nrf52832_ble — 383.6 KiB → 383.6 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 349984	   8620	  34160	 392764	  5fe3c	rmk-nrf52832

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 349976	   8620	  34160	 392756	  5fe34	rmk-nrf52832

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +58%     +26  [ = ]       0    [Unmapped]
  +0.0%      +8  +0.0%      +8    .text
  -0.0%     -30  [ = ]       0    .strtab
  -1.1%     -88  [ = ]       0    .debug_abbrev
  -0.1%    -235  [ = ]       0    .debug_line
  -0.0%    -245  [ = ]       0    .debug_str
  -0.0%    -360  [ = ]       0    .debug_info
  -0.0%    -924  +0.0%      +8    TOTAL
use_rust/nrf52840_ble — 437.1 KiB → 437.1 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 394960	   8628	  44008	 447596	  6d46c	rmk-nrf52840

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 394960	   8628	  44008	 447596	  6d46c	rmk-nrf52840

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1%    +450  [ = ]       0    .debug_line
  +0.0%    +208  [ = ]       0    .debug_info
  +0.0%    +110  [ = ]       0    .debug_str
  +0.5%     +39  [ = ]       0    .debug_abbrev
  +0.0%     +30  [ = ]       0    .strtab
  -0.0%      -2  [ = ]       0    .debug_loc
  -6.5%      -3  [ = ]       0    [Unmapped]
  +0.0%    +832  [ = ]       0    TOTAL
use_rust/nrf52840_ble_split (central) — 503.7 KiB → 503.7 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 454924	   8684	  52200	 515808	  7dee0	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 454932	   8684	  52200	 515816	  7dee8	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0% +1.19Ki  [ = ]       0    .debug_str
  +0.1%    +499  [ = ]       0    .debug_line
  +0.0%    +405  [ = ]       0    .debug_info
  +0.0%     +75  [ = ]       0    .strtab
   +19%      +9  [ = ]       0    [Unmapped]
  -0.0%      -8  -0.0%      -8    .text
  +0.0% +2.14Ki  -0.0%      -8    TOTAL
use_rust/nrf52840_ble_split (peripheral) — 318.2 KiB → 318.2 KiB (+0.00% ⬆️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 290120	   8500	  27216	 325836	  4f8cc	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 290116	   8500	  27216	 325832	  4f8c8	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +481  [ = ]       0    .debug_str
  +0.0%     +59  [ = ]       0    .strtab
  +0.0%      +4  +0.0%      +4    .text
  -7.0%      -4  [ = ]       0    [Unmapped]
  -0.0%     -96  [ = ]       0    .debug_line
  -0.0%    -236  [ = ]       0    .debug_info
  +0.0%    +208  +0.0%      +4    TOTAL
use_rust/pi_pico_w_ble — 668.9 KiB → 668.9 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 631380	      0	  53552	 684932	  a7384	rmk-pi-pico-w

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 631380	      0	  53552	 684932	  a7384	rmk-pi-pico-w

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1% +2.33Ki  [ = ]       0    .debug_str
  +0.1%    +332  [ = ]       0    .strtab
  +0.0% +2.66Ki  [ = ]       0    TOTAL
use_rust/rp2040 — 146.6 KiB → 146.6 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 135384	      0	  14692	 150076	  24a3c	rmk-rp2040

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 135388	      0	  14692	 150080	  24a40	rmk-rp2040

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0%    +173  [ = ]       0    .debug_str
  +0.0%     +75  [ = ]       0    .debug_line
  +0.0%     +28  [ = ]       0    .debug_info
  +0.0%     +10  [ = ]       0    .debug_loc
  +0.0%      +9  [ = ]       0    .strtab
   +18%      +9  [ = ]       0    [Unmapped]
  -0.0%      -4  -0.0%      -4    .text
  +0.0%    +300  -0.0%      -4    TOTAL
use_rust/rp2040_split (central) — 159.3 KiB → 159.3 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 147656	      0	  15468	 163124	  27d34	central

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 147656	      0	  15468	 163124	  27d34	central

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1% +1.93Ki  [ = ]       0    .debug_str
  +0.2%    +265  [ = ]       0    .strtab
  +2.6%      +1  [ = ]       0    [Unmapped]
  -0.0%     -46  [ = ]       0    .debug_info
  -0.1%    -152  [ = ]       0    .debug_line
  +0.0% +1.99Ki  [ = ]       0    TOTAL
use_rust/rp2040_split (peripheral) — 28.2 KiB → 28.2 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  25716	     60	   3060	  28836	   70a4	peripheral

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  25716	     60	   3060	  28836	   70a4	peripheral

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.1%    +585  [ = ]       0    .debug_str
  +0.1%     +28  [ = ]       0    .strtab
  +4.8%      +3  [ = ]       0    [Unmapped]
  +0.0%    +616  [ = ]       0    TOTAL
use_rust/stm32f1 — 61.6 KiB → 61.6 KiB (+0.00%)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
  56260	     28	   6808	  63096	   f678	rmk-stm32f1

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
  56260	     28	   6808	  63096	   f678	rmk-stm32f1

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL
use_rust/stm32h7 — 119.5 KiB → 119.5 KiB (+0.00% ⬇️)

cargo size (PR):

   text	   data	    bss	    dec	    hex	filename
 107108	    320	  14980	 122408	  1de28	rmk-stm32h7

cargo size (main):

   text	   data	    bss	    dec	    hex	filename
 107116	    320	  14980	 122416	  1de30	rmk-stm32h7

Bloaty diff (PR vs main):

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.2%    +283  [ = ]       0    .debug_line
  +0.0%     +46  [ = ]       0    .debug_info
   +21%      +8  [ = ]       0    [Unmapped]
  -0.0%      -8  -0.0%      -8    .text
  -0.0%     -10  [ = ]       0    .debug_loc
  -0.2%    -191  [ = ]       0    .strtab
  -0.1% -2.06Ki  [ = ]       0    .debug_str
  -0.0% -1.93Ki  -0.0%      -8    TOTAL

@tianrking
tianrking marked this pull request as ready for review July 26, 2026 17:31

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46d66a2247

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rmk-config/src/layout.rs
Comment on lines +267 to +269
let alias_len = current_keys[alias_start..]
.find(char::is_whitespace)
.unwrap_or(current_keys.len() - alias_start);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make Unicode alias separators valid keymap whitespace

When an alias is followed by non-ASCII whitespace such as NBSP or an ideographic space, this scan ends the alias at that character but leaves the separator in next_keys. The unchanged keymap.pest WHITESPACE rule accepts only space, tab, newline, and carriage return, so the Unicode separator is absorbed into an adjacent loose_identifier or causes parsing to fail, producing an invalid action or incorrect key count. Normalize/consume these separators during expansion or extend the grammar to recognize the same whitespace set.

Useful? React with 👍 / 👎.

@HaoboGu

HaoboGu commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

This cannot close #569

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support non-english characters in keymap setting

2 participants