Replies: 8 comments 14 replies
-
|
The words in the second list seem to be two concatenated words, missing the space in between: internacionalactualización --> "internacional actualización" (international update) Some are typos: internacionalzación --> "internacionalización" BTW I thought that maybe you could add the grave and circumflex accents to your list of characters: àèìòù ÀÈÌÒÙ âêîôû ÂÊÎÔÛ While they are not used in spanish, they could appear somewhere like for instance the literal name of a frech person's name, location or movie, like with english. |
Beta Was this translation helpful? Give feedback.
-
|
Can't think of an easy/fast way to figure out if it is a concatenated word. It is doable, but not fast. I will have to ignore them. Will make some program changes to try to find more plurals, and then see what is fixable and what must be ignored. My list of characters is going to be the "baseline" that any keyboard layout MUST provide if it claims to be "Spanish". On the flip side, all input texts may only contain those characters, so that texts and keyboards are in agreement. Other characters are optional. |
Beta Was this translation helpful? Give feedback.
-
How many are we talking about? I think I can manually clean a txt file of that rather quickly (it's basically hitting space).
Ah, ok, with the goal in mind, it's clearer. In that case, I think you should drop these two symbols: § ¶ Technically, they are spanish punctuation symbols. But they are not on qwerty-es, and noone, ever, uses them in any kind of text. The frequencies are negligible as well: Also the yen and pound are not in qwerty-es. I think they're useful. but they don't follow your rule of being in a spanish keyboard. But I think in an optimized kb it'd be nice to have them: AlGr + L for £, and AltGr + Y for ¥. Also, I wonder why |
Beta Was this translation helpful? Give feedback.
-
Okay, that also reduces bigram list a lot.
Those are not in the Leipzig corpus ( I scanned for some of them to convince myself.). could be because of their sources ... If I get Gutenberg to work some should show up. I had similar issues with the English corpus that I made (no Leipzig). Current "Spanish" character list is |
Beta Was this translation helpful? Give feedback.
-
|
You can use MOAR backticks :) Here I opened with 2 backticks, closed with two backticks. The escaped backtick shall not be next to the code backticks, so I added a space at the end. |
Beta Was this translation helpful? Give feedback.
-
|
Are these actual words? Here's the top 100 of the others... I'm guessing they are all okay. As you can see the counts are getting low. éste 52797 Thanks, Ian |
Beta Was this translation helpful? Give feedback.
-
|
The majority of those are correct words. Perhaps a little on the rare side, meaning they're not very common, and maybe that's why they aren't in the dictionary. But they are fine. Some have typos, mostly with the accented vowels. Let me try to catch them all: comprension --> comprensión (understanding) A special case is "ícono". The typical form is "icono" (unaccented) but I just checked and accented is also correct. It's more likely a typo, but it happens to be valid. I think those would be all, if I haven't missed any. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. I guess I will use regex to replace them in the corpus. |
Beta Was this translation helpful? Give feedback.
-
|
As Arno say, we're getting to the point of diminishing returns. I'm down to the "7 counts" in the unknown words, and the Capitalised ones are starting to look a bit suspicious, so will stop with that process. If we can maybe do another 200 regular words, then I will drop the rest of the unknown (and probably rare) words and make final 'cleaner' corpus. Metrics: 100 unknown words: abstenciones 1451 Thanks, Ian |
Beta Was this translation helpful? Give feedback.
-
|
Hi there, As the last batch, the majority of these words are valid, though a little uncommon, with a few typos in between: situaci --> situación The rest are OK. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, have updated and rerunning. |
Beta Was this translation helpful? Give feedback.
-
|
@iandoug & @Lobo-Feroz -- thank you so much for your persistence and diligence! |
Beta Was this translation helpful? Give feedback.
-
|
Last batch, thanks. I looked at the Gutenburg code, it pulls all books .. for English, I discarded any that had non-US-ANSI chars. That won't work here. Instead I got a book list which shows language, will parse that then use wget/curl to pull the Spanish ones. interactúan 980 |
Beta Was this translation helpful? Give feedback.
-
|
Same same. Mostly uncommon words, some typos: travs --> través This one is special: oacute oacute is not spanish, in fact I think this is html markup to display an ó, the complete form being maybe in the rest of the words you have more like this: aacute eacute iacute uacute ntilde and others here's a list: https://www.ou.edu/research/electron/internet/special.shtml |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, you are correct about it being HTML. |
Beta Was this translation helpful? Give feedback.
-
|
Had some other stuff I needed to sort out over our long weekend, hope to carry on with this today .. one more work work thing to sort out first. |
Beta Was this translation helpful? Give feedback.
-
|
@iandoug -- After years of work, I have created a crowdsource-data-driven, multi-objective approach to optimizing keyboard layouts that I am much happier with. Rather than design based on my own intuitions, I collected data from hundreds of typists about what key-pairs they prefer to type, and empirically derived ergonomics objectives and corresponding key-pair scoring criteria. I am preparing to submit a paper about this work for publication by Halloween (personal deadline) and include a few metrics for evaluation that I developed, but I would like to include some independent online keyboard layout scorer results. In addition to KLAnext, the only online layout scorers I could find are Colemak effort (https://colemakmods.github.io/mod-dh/compare.html), Cyanophage (https://cyanophage.github.io/), Oxey (https://oxey.dev/playground/index.html), and keysolve-web (https://galileoblues.github.io/keysolve-web/). What do you think of these? What do you recommend? |
Beta Was this translation helpful? Give feedback.
-
|
@iandoug -- I ran across this stats table compiled from Cyanophage. Unless you have a better suggestion, I'll include these data for comparison: |
Beta Was this translation helpful? Give feedback.
-
|
Hi Arno I have major issues with cyanophage, oxey, keysolve, and the bot they use on the Discord server. Firstly, the physical layouts for the first two are not ANSI/ISO. The way Enter, Shift, and even Space are handled, if at all by the analyzer, hugely affects various metrics. You can't just ignore them or make assumptions. Many of the keys are missing. The digits 0, 1, 9 are more common than j, q, z, but are not included. Then we have WHAT is measured. They have "skipgrams". I would argue this depends on the typist. Then there is the focus on "rolls", with the idea that "in-rolls good, out-rolls bad". On the one hand, this ignores the way language works ... eg THougHT ... siNG, iGNore ... one way may be more common, but you will get rolls both ways. Pianists show that there is no speed difference between in or out, so what is the point of the metric? There is also "redirects" or "pinballs" ... again, from my point of view, this is a made-up metric that does not affect anything. Music relies on redirects to create "hooks", and musicians do not complain about them. It's a non-issue. Then we have the corpora ... AFAIK they use corpora from one of the typing sites, the Discord bot can use Shai's corpus IIRC, and some others. They work by analysing bigrams or trigrams, but then you need to look at how good the corpus is that they analysed. Did they include all chars on the keyboard? Do they include Caps separately to lower (and thus shift)? Do they include Enter and Space? If not, then the stats are meaningless. There are other analysers, including other spins on KLA. You can find links here under Layout analysis: https://www.keyboard-design.com/tools.html See also the Keyboard Layouts (3rd edition) but with the caveat that it is based on the analyzers above and thus I have major issues with it where the metrics are involved. Same with Pascal's list. Dario's analyser (link on my Tools page) is good, but you need to decide on which options you like. Okay, enough for now. May edit later. |
Beta Was this translation helpful? Give feedback.
-
|
@iandoug -- Thank you for your thoughtful response! I hear your concerns. Recognizing that the world of typing metrics is a complex one, I was hoping for a comparison focused on simple, interpretable measures comparing lowercase letters that is independent of capitalization, punctuation, spaces, and minimally affected by ANSI/ISO differences. Given these constrains, which do you recommend? My study includes a statistical analysis to determine the degree to which reports of typing preference support a few of the assumptions above. I eagerly await any article you end up writing detailing your concerns! |
Beta Was this translation helpful? Give feedback.
-
|
I should add that I need a defensible argument for why I chose the metrics/analyzers that I did in the publication. So any reasoned selection you can think of would be greatly appreciated! If KLAnext/Dario/others, please make recommendations for appropriate options. |
Beta Was this translation helpful? Give feedback.
-
|
@iandoug and @Lobo-Feroz -- As mentioned in my comment above, I have developed a new method for generating optimal keyboard layouts in different languages. English and Spanish are almost ready. I wanted to ask you both: do you have any concerns or recommendations based on the work you did to generate the revised Leipzig Spanish corpus or based on the first version of Engram-es? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Declutter threads.
From "known words" list:
internacional 157568
internacionalice 14
internacionalicemos 4
internacionalicen 6
internacionalidad 107
internacionalismo 291
internacionalista 610
internacionaliza 26
internacionalizaba 2
internacionalización 1898
internacionalizado 63
internacionalizamos 1
internacionalizan 11
internacionalizando 22
internacionalizar 232
internacionalizara 1
internacionalizaran 1
internacionalizaremos 1
internacionalizaron 11
internacionalizará 2
internacionalizaría 1
internacionalizó 42
internacionalmente 3514
From "unknown words" list: Are these (generally) supposed to be one word? Am trying to differentiate between "normal usage" and badly collected/cleaned text.
internacionala 4
internacionalactualización 2
internacionalalex 1
internacionalallan 1
internacionalamente 1
internacionalamerica 1
internacionalamericanaaustraliana 1
internacionalamérica 1
internacionalanimadora 1
internacionalargentina 1
internacionalargentino 1
internacionalarévalo 1
internacionalasdi 2
internacionalasistencia 1
internacionalasí 1
internacionalatentados 1
internacionalatlético 1
internacionalaunque 2
internacionalavisos 1
internacionalavtocomplex 1
internacionalbajo 1
internacionalbarcelona 1
internacionalbueno 1
internacionalcampeonato 1
internacionalcapítulo 1
internacionalcbi 1
internacionalch 1
internacionalchile 2
internacionalchilenos 4
internacionalcolombia 1
internacionalcomo 1
internacionalcon 3
internacionalconsecuencias 1
internacionalconsultor 1
internacionalcontactos 1
internacionalcoritiba 1
internacionalcruz 1
internacionalcti 1
internacionalcuando 1
internacionalcumbre 2
internacionalcuál 1
internacionalcáffaro 1
internacionald 1
internacionaldaniele 1
internacionalde 6
internacionaldeberia 1
internacionaldebido 2
internacionaldeclaran 1
internacionaldefinen 1
internacionaldenunciará 1
internacionaldesde 1
internacionaldetalla 1
internacionaldiario 1
internacionaldivisión 1
internacionaldocentes 1
internacionaldt 1
internacionaldónde 1
internacionale 10
internacionaleden 1
internacionaleei 1
internacionalees 1
internacionalel 17
internacionaleliminatorias 5
internacionalels 1
internacionalemente 4
internacionalemilio 1
internacionalemnte 1
internacionalemtne 1
internacionalen 6
internacionalenfrentar 1
internacionalentrenador 1
internacionalequipo 1
internacionalerp 1
internacionales 57489
internacionalesa 1
internacionalesactualmente 1
internacionalesaguas 1
internacionalesapuntan 1
internacionalesarzobispo 1
internacionalesasi 1
internacionalesatentados 1
internacionalesbank 1
internacionalesbogotá 1
internacionalesbolivia 1
internacionalesbush 1
internacionalescanciller 1
internacionalescarlos 1
internacionalescarrá 1
internacionalescasi 1
internacionalescelebran 1
internacionaleschina 1
internacionaleschávez 1
internacionalesclasificados 1
internacionalescolombianos 1
internacionalescomienza 1
internacionalescondecora 1
internacionalesconfirman 1
internacionalesconflicto 1
internacionalescontinúa 1
internacionalescrucial 1
internacionalescue 1
internacionalesde 2
internacionalesdebates 1
internacionalesdiosdado 1
internacionalesdirección 1
internacionalesdos 1
internacionalesdía 1
internacionaleseconómicos 1
internacionalesecuador 1
internacionaleseditar 1
internacionalesedward 1
internacionaleseeuu 1
internacionalesejército 1
internacionalesel 5
internacionaleseligen 1
internacionalesemilio 1
internacionalesempresa 1
internacionalesen 1
internacionalesentre 1
internacionaleses 1
internacionalesexigirán 1
internacionalesfavor 1
internacionalesfidel 1
internacionalesfin 1
internacionalesforo 1
internacionalesfrancia 1
internacionalesfuerzas 1
internacionalesgobierno 1
internacionaleshija 1
internacionaleshomenajean 1
internacionaleshttpwww 1
internacionalesintensifican 1
internacionalesinter 1
internacionalesinternas 1
internacionalesintervención 1
internacionalesinvita 1
internacionalesjornada 1
internacionalesla 5
internacionaleslas 2
internacionalesldquopero 1
internacionaleslondres 1
internacionalesmaria 1
internacionalesmartes 2
internacionalesmerkel 1
internacionalesmoscu 1
internacionalesnadie 1
internacionalesnorcorea 1
internacionalesobama 1
internacionalesola 1
internacionalesonu 1
internacionalespanama 1
internacionalespara 1
internacionalesparlamento 1
internacionalespor 1
internacionalesportal 1
internacionalespresidente 3
internacionalespresionan 1
internacionalesprimer 2
internacionalesprocurando 1
internacionalesque 1
internacionalesquinta 1
internacionalesrealizan 1
internacionalesreanudaran 1
internacionalesreelección 1
internacionalesretiran 1
internacionalesrodriguezunju 1
internacionaless 1
internacionalessanto 3
internacionalessin 1
internacionalessolo 1
internacionalestambién 1
internacionalesteban 1
internacionalesto 1
internacionalestrella 1
internacionalestrump 1
internacionalestá 1
internacionalesuniversidad 1
internacionalesunsam 1
internacionalesvenezuela 1
internacionalesverifica 1
internacionalesvie 1
internacionalesy 1
internacionales¨ 1
internacionalexige 1
internacionalexigen 1
internacionalfigo 1
internacionalfmi 4
internacionalfue 1
internacionalfundación 1
internacionalgareca 1
internacionalgoles 1
internacionalgran 1
internacionalgremio 1
internacionalguardiola 2
internacionalha 1
internacionalhay 1
internacionalhelicobacter 1
internacionalheynckes 1
internacionalhrw 1
internacionalhttpst 1
internacionalhttpwww 1
internacionali 1
internacionaliación 1
internacionalicardi 1
internacionalidades 89
internacionalifi 1
internacionalimagen 1
internacionalindia 1
internacionalinflación 1
internacionalinicia 1
internacionalinstituto 1
internacionalinteligencia 1
internacionalinternacional 1
internacionalinterpol 1
internacionalinvitado 1
internacionalis 3
internacionalisaac 1
internacionalismohttpwww 1
internacionalistaantiglobalización 1
internacionalistala 1
internacionalistaliga 1
internacionalistasolidaridad 3
internacionalius 1
internacionalivan 2
internacionalizaci 15
internacionalizacin 19
internacionalizacioacuten 1
internacionalizacion 2
internacionalizaciones 1
internacionalizació 1
internacionalizacióncolocandose 1
internacionalizaciónlan 1
internacionalizada 28
internacionalizadas 38
internacionalizador 1
internacionalizadora 1
internacionalizarla 3
internacionalizarlas 1
internacionalizarlo 6
internacionalizarme 3
internacionalizarnos 3
internacionalizarse 64
internacionalizarsey 1
internacionalize 1
internacionalizándose 3
internacionaljefe 1
internacionalk 1
internacionalla 20
internacionallas 3
internacionalle 2
internacionalledecky 1
internacionallejos 1
internacionalleyenda 2
internacionalllegando 1
internacionallo 3
internacionallocal 1
internacionallorenzo 1
internacionallos 1
internacionalluego 1
internacionalmaradona 1
internacionalmarketing 1
internacionalmaría 1
internacionalmayores 1
internacionalmenteacoge 1
internacionalmentede 1
internacionalmentedesde 1
internacionalmenteen 1
internacionalmenteinditex 1
internacionalmenteno 1
internacionalmento 1
internacionalmete 1
internacionalmisión 1
internacionalmourinho 3
internacionalmujer 1
internacionalmundial 1
internacionalmuralista 1
internacionalnacional 1
internacionalneymar 1
internacionalno 1
internacionalnos 1
internacionalobstaculizar 1
internacionaloea 1
internacionalopositores 1
internacionalordóñez 1
internacionalpara 2
internacionalparaliza 1
internacionalpaula 1
internacionalpelé 1
internacionalpermanecidas 1
internacionalpero 1
internacionalpochettino 1
internacionalpopulismo 1
internacionalpor 1
internacionalpreguntas 38
internacionalproblema 1
internacionalpronóstico 1
internacionalpágina 1
internacionalque 4
internacionalqueda 1
internacionalqué 1
internacionalramón 1
internacionalreal 1
internacionalrecibiría 1
internacionalreforzarán 1
internacionalregión 1
internacionalren 1
internacionalreporteros 1
internacionalreunión 1
internacionalribéry 1
internacionalriverboca 1
internacionalrueda 1
internacionalrumbo 13
internacionalsan 1
internacionalsanta 6
internacionalsantiago 1
internacionalse 2
internacionalsean 1
internacionalsección 1
internacionalselecciones 4
internacionalselección 3
internacionalsevilla 1
internacionalsi 1
internacionalsimeone 1
internacionalsin 1
internacionalsinceridad 1
internacionalsino 1
internacionalsorprendido 1
internacionalsúper 1
internacionalteléfono 1
internacionaltoutain 1
internacionaltreinta 1
internacionaltu 1
internacionalun 3
internacionalusted 1
internacionalvarios 2
internacionalvehículos 1
internacionalvenezuela 2
internacionalvideochatinicio 219
internacionalviernes 1
internacionalvulneración 1
internacionalyo 1
internacionalzación 1
internacionalísimo 1
Thanks, Ian
Beta Was this translation helpful? Give feedback.
All reactions