Pasting unicode characters from clipboard doesn't work

[lithammer]

Basically pasting unicode characters from the clipboard produces some kind of escape sequence (or garbage?).

This is what it looks like in Neovim when I try to paste → for example:

And this is what regular Vim does:

This is on OS X (10.10.3). And I used -u NONE. You can pull up a table for unicode characters by pressing ctrlcmdspace

[mhinz]

I can't reproduce that with the latest neovim on neither 10.10.2 nor 10.10.3.

1. If you copy an unicode character to the clipboard, what does the output of :reg for "* and/or "+ look like?
2. I'm not sure if this can be an issue, but what's the output of :echo &clipboard?

[lithammer]

I guess I forgot to mention that I used cmdv to paste or clicking via the "Emoji & Symbols" menu (see screenshot).

Here's part the output of :reg:

:reg
--- Registers ---
"-    <path>
"*   →
"+   →
".   â<92><80>kb<80>kb

So actually pasting with "*P or "+P works as intended.

[yagebu]

I can reproduce this with the latest homebrew-built neovim on OS X; on Arch Linux I can't reproduce it. This also happens when I type Unicode characters directly (using a non-US keyboard). Typing € or ß for example produces garbage.

[mhinz]

Hmm, pasting via "+p and just double-clicking an emoticon in that dialog inserts it in Vim just fine.

@Renstrom How did you install neovim?

[lithammer]

How did you install neovim?

I installed using Homebrew as described in neovim/homebrew-neovim.

This also happens when I type Unicode characters directly (using a non-US keyboard). Typing € or ß for example produces garbage.

I'm also getting this, I'm using a non-US keyboard as well (Swedish to be exact). Typing € prints â¬

[fwalch]

Just as an idea if you want to try, does it work with https://github.com/rogual/neovim.app? Or with a different terminal emulator?

[mhinz]

As far as I see it, 2 people can reproduce it only with neovim built using that homebrew recipe.

Bruteforce method:

$ cd /tmp && git clone https://github.com/neovim/neovim.git && cd neovim && make && VIMRUNTIME=/tmp/neovim/runtime build/bin/nvim

Does it work there?

[kopischke]

I cannot reproduce this on OS X 10.10.3 with NVIM 0.0.0-alpha+201504082319 (compiled Apr 9 2015 16:29:30) Commit: bae1da3, German locale, German keyboard map. Both using the character picker and inserting extended characters via the usual alt+<char> mechanism work just fine in Neovim running in Terminal.app. So does pasting from the system pasteboard via cmd-v.

[yagebu]

I have found the culprit (at least in my setup). The locale was not fully set. Apparently the setting "Set locale variables automatically" doesn't do enough for neovim. It only sets LC_CTYPE=UTF-8 in iTerm2 (same for a similar setting in Terminal.app). Setting LC_ALL to some sensible locale fixed it (More precisely LC_MESSAGES did the trick)

I'm not really sure which of the two is the right environment variable to consider, but as standard vim worked as we expected for both myself and @Renstrom, it might make sense to consider both.

Thanks for all the quick help here!

[lithammer]

It seems you're correct. This is the locale I had before.

$ locale
LANG="en_US"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

To get stuff working I had to change LANG:

-LANG="en_US"
+LANG="en_US.UTF-8"
 LC_COLLATE="en_US.UTF-8"
 LC_CTYPE="en_US.UTF-8"
 LC_MESSAGES="en_US.UTF-8"
 LC_MONETARY="en_US.UTF-8"
 LC_NUMERIC="en_US.UTF-8"
 LC_TIME="en_US.UTF-8"
 LC_ALL="en_US.UTF-8"

Thanks for the help guys!

[kopischke]

@yagebu

(same for a similar setting in Terminal.app)

Nitpick: Terminal.app seems to do the right thing in non-English locales. My locale output confirms the diagnosis, BTW:

LANG="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_CTYPE="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_ALL=

[lithammer]

I guess a takeaway from this is that currently Neovim and Vim differ in how they handle locale settings.

[yagebu]

@kopischke: after some testing it seems Terminal.app does the right thing if and only if it can set all locale variables to the same locale. If region and language (in OS X prefs) are set to different things, instead of setting the locale accordingly, it just doesn't set it (except for LC_CTYPE). The behaviour of iTerm seems to be identical...