Fix rendering of multi-byte strings

[mattn]

https://docs.microsoft.com/en-us/windows/console/reading-and-writing-blocks-of-characters-and-attributes

https://docs.microsoft.com/en-us/windows/console/char-info-str

In latest cmd.exe, multi-byte strings must be handled with attributes of COMMON_LVB_XXX.

[mattn]

This fix issue that trembox-go on Windows Terminal does not work correctly with multi-byte strings.

[scrouthtv]

First of all thanks for the PR!
I have some questions:
I suppose an example for multibyte characters are chinese characters, e. g. 在.
I am testing this character using editbox.go provided in the _demos folder.

Current status with nsf/master is that multibyte characters overdraw some things:

Now using your fix, the same scenario using Windows 10 v19041 looks like this:

Three things I've noticed:

1. The bar at the end moves one character to the right
2. There is a space after the multibyte character
3. The cursor is off by one (I can't go further to the right, and deleting deletes the character right off the cursor, not left of it)

Now regarding 2. I thougt there is an issue with multibyte characters and my font, however this is the very same window but vim open and the character renders properly:

Do you have any clue how to further fix these?
I checked editbox.go and

• The bar at the end is drawn at an arbitrary position using termbox.SetCell (see lines 245 and 254)
• The character width is calculated using runewidth.RuneWidth (see line 13)

I've run all my tests in the new Windows Terminal, inserting multi-byte characters also does not work in a plain cmd.exe for me:

Notice how I first successfully insert the chinese character in the Windows Terminal, and then run the very same test in a plain cmd.exe but I can't insert the character using Insert.
I think this one is an issue with cmd.exe that we can't fix, since I can also not insert a multibyte character in the plain cmd prompt using Insert. I just noticed that raw_input.go is broken on Windows, so I can't test that.

I also would like to know whether this fix is backwards compatible with anything older than Windows 10 (7, 8, ...)?

Thank you for your patience and have a nice day!

[mattn]

What is your console code page? editbox.go should replace bars with ascii-borders if using CJK.

[scrouthtv]

My Windows language is set to English if that's what you mean. I simply copy-pasted a CJK character from google.

[mattn]

It's an East Asian Ambiguous Width. The width of bar might be changed with locales you are using. Also the width is not compatible on Windows and wcwidth(2). go-runewidth follows wcwidth(2).

[mattn]

Anyway, this is not related on this pull-request.

[scrouthtv]

Thank you. But what is about the cursor issue and the spacing after the cjk character (both in the second gif)? Am I the only one having it? Your screenshot doesn't look like it.

[mattn]

What value do you get with chcp on cmd.exe ?

[scrouthtv]

Active codepage: 437

[mattn]

Could you please try this code with go run ?

package main

import (
	"fmt"
)

func main() {
	fmt.Println(`
┌──┐
│在│
└──┘
`)
}

[mattn]

This is result on Linux (mintty terminal)

[scrouthtv]

PowerShell in the new Windows Terminal

[scrouthtv]

cmd.exe and pwsh.exe in the old conhost

[mattn]

@nsf the issue @scrouthtv mentioned is another issue that must be fixed. So let's merge this?