Rename symbols from `...utf8...` to `...cesu8...`

[martijnthe]

It wasn't obvious to me the resulting string for APIs like jerry_string_to_char_buffer() isn't actually UTF-8 encoded, but CESU-8. (FWIW, I understand and agree with the rationale behind using CESU-8 vs UTF-8 (as per the discussion in #616).)

It also seems like in the implementation itself, functions are sometimes labeled with ..._utf8_..., but in reality use CESU-8 (for example ecma_string_copy_to_utf8_buffer()).

I would like to suggest 3 things:

1. Rename all symbols (functions names, types, etc) that "falsely advertise" that they are dealing with UTF-8. For people that are new to the project and don't know about the recent transition from using UTF-8 to CESU-8 internally, this is very confusing.
2. Clarify in the docstrings with each API that can hand back/copy out a string to the client of JerryScript, is CESU-8 encoded, and for APIs that use strings as input, specify that they need to be CESU-encoded.
3. Over time, add APIs/wrappers for converting between CESU-8 <=> UTF-8.

[zherczeg]

Seems like a reasonable request.

[zherczeg]

Is there any public API which is misleading? We will fix it before the 1.0 release.

For internal types and functions we will probably do these renamings with other cleanups. Since they are internals, we will do this in due course.

[martijnthe]

Is there any public API which is misleading?

Hmm, no not really.
I do think it makes sense to have the public API deal with UTF-8 encoded strings by default instead of CESU-8, just because UTF-8 is very common while CESU-8 is not (and as per the spec it's not intended for "open information exchange").

If this would be a post-1.0 feature, maybe it makes sense to add ...cesu8... to the existing APIs and reserve the symbol names w/o such a post-fix for UTF-8?

[zherczeg]

I wouldn't change the current function names, because these are the "native" types. We could add utf-8 variants later emphasizing that those require conversions, which are costly.

[DemiMarie]

I believe that switching to CESU-8 may have been a mistake. The problem is that CESU-8 generally must be converted to UTF-8 before it can be shared with the outside world. For instance, if I want to bind to Jerryscript from Rust I need to either:

• do an O(n) conversion every time
• not use the built-in str type for JS strings.

[zherczeg]

JavaScript native character type is 16 bit character. Splitting unicode characters on-the-fly adds too much work for a 80MHz CPU and code complexity which increases binary size. We can add cesu-8 utf-8 conversion utilities though. The conversion is cheap on desktops/servers.

[robertsipka]

#1430 was merged, so now we able to create a string from a valid UTF-8 string via the API.

[zherczeg]

Closing due to inactivity. Please reopen if needed.