Tracking issue for `str_utf16` stabilization

[aturon]

The str_utf16 feature tracks APIs like str::utf16_units that support utf16 encoding. These APIs are working well today, but it would be good to have a clear picture of the final state of APIs we'd like to have for various string encodings in std, and how that relates to other unicode support crates.

cc @SimonSapin

[SimonSapin]

Regarding naming: utf16_units is not a good name. As far as I can tell, the choice of "code unit" in the Unicode Standard is mostly “what’s another word for "thing" that we haven’t used yet?” I suggest renaming str::utf16_units to str::encode_utf16 (and its return type accordingly, if it stays around at all).

Regarding the API: &str -> Iterator<Item=u16> in the middle of the balance between terseness and generality:

• If what you want is a Vec<u16>, having the method return an iterator makes you write "redundant" .collect() (or even .collect::<Vec<_>>()).
• If what you have is an iterator of char, collecting it into a String is wasteful as the only thing the method does with &str is get a str::Chars iterator out of it.

I suggest changing str::encode_utf16 to go all the way to the "convenience" side and make it return a Vec<u16>. (This is symmetrical with String::from_utf16* where neither input or output is an iterator.) char::encode_utf16 (#27784) can be used directly to avoid extra allocations in cases where a different type of input or output is desired.

[aturon]

While this still isn't ready for stabilization, I'm nominating for libs team discussion about @SimonSapin's proposals.

[retep998]

I've seen several people try to use this rather than OsStrExt::encode_wide when trying to work with Windows API. Perhaps some sort of warning about this would help.

[SimonSapin]

@alexcrichton why remove the needs-decision tag? What do you think of the changes proposed in my previous comment?

[alexcrichton]

Ah I misinterpreted @aturon's original intention, but the libs team decided yesterday to talk about this during the next triage meeting, and I-nominated is now our best way to do that, so I'm gonna leave as is :)