Why should strings be lists of Unicode Scalar Values?

[lukewagner]

This issue lays out the reasoning for why I think strings should be lists of Unicode Scalar Values (as currently written in the explainer). This is a fairly nuanced question with the reasoning currently scattered around a number of issues, repos and specs, so I thought it would be useful to collect it all into one focused issue for discussion. The issue reflects discussions with a bunch of folks recently and over the years (@annevk, @hsivonen, @sunfishcode, @fgmccabe, @tschneidereit, @domenic), so I won’t claim credit for the reasoning. Also, to be clear, this issue only answers half of the overall question about string encoding, but I think it’s the first question we have to answer before we can meaningfully talk about string encodings.

(Note: I intend to update the OP in-place if there are any inaccuracies so that it represents a coherent argument.)

First, a bit of context:

Current proposal

As background, the Unicode standards provides two relevant definitions:

• Code Point: an integer referring to the Unicode codespace in the range [0, 0x10FFFF].
• Unicode Scalar Value (USV): a Code Point other than a surrogate and thus an integer in one of ranges [0, 0xD7FF] or [0xE000, 0x10FFFF].

Based on these definitions, the current explainer proposes:

• The char interface type is a USV
• The string interface type is an abbreviation for list char

Thus, string, as currently proposed, contains no surrogates (not just no lone surrogates). For reference: a pair of surrogate Code Units in a valid UTF-16 string is decoded into a single USV and thus valid UTF-16-encoded strings will never decode to strings containing any surrogates.

This is not an encoding or in-memory-representation question

The question of whether strings are lists of Unicode Scalar Values is not a question of encoding or memory representation; rather, it’s a question of: “what are the abstract string values produced by decoding and consumed by encoding?”. Without precisely defining what the set of possible abstract string values is, we can’t even begin to discuss string encoding/decoding since we don’t even know what it is we’re trying to encode or decode. This is especially true in the context of Interface Types, where our goal is to support (via adapter functions) fully programmable encoding/decoding in the future.

Thus, if we’re talking about the abstract strings represented by languages like Java, JS and C#, we’re not talking about “WTF-16” (which is an encoding); we’re talking about “lists of code points not containing surrogate pairs (but potentially containing lone surrogates)”, which for brevity I’ll call Wobbly strings, since Wobbly strings are what a Java/JS/C# string can be faithfully decoded into and encoded from. In particular, a Wobbly string can be encoded by either WTF-8 or WTF-16. Note that the set of Wobbly strings is subtly different and smaller than “lists of Code Points” because surrogate pairs decode into necessarily-non-surrogate code points, so there is no way for a Java/JS/C# string to decode into a surrogate pair. The only major languages I know of whose abstract strings are actually “lists of Code Points” are Python 3 and Haskell.

This is a Component Model question

As of our recent CG-05-25 polls, the Interface Types proposal now has the goals and requirements of the Component Model (as presented and summarized). Concretely, this means we’re explicitly concerned with cross-language/toolchain composition, virtualizability and embeddability, which means we’re very much concerned with whether interfaces using string will be consumable and implementable by a wide variety of languages and hosts with robust, portable behavior. Thus, use cases exclusively focused on particular combinations of languages+hosts may need to be solved by separate proposals targeting those specific languages+hosts if they are in conflict with the explicit goals of broad language/host interoperability.

With all this context in place, I’ll finally get to the reasons for defining string to be a list of USVs:

Reason 1: many languages have no good way to consume surrogates

I think there are a few categories of affected languages (this is based on brief spelunking, so let me know if I got this wrong and I’ll update it):

First, there are languages that simply fix UTF-8 for their built-in string type, in some cases exposing UTF-8 representation details directly in their string operations. The popular languages I found in this category are: Elixir, Julia, Rust and Swift.

Second, there are languages which define strings as “arbitrary arrays of bytes”, leaving the interpretation up to the library functions that operate on them. For the languages in this category that I looked into, the default encoding (for source text and string literals and sometimes built-in syntax like iteration) is increasingly implicitly assumed to be UTF-8 (due to the fact that, as detailed below, most I/O data is UTF-8). While it may seem like these languages have the most flexibility (and thus ability to accommodate surrogates), when porting existing code, the implicit dependency on UTF-8 (in the form of calls to UTF-8-assuming library functions scattered around the codebase) makes targeting anything other than UTF-8 challenging. The popular languages I found in this category are: C/C++, Go, Lua, PHP and Zig.

Third, there are languages that support a variety of encodings and conversion between them, but still disallow surrogates (among other reasons being that they aren’t generally transcodable). The popular languages I found in this category are: R and Ruby.

In all of these categories, the author of the toolchain that is binding the language to the Interface Types string has no great general option for what to do when given a surrogate:

• Make incoming surrogates trap. This approach is attractive as it simply makes surrogates “someone else’s fault”, and thus not a corner case that all code in the language’s ecosystem has to worry about. This is an easy answer to pick, however it would make these languages second-class in the component ecosystem because they wouldn't be able to implement the same APIs.
• Replace incoming surrogates with the replacement character. This happens by default in many places in many of the above languages that I saw, so it’s also a reasonable default option that avoids putting any burden on the language ecosystem at large. But, as with the previous option, this would make these languages second-class as they wouldn’t be able to faithfully implement the same APIs as other languages.
• Produce non-UTF-8 byte strings. This isn’t possible for languages in the first and third categories and risky for languages in the second, due to the increasingly prevalent implicit assumption of UTF-8 noted above. Moreover, unlike the above two options, this is not a “spot fix”: it requires all ported code to use the appropriate non-UTF-8 string operations.
• Escape surrogates into valid strings. This could make various simple round-tripping use cases Just Work, without hitting the above snags, but this option implicitly introduces a new micro-format that will need to be supported by any non-trivial string operation that works with the contents of the string (e.g., file system operations), so it’s also not a “spot fix”; it needs ecosystem adoption. Also, escaping can introduce collisions (leading to data corruption) with pre-existing strings since there are no code point sequences reserved for this purpose.
• Produce a non-standard/builtin string. This option either requires large-scale changes (converting whole codebases to using the new, non-standard string), which blocks porting use cases, or requires a coercion some time later into the standard string, which means picking one of the above options.

For any particular use case, one of these options may be obvious. However, toolchains have to handle the general case, providing good defaults. In addition to the variable ecosystem cost of the different options, there is also a fixed non-negligible cost in wasted time for the N teams working on the N language toolchains, each of which will have to page in this whole problem space and wade through the space of options. In contrast, with a list of USVs, all the above languages can just do the obvious thing they’re already doing.

Reason 2: strings will often need to be serialized over standardized protocols and media formats, which usually disallow surrogates

A common use of Interface Types will be to describe I/O APIs (e.g., for passing data over networks or reading/writing different media formats). Additionally, several of the component model’s virtualizability use cases involve mocking non-I/O APIs in terms of I/O (e.g., turning a normal import call into an RPC, logging call parameters and results, etc). In both these cases, surrogates run in direct conflict with the binary formats of most existing standard network protocols and standard media formats.

In particular, just considering Web-relevant character sets:

• The RFC 2277: IETF Policy on Character Sets and Languages specifies that “When using other charsets than UTF-8, these MUST be registered in the IANA charset registry, if necessary by registering them when the protocol is published.”, and there are no IANA charsets that include surrogates.
• The W3C Architectural Specification specifically calls out “Specifications MUST NOT allow the use of surrogate code points.”
• The preface of the WHATWG Encoding Standard says “The UTF-8 encoding is the most appropriate encoding for interchange of Unicode, the universal coded character set. Therefore for new protocols and formats, as well as existing formats deployed in new contexts, this specification requires (and defines) the UTF-8 encoding.”
• If a format is to support line-breaking or collation, the Unicode specification says the behavior given surrogates is undefined, possibly resulting in an error.
• All XML-based formats reject surrogates.
• Popular RPC protocols such as Protobufs and CapnProto mandate UTF-8 for strings/text.
• GraphQL strings mandate UTF-8.

On the Web, new APIs and formats created over the last 10 years simply mandate UTF-8, including:

• RFC 8259, for all JSON documents that aren’t shared as part of a closed ecosystem.
• JS files loaded in newer Worker and ES Module contexts.
• The WebSockets text stream APIs.
• The json and text getter functions of fetch, XHR and Blob APIs.

There’s also a recent proposal to make this direction more-officially part of the W3C’s design principles.

Thus, insofar as a string needs to transit over any of these protocols, formats or APIs, surrogates will be a problem and the implementer of the mapping will have roughly the same bad options listed above as the language toolchains have.

While it’s tempting to say “that’s just a specific precondition of particular APIs, not the string type’s problem”, the virtualization goals of the component model mean that any interface might be virtualized, so the fact that a string is being used for one of the above is not a detail of the API. In contrast, all these protocols and formats can easily represent lists of USVs.

Reason 3: even the WTF-16 languages will have a bad time if they actually try to pass surrogates across a component boundary

Because of the above two reasons, from the perspective of a WTF-16-language-implemented component, it is a very risky proposition to pass a surrogate across a component boundary (parameter of an import or result of an export). Why? Because there’s no telling whether the other side will trap, convert the surrogate into a replacement character, get mangled or trigger undefined/untested behavior. As an author of a component, there’s also not a fixed set of clients or hosts (that’s the point of components).

Thus, to produce widely-reusable and portable components, even a toolchain for a language that allows lone surrogates would be advised to conservatively scrub these before passing strings to the outside world. In a sense, this is nothing new on the Web: despite JSON being derived from JS, JSON doesn’t allow surrogates while JS does, thus there is an inherent scrubbing process that happens when JS communicates with the outside world via JSON (and similarly with WebSockets, fetch(), etc). Accordingly, the WTF spec specifically advises against ever being used outside of “self-contained systems”.

As an illustrative example: consider instead defining string to be a list of Code Points. As explained above, this would mean string was a superset of the Wobbly strings supported by Java/JS/C#. Why might we do this? For one thing, it would capture the full expressive range of Python 3 and Haskell strings and APIs (which is the same argument for supporting Wobbly strings, just for a smaller set of languages). For another, it would give us a simple definition of char (= Code Point) and string (= list char), which has a number of practical benefits (in contrast to Wobbly strings, which cannot be a “list char” for any definition of “char”). However, now the vast majority of languages and hosts would have to resort to a variant of the abovementioned workarounds which means Python 3 and Haskell would have a Bad Time attempting to actually take advantage of this increased string expressivity. Thus, there would be a distributed cost without a commensurate distributed benefit. I think the situation is the same with Wobbly strings, even if the partitioning of languages is different.

What about binary data in strings?

One potential argument for surrogates is that they may be necessary to capture arbitrary binary data, particularly on the Web. To speak to this concern, it’s important to first clarify something: Web IDL has a ByteString type that is used for APIs (predominantly HTTP header methods like Headers.get()), where a ByteString is intentionally an arbitrary array of bytes. However, ByteString does this not by interpreting a JS string as a raw array of uint16s (which would have a problem representing byte strings of odd length), but by requiring each JS string element (a uint16 value) to be in the range [0, 255], throwing an exception otherwise. Since surrogates are outside the range [0, 255], this means that the one place in the Web Platform where binary data is actually appropriate, surrogates are irrelevant.

Outside ByteString use cases, there’s still a theoretical possibility of wanting to round-trip binary data through DOMString APIs. Talking to folks who have worked for years on the Web IDL and Encoding specs (@annevk, @hsivonen, @domenic), they’re not aware of any valid use cases for such usage of DOMString. Indeed, the TextDecoder API does not provide any way to produce a non-USVString, due to this same lack of use cases. In fact, there is currently no direct way (i.e., not involving String.fromCharCode et al) to decode an array of bytes into a non-USVString on the Web Platform today.

Instead, the natural way for a component to pass binary data is a list u8 or list u16, using JS glue code to convert the byte array into a JS string. If these use cases were found and found to be on performance sensitive paths in real workloads on the Web, then it seems like a Web-specific solution would be appropriate, and I can think of a number of options for how to optimize this path by adding things to the JS API. But ultimately, as an optimization, I don’t think this is something we should preemptively add without compelling data.

[aardappel]

Wrapping my head around why this is necessary, I found it funny that the reason this issue even exists is because we have a range of values that are dedicated to indicating extension in a 16-bit scenario, that do not overlap (have a unique value) regardless of whether we are actually using a 16-bit encoding. Most encodings that provide extension values overlap with the values being encoded, e.g. in UTF-8, values 0x80..0xFF indicate an extension is present, but these are not unique as code points since 0x80..0xFF in code points are actual characters. So this question never comes up. So UTF-16 would have been better off with an encoding that overlaps, but I guess that would have made it less easy for software to ignore that UTF-16 is actually variable size..

[dcodeIO]

Interesting thought :) Makes me wonder in turn if the Unicode standard could reasonable settle this decade long issue by adopting what WTF-8 does, and perhaps, say, assigning the visual representation � to (lone) surrogate code points as well. Then it would overlap, roundtrip both ways, that is if I am not missing something, but most notably render future discussions for or against obsolete. Perhaps not labeling it "potentially ill-formed" but "relaxed", "lenient" or "practical" would have helped the situation as well, heh. Would require, however, that UTF-8 implementations need to merge previously split surrogates upon concatenation, and yeah, that would be a considerably large nudge. Too large, most likely.

[dcodeIO]

Thanks for the thorough writeup, Luke :)

This is a Component Model question

As of our recent CG-05-25 polls, the Interface Types proposal now has the goals and requirements of the Component Model (as presented and summarized).

I have been one of the against votes, so perhaps it makes sense to explain why I voted this way. My impression was that deciding for all the next steps was too early. String semantics/encodings in particular have been hotly discussed in the past without resolution so far, so it felt a little odd to me to tie this question (i.e. concretely propose USVs/UTF-8 as if it was natural) to the component model. I worried that the group has not been sufficiently informed on this particular ingredient before polling, which was my motivation for rushing out my presentation with my concerns the weekend before to provide some background. I recognize that this may not have been your intention, of course, but this was my thought process at the time. I appreciated that you clarified during the meeting that we are not actually polling on strings, but now I must admit that I am a little confused as the presence of the component model is used as an argument.

I also voted neutral on the general direction of the component model, because I do not see how it helps the more Web-focused use cases and anticipations I am seeing. Now I was not against it (if others want components I would be fine with it), but if it turns out that its existence is used to justify harming other straight-forward use cases (say where one's component is basically the combination of Wasm + JavaScript), I would decide differently in the future when similar questions arise.

Concretely, this means we’re explicitly concerned with cross-language/toolchain composition, virtualizability and embeddability, which means we’re very much concerned with whether interfaces using string will be consumable and implementable by a wide variety of languages and hosts with robust, portable behavior.

Here, I believe the "robust" part goes both ways:

• If we pick "list of Unicode Scalar Values" (subset, UTF-8/16) we are going to randomly break some languages and their JS interop
• if we pick "list of Unicode Code Points except surrogate pairs" (superset, WTF-16/8) we are not hard breaking anyone

I understand of course that

Reason 1: many languages have no good way to consume surrogates

but I think this is a rather weak argument compared to making occasional breakage the default for many languages on the contrary. I, and perhaps others as well, would prefer a component that works 100% of the time with an additional check over a component that works just 99.9% of the time while risking anything from annoyances to hazards otherwise. A fuzzer for instance will find this reliably, and so will millions of unintentionally fuzzing users who are not necessarily aware of all the ins and outs of string encodings.

Also

Reason 2: strings will often need to be serialized over standardized protocols and media formats, which usually disallow surrogates

may be true, but is in my opinion not a very compelling precedent for what should happen in between two function calls. The more modular Wasm becomes, the harder it will become to tell where a function lives respectively if a string argument/return crosses a boundary or not. Plus, what may work today may stop working in the future, and we are certainly on a trajectory towards more breakage, not less.

As such I do not think that basing

Reason 3: even the WTF-16 languages will have a bad time if they actually try to pass surrogates across a component boundary

on the above two reasons is very meaningful. The typical case for these languages will most likely be to interface with modules written in the same language or with JavaScript. Perhaps also WASI here or there, but WASI mostly consumes strings, say as file system paths, which are fine to be "not found", while otherwise returning either raw bytes or well-formed strings anyhow. As such I would question the practical value of this reason.

And of course this does not only apply to

What about binary data in strings?

in that some languages, by string API design, make it overly easy to accidentally split a surrogate pair into half (can be as easy as a substring(0, 1), sometimes yielding something akin to "binary data"), especially when a user is unaware of the underlying string encoding details. And the creators of these languages didn't deliberately decide to make it that easy, but it's merely a side-effect of them valuing backwards-compatibility even more when they upgraded from UCS-2 to UTF-16 with the Unicode Standard not leaving them a better choice. So I think we should do the same, and value backwards-compatibility over Unicode hygiene, which may seem useful in theory but in practice often is the opposite, so we can serve these languages well.

If these use cases were found and found to be on performance sensitive paths in real workloads on the Web, then it seems like a Web-specific solution would be appropriate

I can appreciate the direction here of at least keeping the door open to solve it in the Web embedding, but I would once more want to emphasize that this is not a problem exclusive to the Web embedding. I do not see, for instance, how a toolchain would decide whether to utilize a different type or just use a string, as it may not have the necessary knowledge upfront and cannot statically evaluate the contents of every string. Also, in many cases, a simple module will just be the entire component. This is also especially problematic in code migration, where I totally expect JavaScript modules becoming replaced with WebAssembly modules over time. In the current form, this can break for any language, while otherwise it would only happen for some of them, and likely not in the typical case.

On a more general note, it was obvious to me since the beginning that higher level languages like Java will have a better stab at seamlessly integrating with the Web platform, not only because some of them share a string encoding with JavaScript, but also because they already have matching concepts of, say, strings being references, that in the ideal case can be shared with JavaScript, say with GC where everything lives in a common heap anyway. The component model with its restrictions, on the other hand, seems as if it is on a different trajectory that may in the future even influence other proposals as a kind of precedent, and this makes me really sad because I always hoped we could embrace the sheer potential of a future where JavaScript and WebAssembly become one ("component").

Lastly, in the presence of a proper escape hatch for affected languages, I would be fine with a default string type that leads us into a well-formed feature. But without a mechanism that reliably prevents occasional breakage, I fear we are about to get into real trouble (i.e. in the worst case CVEs on or in combination with the IT MVP for applications that worked perfectly fine including when transpiled to JS but not anymore when compiled to Wasm), and would hence strongly prefer "list of Unicode Code Points except surrogate pairs" (WTF-*) over the much less compelling argument of Unicode hygiene.

[lukewagner]

I appreciated that you clarified during the meeting that we are not actually polling on strings, but now I must admit that I am a little confused as the presence of the component model is used as an argument.

I don't think it's possible to decide an issue like this one in the absence of an agreed-upon set of goals, use cases and requirements, which is what "the component model" refers to. Now that we have strong CG agreement on this scope, it gives us the appropriate context in which to discuss this question. I don't think there's an alternative approach to hard questions like this.

I also voted neutral on the general direction of the component model, because I do not see how it helps the more Web-focused use cases and anticipations I am seeing.

It's totally reasonable not to be particularly motivated by the component model -- it's not expected to address 100% of use cases or be a universal answer to all interoperability questions; that's why we've adopted a layered approach. However, it's clear that there are many other folks who are strongly motivated by these goals, so I don't think we can simply set aside the component model in this discussion. For a different set of goals, a different proposal is appropriate.

If we pick "list of Unicode Scalar Values" (subset, UTF-8/16) we are going to randomly break some languages and their JS interop

It won't be random, as it will happen quite regularly and independent of the context in which the component is embedded. In contrast, allowing surrogates to cross component boundaries would lead to the random (from the perspective of an individual component) failures. This is the crux of the matter when you consider the full goals of the component model. I agree that if you're restricting your set of goals to focus more exclusively on JS and Web this is less of a concern, but that's not the context of this layered proposal.

I want to reemphasize another point which is: like wasm, the component model is not a one-shot standard that has to have everything from day 1. Like wasm, our goal is to start with an MVP and iterate based on real-world experience. Thus, the question to ask isn't: "do there exist any use cases for passing surrogates?" but, rather, "will the initial release of the component model not be viable without the ability to pass surrogates between components?". The data we have here is the years of experience of folks working on Web standards suggests that surrogates are not necessary, which is supported by all the standards evolution described above.

[dcodeIO]

Given Murphy's law, what you are proposing seems unnecessarily risky. The real-world experience we want to obtain is to some extent (rare) breakage, and that doesn't make it a proper foundation to build a house on for me. I would once more like to remind of the claim that post-MVP "is just an optimization", which is what we voted on, but as far as I am concerned remains unproven, plus not everyone in the group may have been properly informed about when placing their vote.

As such I would argue that we are better off when starting with the more inclusive string semantics that are true to the just-an-optimization aspect and gives us the foundation we need to iterate in the future. This adds three more options at the end of the day:

1. restricting string semantics post-MVP when we can actually be 100% sure
2. accounting for this case in well-formed languages' standard libraries (merge lead/trail surrogates upon concat)
3. adding an ideal copy-only WTF16String to well-formed languages' standard libraries or ecosystems (i.e. opt-in when interfacing with JS)

Pre-existing "experience of folks working on Web standards" doesn't convince me at least, especially because we are going to compile a lot of stuff to the Web that hasn't been there before.

Btw, it could be as integrated as adding an option like list.lower_canon sanitize=true/false while even retaining the option of well-formedness. I would be fine with that, as I consider well-formed use cases important as well, but I would not be OK with leaving the more inclusive use case out. Add UTF-16/32/Latin-1 to the mix one day and we are in a position that we may not even need adapter functions (for strings) anymore. Respective MVP requirement could be: Support Unicode-like string encodings observed in the wild without the need for adapter functions. That's inclusive, that's neutral, that's minimum viable imo. And, of course, I would finally shut up. Even better, I would be fully on board :)

[lukewagner]

Today, whenever JS talks to the outside world (through HTTP APIs, JSON, gRPC, etc), surrogates are replaced with replacement characters and noone considers this data loss because that's simply the expectation when talking to the outside world. As shown in this slide, the component model is not meant to take the place of language-specific modules/packages/libraries, but, rather, encapsulate linked collections of these (making components more like lightweight processes). Thus, the component model is explicitly adopting the same "talking to the outside world" model where surrogates are not expected.

If we are really worried about silent breakage, though, the fix would be to have surrogates trap instead of producing replacement characters, so the errors could be caught early and fixed easily. I'm open to discussing that more.

(Also, just to clarify, "post-MVP" doesn't refer to a singular follow-up proposal (adding adapter functions), but rather a long sequence of feature proposals over time, the same as with core wasm. Thus, post-MVP is not restricted to only being for optimization by any means.)

[dcodeIO]

I am still not convinced that these APIs, that require well-formedness under the hood by definition of being HTTP APIs, JSON or gRPC, set a compelling precedent for what shall happen in between function calls of modules potentially written in the same language, or when interfacing with JS. As far as I am concerned we are comparing apples and oranges here, since a bunch of function calls that do not sanitize (as it would risk to corrupt half-way) typically happen prior to hitting say the HTTP API that does sanitize because it has a very good reason to. Anticipating that every Wasm module utilizing IT is akin to these APIs seems like quite a stretch.

The theoretical other extreme would be to consider that every string API call should sanitize, which would break languages straight away. We are somewhere in the middle, and given that there are even more APIs in JS for instance (that can be considered modules), and that these deliberately do not sanitize since there is no reason to risk that, I would say we are much closer to function calls here. Unless we want to encourage shipping monoliths only, perhaps, but I am not sure that's a goal :)

(Btw, I'd absolutely prefer replacement over erroring for separate reasons if my viewpoint cannot find consensus. Appreciate UTF-16/Latin-1 being considered.)

[lukewagner]

I think there is plenty of room in the wasm ecosystem for new ABIs specifically designed for allowing closely-related languages to integrate more tightly than the component model allows. This is already the case with the existing tooling-conventions ABI which is the basis for C/C++/Rust(/FORTRAN?) to link together and pass pointers to memory and functions back and forth. I can imagine another totally different ABI designed specifically around native-JS integration that could be more like what you want. But for the component model, the virtualization goals imply that a component should never assume it knows the language (and whether its wasm or native) of its imports nor of the caller of its exports.

[dcodeIO]

Do you think this could be designed in a way that it becomes composable? Say, either use just Interface Types to achieve something lightweight as I am envisioning, or Component Model over Interface Types for stronger guarantees? Like, so far it really only differs in string well-formedness as far as I am concerned.

One way to achieve this perhaps could be if JS could participate in a Component (achieving Wasm + JS inside) as if it were just one of many modules, but outside of the component we'd enforce stricter guarantees that are useful in more complex scenarios.

[dcodeIO]

Some use cases I have in mind are:

• Use APIs that give you unpaired surrogates with WebAssembly. This appears to be true for keyboard events in browsers currently, and there may be many more anomalies like this in the various languages that we want to compile to Wasm.
• Compile C#/Java/JS-like string manipulation routines to WebAssembly and use them as a module, either from within JavaScript or from other WebAssembly modules. Would currently sanitize ephemeral isolated surrogates early and make it unfeasible. One more concrete use case here is to provide a JSString for interop purposes that can be used in Non-Web environments.
• Compile a StringBuilder or similar string utility to WebAssembly and use it as a module. Similar to string manipulation routines, this would currently prematurely sanitize and be unfeasible.
• Compile string encoders or decoders to WebAssembly and use them as a module, either from within JavaScript or other WebAssembly modules. Would currently be unfeasible for Unicode-like encodings because of mandatory lossiness.

On the other hand, there is one fun use case being possible with the USV restriction: wasm-string-sanitize, a practically zero code string sanitizer that works universally across every environment supporting Interface Types. Not saying that someone should build that, as it would lead the whole thing ad absurdum, but perhaps good to have in mind that someone could indeed build this.

[kripken]

I apologize if this is a digression (but maybe it can help?). It seems to me that there are really two use cases here:

1. For the Web interop use case, it seems natural to me to use an externref to a JS string anyhow. As @lukewagner suggested in the past, pre-importing of JS methods for string operations could make that fast, if a wasm module needs those methods. That is, this path would not use interface types. Looking at it from another angle, interface types works under the assumption of shared-nothing, so the normal thing is to copy a string when crossing the boundary. But for JS interop we want to use actual JS strings without copying - like languages compiled to JS do today. For wasm to really fit into that JS ecosystem of languages, it needs to use externref.
2. For the wasm components use case, I agree UTF8 is the cleaner solution. Yes, that means some amount of copying and conversions for some languages, but that's expected in the shared-nothing model, so we may as well use the best encoding, UTF8. Perhaps the type could even be renamed from string to utf8 to make that explicit?

(Btw, note that wasm on the web would use option 2 when it wants to use interface types to communicate with wasm components; point 1 is just about interop with JS.)

I think there is no perfect way to have optimal web interop as well as optimal UTF usage at the same time / with the same code. We will have the downside of people compiling differently when JS interop is their preference, but that will be the same with a wasm VM embedded on the JVM or CLR I believe, where, again, using the native string may be important for speed and compatibility.

So in practice we may see these two patterns of usage, externref for "host strings" and IT strings for "generic string type among wasm components". It would have been really nice if we could have just one pattern, of course. But wasm is a low-level assembly, and overlapping solutions can make sense to get optimal behavior on different use cases.