Delimiting patterns: `[ ... ]` vs `{ ... }`

[mihnita]

@markusicu wrote in #230 (comment):

… please use curly braces for those delimiters; square brackets are common enough in real text. Together with using curly braces for placeholders, only {} are special and need escaping.

{Hello world!}
{Hello {$name}!}

[mihnita]

Comments migrated from the slides

Options for consideration

---

Slides comment, Stanisław Małolepszy (@stasm), 11:10 PM Apr 21

(Forked from another comment) @mihnita wrote:

Why [ ] and not { }? Cases A and B (maybe also C and D) would work just fine with { } instead of [ ]

I suggested [ ] because I'd like to make it clear which delimiters are for code, and which are for text. It's always bothered me that ICU MF used { } to mean both of these things.

With [ ], it becomes instantly clear which text is translatable. This is particularly important when the source language is English, because then the source text is written in the same language as the names of functions, options, option values, and CLDR grammar features are.

With { }, I agree it's still possible to make it unambiguous to parsers, but I hypothesize that the cognitive cost for humans is greater than the risk of having to escape [ ].

---

Slides comment, Mihai Nita (@mihnita), 2:03 PM Apr 21 (edited 2:05 PM Apr 21)

Note for A and B: why [ ] and not { }?
Pros, cons?
Yes, less { } nesting, maybe a bit more readable.
But makes [ ] escapable. And [ ] is more commonly used than { }

Any extra thing with special meaning needs a justification.

[stasm]

As I stated in the comment reproduced above, I have a strong preference for using square brackets [...] for delimiting translatable text. Moreover, I think it would be a mistake to specifically use curly braces {...} for this purpose.

From the point of view of discoverability of syntax rules, I'd argue that it's easier to teach users that { means "open a code block" rather than that it means "delimit code or text, depending on the context".

For example, switching to {...} and using our current develop syntax, I find it confusing to imagine how the following message would parse in users' heads:

{$userName :gender}
    masculine {{$userName} added a new photo to his album.}
    feminine {{$userName} added a new photo to her album.}
    _ {{$userName} added a new photo to their album.}

This is made worse by the fact that the CLDR names like masculine, feminine, the names of variables like $userName, and lastly the names of functions like :gender are all most likely going to be written in English. The same language that a majority of source translatable content will be in.

This is why I'm advocating that one of our design goals be to make it as obvious as possible which parts of the message are meant to be translated. I'm proposing that we use square brackets to that effect:

{$userName :gender}
    masculine [{$userName} added a new photo to his album.]
    feminine [{$userName} added a new photo to her album.]
    _ [{$userName} added a new photo to their album.]

[markusicu]

• I think that [] are common enough in regular text that having to escape them would be more cumbersome than it's worth.
• I don't think that using {} for delimiters of both patterns and placeholders is overly confusing. That's not been a pain point with ICU MessageFormat, for example.
• The [] lend themselves much more for enclosing selectors and selection keys.

[stasm]

• I think that [] are common enough in regular text that having to escape them would be more cumbersome than it's worth.

I've been trying to collect some data on this. I estimate that fewer than 0.5% of messages contain square brackets in English and many other European languages. The percentage goes up for Japanese and Chinese (~1.5%) because these languages use square brackets as regular punctuation marks.

I think it's fair to assume that brackets are on average more common than curly braces ({}), but less common than round parentheses (()). At which point do we decide that they're common enough?

Also, we might want to consider which languages are most common as source languages, as they will be edited by hand more often than others. OTOH, target languages are more likely to be edited via tooling, in which case escaping requiremets matter less.

• I don't think that using {} for delimiters of both patterns and placeholders is overly confusing. That's not been a pain point with ICU MessageFormat, for example.

I can only offer my personal experience as well as some anecdotal data back from my Mozilla days. I've always found it very confusing that ICU MF used braces for both code and text. This was perhaps made harder for me because ICU MF also didn't use any prefixes for argument names nor functions, which could result in messages like {count, plural, one {One thing.} other {{count, number} things.}}, where every single piece of code is a valid English word, and there are no cues as to what is localizable and what is code.

[aphillips]

In my experience, any requirement for escaping in the plain text is a source of error. Translators and other non technical people are not prone to get it right and tooling only gets us so far.

I agree that the MF keywords provoke overtranslation and protection of peaceables is a constant worry.

I do prefer that in-text placeables look as codelike as possible and, as mentioned, that escapes be kept to an absolute minimum. The outer "quotes" don't matter so much (translators and others mostly won't see them) but using square brackets would require them to be escaped in line. Hence a preference for curly brackets.

[macchiati]

I echo "Translators and other non technical people are not prone to get it right and tooling only gets us so far."

…

On Tue, May 31, 2022 at 11:06 AM Addison Phillips ***@***.***> wrote: In my experience, any requirement for escaping in the plain text is a source of error. Translators and other noob technical people are not prone to get it right and tooling only gets us so far. I agree that the MF keywords provoke overtranslation and protection of peaceables is a constant worry. I do prefer that in-text placeables look as codelike as possible and, as mentioned, that escapes be kept to an absolute minimum. The outer "quotes" don't matter so much (translators and others mostly won't see them) but using square brackets would require them to be escaped in line. Hence a preference for curly brackets. — Reply to this email directly, view it on GitHub <#255 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMFH4UQSZM36W5BMMPTVMZIL7ANCNFSM5VWCXF7A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[stasm]

The outer "quotes" don't matter so much (translators and others mostly won't see them) but using square brackets would require them to be escaped in line. Hence a preference for curly brackets.

If we assume that translators and others mostly won't see the outer delimiters (which I agree with), it would be helpful from the tooling point of view to be able to also assume that a literal delimiter typed by a translator is meant to be part of the translation, and hence requires escaping when serialized to MF2 syntax.

This is the case for square brackets [] as pattern delimiters, because they have no other syntactical meaning. Does it also hold for the curly braces {}, which can also be used inside patterns to delimit placeholders?

Consider a message [Hello], using [] to delimit patterns:

• A translator will see it as Hello.
• If they now type Hello [square, the tooling can serialize this as [Hello \[square], because any square bracket typed by a translator is guaranteed to not be a pattern delimiter.

OTOH, consider a message {Hello}, using {} to delimit patterns:

• A translator will see it as Hello.
• If they now type Hello {curly, this actually looks like an unclosed placeholder. For someone familiar with MF2 syntax (and not relying only on CAT's editing UI) this could be a source of confusion, and could in fact lead to spurious and redundant escaping by hand.

[markusicu]

If they now type Hello {curly, this actually looks like an unclosed placeholder.

Only to a developer editing the message string, and they need to learn which characters may be or must be escaped.

A translator normally works in a tool that hides as much syntax as possible, so if they type any syntax-relevant character, the tool must escape it.

For reasons stated here and echoed by Addison and Mark, I favor curly braces to enclose patterns pretty strongly.

I also favor using square brackets for selector lists and variant key lists. If we could agree on syntax that is just as easy to parse and read but without square brackets for those lists, then maybe they could be ok for enclosing patterns. But I don't think that they are better for enclosing patterns than curly braces.

[stasm]

For reasons stated here and echoed by Addison and Mark, I favor curly braces to enclose patterns pretty strongly.

I see and acknowledge the agreement forming about using curly braces, but I'd like someone to address the concerns I had about mixing translatable and non-translatable parts of the message by using the same syntax for them.

So far we're in a situation where one group says "we don't think it's overly confusing", to which the other group responds "indeed, it is confusing to us and we've seen others confused as well". How do we get out of this?

I think we should take a step back and:

1. consider who the primary audience and authors of raw-syntax messages will be,
2. evaluate how much tooling and CAT tools can help, and in particular, what it cannot do,
3. decide what the acceptable cost of avoiding vs. requiring escaping is, i.e. what is the "absolute minimum" of escaping that we're not willing to go over, and why.