Extra spaces in markup

[mihnita]

Our current grammar for markup is this:

markup = "{" [s] "#" identifier *(s option) *(s attribute) [s] "}"  ; open
       / "{" [s] "#" identifier *(s option) *(s attribute) [s] "/}" ; standalone
       / "{" [s] "/" identifier *(s option) *(s attribute) [s] "}"  ; close

That's what it looks like after adding adding options to close.
But the issue is independent.

I would like to make the case that the first space after { is not only unnecessary, but somewhat problematic.

1. It forces us into a (possibly big) lookahead.
   We see the {, and then we might have to "consume" a lot of spaces to know where we are (markup or expression)
   I decided to strikethrough because it is the least compelling argument. It is not about the parsing, at all. [mihnita]
2. The open / close attribute is something associated to the whole markup, not the identifier.
   It is this whole marker that is standalone or close, not the attribute
3. It is somewhat internally inconsistent.
   The / at the end of the standalone is tied with the closing }, there is no [s] in between them.
   This was true before PR Add options to close (spec) #649, but it is more visible now.
4. It is even invalid in HTML and XML.
   None of this works: < b> and < /b> and </ b>

[mihnita]

For convenience I will include the HTML below unprotected, so that we can see it in the issue:

<p>Say something <i>italic</i> and last (works).</p>
<p>Say something <  i>italic</i> and last (fails).</p>
<p>Say something <i  >italic</i> and last (works).</p>
<p>Say something <i>italic<  /i> and last (fails).</p>
<p>Say something <i>italic</  i> and last (fails).</p>
<p>Say something <i>italic</i  > and last (fails).</p>

Say something italic and last (works).

Say something < i>italic and last (fails).

Say something italic and last (works).

Say something italic< /i> and last (fails).

Say something italic and last (fails).

Say something italic and last (fails).

[mihnita]

Proposed change:

markup = "{" [s] "#" identifier *(s option) *(s attribute) [s] "}"  ; open
       / "{" [s] "#" identifier *(s option) *(s attribute) [s] "/}" ; standalone
       / "{" [s] "/" identifier *(s option) *(s attribute) [s] "}"  ; close

to:

markup = "{#" identifier *(s option) *(s attribute) [s] "}"  ; open
       / "{#" identifier *(s option) *(s attribute) [s] "/}" ; standalone
       / "{/" identifier *(s option) *(s attribute) [s] "}"  ; close

[mihnita]

Note: this is not the same as the expressions.
So there is no need to be consistent, because the similarity is only superficial.
They might look the same, but the meaning is not the same.

There detecting { tells us we are in an expression, no need for extra lookahead.
And the "decorations" { |foo| } or { $foo } or { :foo } DO belong to "foo".
It is "foo" that is a literal / operand / function, not the expression itself.

[aphillips]

I think this is a non-starter? Our syntax is very consistent about optional whitespace: we are very lax when it's optional, especially inside expressions. The {/} are never part of any construct.

I think the thing that might be confusing here is that when you write the syntax using identifier directly in the markup production (which I think is a natural thing to do), so you don't have the insulation that function or annotation do, in which the sigil is clearly attached to the identifier:

markup = "{" markup-identifier *(s option) *(s attribute) [s] "}" <- this doesn't work because of standalone
markup-identifier = ( "#" / "/" ) identifier

All of our other identifiers and sigil-introduced tokens are of the sigil-identifier flavor. For consistency, this should be to. There is some lookahead to find type, but it's consuming whitespace (so can be optimized).

[mihnita]

Our syntax is very consistent about optional whitespace: we are very lax when it's optional

Agree, but I don't think this is optional, and the sigils should not be attached to the identifier.
They mark the start / end of the markup.
In the same class as {{ and }}, where we don't allow spaces between the curly braces.
And with the end of standalone, where we don't allow space between / and }.

Right above your comment I explain why I think this is not at all the same.

the thing that might be confusing here is that when you write the syntax using identifier directly in the markup production

I am not judging this based on the grammar, I am judging it as a user seeing the syntax.

Start, {#foo}some text{/foo} and more.

What I see is {# ..... }, delimiters.

These are spaces that I agree are optional:

Start, {#   foo    }some text{/    foo   } and more.

{# marks the beginning of an open / standalone marker,
{/ the beginning of a close marker,
and /} marks the end of a standalone marker.
They are delimiters, same as {{...}}, or |...|.
One "squints" at {/....} and says: ah, this is a closing marker. The / is not a sigil of the name part.

There is not markup (that I know of) that allows these kind of spaces.
If there is one, I would like to see it.

[eemeli]

I think it's way too late to introduce this syntax change for consideration, and I do not think we should consider this for LDML 45.

As an implementer, I'd like to note that I had no difficulty dealing with the current syntax. In a pattern, from the { you can see that you have some expression or markup starting, and then you need to look past any subsequent whitespace to determine what that is.

[mihnita]

As an implementer, I'd like to note that I had no difficulty dealing with the current syntax

This is not about implementing. That is point 1 of 4, and in fact the weakest one.
Should not have been the first one (or included at all, probably)

It is that it does not make sense semantically, as a user, and no other system does that.