Basic Fluent implementation

[Manishearth]

Moved over from #732 (reverted due to bugs)

Rebased and updated to work. Contains a single test string in use.

[Manishearth]

@renyuneyun the bugs were that the provider was still not static (and i don't think it was getting initialized properly)

Furthermore, the directory needs to be present for this to work.

One concern I have is that the code for replacing ::foo:: is quadratic, it's going to cause a copy for each text insertion it does. Is there a way we can hook into the templating system instead?

[Manishearth]

I think this would work better as a handlebars helper

[Manishearth]

Alright, I've got a plan in place for supporting /paths

[Manishearth]

Todo (mergeable without this):

• Implement fallback to English when the message isn't formattable
• Figure out what to do with markup

[Manishearth]

An interesting case is when we have markup interspersed.

The tagline is an example, currently I do tagline = A language empowering everyone { $linebreak } to build reliable and efficient software. and then invoke it as {{text tagline linebreak="<br class='dn db-ns'>"}} so that the appropriate linebreak gets inserted. I mostly do this because I wanted a way to test variable arguments in Fluent, but it's also one way of fixing it. The other way is to hardcode the `
in the FTL code, which isn't great.

A similar situation is the text:

Whether you prefer working with code from the command line, or using
        rich graphical editors, there’s a Rust integration available for your
        editor of choice. Or you can build your own using the
        <a href="https://github.com/rust-lang/rls">Rust Language Server</a>.

I'm not sure what I should do with the link there. I have three options:

• include it as is in the FTL file with a placeholder for RLS, Or you can build your own using the <a href="/rust-lang/rls"> { rust-language-server }</a>. Icky and forces translators to much with markup. Which may not be a big deal here, the material being translated is already technical so translators may have decent familiarity with markup.
• Have markup variables: Or you can build your own using the {$link-open} { rust-language-server } { $link-closed }. Icky, but gets the job done.
• Have a custom function: Or you can build your own using the { LINK ($url, text: {rust-language-server }) } (I'm not sure if you can resolve function parameters that way, if you can't we can always do it within the function). This is kinda nice but maybe too much work.

cc @zbraniecki

[polarathene]

Is there an active repo where translations are being done, or is that on hold until this feature is implemented?

Besides the example with an anchor tag, what about the install page which uses JS to decide what path to show depending on detected OS?:

      <p>
        In the Rust development environment, all tools are installed to the
        <span class="platform-specific not-win di">
          <code>~/.cargo/bin</code>
        </span>
        <span class="platform-specific win dn">
          <code>%USERPROFILE%\.cargo\bin</code>
        </span> directory,
        and this is where you will find the Rust toolchain, including
        <code>rustc</code>, <code>cargo</code>, and <code>rustup</code>.
      </p>

I'm not familiar with Fluent myself, but afaik it allows you to indicate where variables should be, in this case they don't really need a translation so the whole two span elements could be inserted as {$install-path} or something?

Technically for a translation, the translator should only need to be concerned about the identifier? None of your suggested approaches do that, is it something that can be done?

Or you can build your own using the { rust-language-server }.

No need to have the open/close tags in the translation file? Seems to be pretty much what you said you do at the start before saying you only had 3 options.

[skade]

@polarathene The translation will be in this repository, hooked into a https://github.com/mozilla/pontoon instance once we got it set up.

In JS, we can use the JS implementation of fluent.

[Manishearth]

@polarthene the reason I can't do that is that I don't know how to nest Fluent calls in Handlebars. Maybe I should build a syntax for it.

[Manishearth]

I guess writing a block helper would work! But that only gives me a single insertion point. Maybe with would help?

[Manishearth]

Ah, so I think I can basically write something like

{{#text rust-integration}}
{{#param rls}} <a href="https://github.com/rust-lang/rls"> {{text rust-language-server}} {{/param}}
{{/text}}

may be tricky to pass down parameters (e.g. casing) though

[renyuneyun]

Ah, actually I was thinking of using this strategy (handlebars helper, though first time heard this name) initially, but couldn't find any document, and found the handlebars implementation seems to be a "internal" thing to Rocket.
Your code demonstrates it is possible. But I still wonder: does handlebars cache the template rendering results? (If not, that's the same as using a fairing in terms of time cost, isn't it?)
But yes, it makes it less easier to misinterprete things / placeholders / commands. And it seems handlebars contexts can be used (!) in this approach, which I agree is very useful.

I'm not sure what the "bugs" your mean? I thought (for this case) whenever the compiler doesn't yield a lifetime problem, the life is correct (i.e. the declared lifetime is the same as the actual behaviour)?
But yes, the directory was assumed to be present... That is a problem.

[renyuneyun]

Does anyone have any clue why this github (online) diff shows everything of i18n.rs, rather than based on the previously implementation (which I saw merged)?
Quite annoying to see old code is shown as new (in green) 🤦‍♂️

[Manishearth]

Does anyone have any clue why this github (online) diff shows everything of i18n.rs, rather than based on the previously implementation (which I saw merged)?

The code was broken and was reverted yesteday. I updated your code to the new edition, rebased, and squashed.

[Manishearth]

Your code demonstrates it is possible. But I still wonder: does handlebars cache the template rendering results? (If not, that's the same as using a fairing in terms of time cost, isn't it?)

No, the problem with the fairing was that it operated on substring replacement, which means that every time you replace an entry you're copying a significant part of the string. Handlebars works on a streaming basis (it doesn't construct the later parts of the string until it is done constructing the earlier parts of the string), it's designed for replacement and is much faster.

Either way, the fairing wouldn't let us use more involved syntaxes: We need to be able to support {{text foo bar="baz"}} and arbitrarily nestable templates (#792 (comment)) as well, and we'd have to invent parsing and syntax for that.

So fairings are a good solution for a first pass (and i was fine with merging it that way), but for the final system we need to use handlebars. Since I'm building on your work here anyway, I thought I'd fix it. Your commit with the fairings support is still here (attributed to you) at the head of this PR.

I'm not sure what the "bugs" your mean? I thought (for this case) whenever the compiler doesn't yield a lifetime problem, the life is correct (i.e. the declared lifetime is the same as the actual behaviour)?

So the directory and stuff wasn't pushed up, and I had weird edition errors because the branch was probably written before the edition upgrade (I didn't investigate, I just fixed it, this might have been a rebase problem).

But also, like I'd done in my example branch you'd need to make both the list of bundles and the list of files in statics; the lifetime breaks down for more complex use cases. It wasn't a bug where it would fail to compile yet, it was one where further extensions would be broken.

[Manishearth]

Interpolation in the latest commit, using {{textparam}}

[Manishearth]

The "correct" solution is something like DOM Overlays , but I don't have time to implement that system in Rust. If someone wants to make a proper fluent handlebars crate, this might be what they should actually do.

[Manishearth]

Added fallbacks, and baseurl.

[skade]

I found nothing except the leftover comments. The page works great locally.

[skade]

Here's a couple of leftover comments.