i18n: Proposal to use jekyll-simple-i18n plugin instead of jekyll-multiple-languages #1503

djoate · 2019-11-18T01:45:57Z

Related to issue https://github.com/privacytoolsIO/privacytools.io/issues/1106.
Tagging @jonaharagon since i18n was your project.

Preview of index.html translation: https://deploy-preview-1503--privacytools-io.netlify.com/es/
Original site: https://deploy-preview-1503--privacytools-io.netlify.com

Sample of partially translated pages:

https://deploy-preview-1503--privacytools-io.netlify.com/es/
https://deploy-preview-1503--privacytools-io.netlify.com/es/providers/ (which you can get to by pressing the green Aprenda Más button under Proveedores on the homepage)
https://deploy-preview-1503--privacytools-io.netlify.com/es/providers/cloud-storage/ (which you can get to by pressing the Servidor Cloud link in /es/providers/)

Every other page has not been translated (pages without translate: true in front matter are not generated)

This is not meant to be a full-fledged PR but rather a proof of concept for a better solution for localizing the site. If this is an acceptable solution, please make this into a branch in this repository so that we can start localizing using this plugin.

With the jekyll-simple-i18n plugin (MIT licensed), this makes translating the site easier to manage. You should visit the plugin's GitHub repository and read up on it, but here are the features of this plugin from their README:

No external dependencies. Plugins utilize existing Jekyll features.
Source strings and page titles can be placed directly in templates for seamless editing and readability.
A ready-to-translate YAML file that includes all of the canonical source strings is generated every time the site is built.
It's easy to add new languages. Just create a single file that contains the translated strings. Everything else happens automatically.
Custom front matter is added to translated pages that can be used within your Liquid templates.
Optional Transifex integration.
Built-in support for hreflang tags.

It's based on Transifex, but it can be used with a different service such as Weblate. I've made some modifications to the plugin (e.g. renaming transifex to weblate and handling null source text).

This PR includes an example of index.html and a part of card.html (The "Learn More" button text) being translated. The plugin did not seem to work with the github-pages gem, so github-pages was switched with jekyll gem (which is what the current i18n branch does anyway). You can go to https://deploy-preview-1503--privacytools-io.netlify.com/es/ (or build locally) to see the following:

https://deploy-preview-1503--privacytools-io.netlify.com gives the original English site.

Here is a snippet of the source code for index.html:

<h1 id="sponsors" class="anchor"><a href="#sponsors"><i class="fas fa-link anchor-icon"></i></a> {% t Sponsors%}</h1>

<div class="alert alert-success" role="alert">
  <strong>{% t New!%}</strong> {% t Showcase your brand as a sponsor of PrivacyTools here and support our mission of creating a world free of mass surveillance!%} <a href="/{% if page.language %}{{ page.language }}/{% endif %}sponsors/" class="alert-link">{% t Learn more...%}</a>
</div>

A snippet of resources.html:

<p><a href="/{% if page.language %}{{ page.language }}/{% endif %}classic/"><i class="fas fa-info-circle"></i> {% t Prefer the classic site? View a single-page layout.%}</a></p>

<div class="row">

  {% capture providers_title %}{% t Providers %}{% endcapture %}
  {% capture providers_page %}/{% if page.language %}{{ page.language }}/{% endif %}providers/{% endcapture %}
  {% capture providers_description %}{% t Discover privacy-centric online services, including email providers, VPN operators, DNS administrators, and more!%}{% endcapture %}

  {% include card.html color="success"
  title=providers_title
  icon="fas fa-server"
  iconcolor="dark"
  page=providers_page
  description=providers_description
  %}

Instead of using keys and two different files, you just wrap the original text around with {% t ... %} tags, and the plugin will automatically key that string (with its own ID) into weblate-source-file.yml. If you are trying to translate things inside of a card, you have to do the same thing as before with capturing text.

The source YAML is generated on build into the root folder of the repo. This source file can then be copied into _data/languages/ and then renamed into one of the languages in the language map to set up a translation. This seems much easier to maintain compared to cross referencing between two different files.

The plugin will also not create multiple keys for duplicates of the exact same string. For example, {% t Worth Mentioning %} will have one key associated with it, and there will only be one key to translate. All other pages that use {% t Worth Mentioning %} will share the same key (however, I've modified it so that, for instance, {% t Worth mentioning %} and {% t Worth Mentioning! %} would have distinct keys)

We would have to replace local links with something like this in order to get the right pages (and I believe external links can be wrapped in translate tags without a problem):

<a href="/{% if page.language %}{{ page.language }}/{% endif %}sponsors/" class="alert-link">{% t Learn more...%}</a>

A porition of the source file, weblate-source-file.yml, looks like this:

---
Prefer_the_classic_site?_View_a_singlepage_layout.: |
  Prefer the classic site? View a single-page layout.

Providers: |
  Providers

Discover_privacycentric_online_services_including_email_providers_VPN_operators_DNS_administrator: |
  Discover privacy-centric online services, including email providers, VPN operators, DNS administrators, and more!

Learn_More: |
  Learn More

It's a different format when compared to what is currently in the i18n branch now, i.e. it has the format

string_key_id: |
  This is a source string from the site

rather than

"string_key_id": "This is a source string from the site"

If this format doesn't work with Weblate, we can change the plugin so that it generates the latter format.

A sample translation into Spanish (using deepl.com) can be found in _data/languages/es.yml:

---
Prefer_the_classic_site?_View_a_singlepage_layout.: |
  ¿Prefieres el sitio clásico? Ver un diseño de una sola página.

Providers: |
  Proveedores

Discover_privacycentric_online_services_including_email_providers_VPN_operators_DNS_administrator: |
  Descubra servicios en línea centrados en la privacidad, incluyendo proveedores de correo electrónico, operadores de VPN, administradores de DNS y mucho más!

Learn_More: |
  Aprenda Más

Known issues

I don't know of a way to make the plugin translate the strings of permalinks. This means that pages such as https://privacytools.io/es/donate will have to stay as /es/donate for now. Update: See comment below since this is actually not an issue.
Because the plugin uses the actual string as the translation ID/key, there may be collisions (e.g. "Learn more..." and "Learn more!" will have the same ID). To help remedy this, I've modified the plugin so that the IDs preserve capitalization and can also contain periods, exclamation marks, and question marks. I've also modified the plugin so that the max length for an ID is 100 characters.
I've had to modify the Gemfile to not use github-pages gem, so github-pages was switched with jekyll gem (which is what the current i18n branch does anyway). The jekyll-sitemap plugin also had to be explicitly added in order for the site to compile.
Breadcrumbs: I'm not a Ruby programmer, so I'm not going to try to make a solution for the breadcrumbs.

To reiterate, this is a proof of concept for a better i18n solution. Feel free to add this as a branch if this seems like an acceptable solution.

Adds in support for https://github.com/signalapp/jekyll-simple-i18n. The plugin did not work with the github-pages gem, so github-pages was switched with jekyll (which is what the current i18n branch does anyway).

* Translate more of index.html, with translations sourced from deepl.com * Relax the plugin to allow capitalization, periods, exclamation marks, and question marks in the Weblate ID for YAML src. This helps differentiate between similar but different strings.

netlify · 2019-11-18T01:46:55Z

Deploy preview for privacytools-io ready!

Built with commit 1472c27

https://deploy-preview-1503--privacytools-io.netlify.com

dawidpotocki · 2019-11-18T02:20:03Z

Instead of using keys and two different files, you just wrap the original text around with `{% translate ... %}` tags, and the plugin will automatically key that string (with its own ID) into `weblate-source-file.yml`. If you are trying to translate things inside of a card, you have to do the same thing as before with capturing text.

That's actually a thing I wanted to build :P.

- I don't know of a way to make the plugin translate the strings of permalinks.

That's good, it's more predictable and easier to implement (like for example if user would switch language while being on some subpage).

jonaharagon · 2019-11-22T20:07:36Z

Okay. This looks good to me. A saner solution, I think. We can't use Transifex. If this works with Weblate, which it should, we're fine.

We don't want to add weblate-source-file.yml to .gitignore, do we? I think the file would need to be in the repo for Weblate to read it.

djoate added 3 commits November 18, 2019 00:09

Add jekyll-simple-i18n plugin with example

c36002d

Adds in support for https://github.com/signalapp/jekyll-simple-i18n. The plugin did not work with the github-pages gem, so github-pages was switched with jekyll (which is what the current i18n branch does anyway).

Key 'Privacy Tools', and get rid of en.yml

8f216de

djoate added 6 commits November 18, 2019 23:09

Add support for translating page.description

cab6bd1

Translate a part of providers page

3abed60

Fix permalinks in page generator

2ace1bf

Partially translate providers/cloud-storage/

9817613

Use 't' instead of 'translate'

52da0f3

Code style cleanup

1472c27

jonaharagon mentioned this pull request Nov 22, 2019

NEW [WIP] Add localization support with jekyll-simple-i18n #1509

Draft

jonaharagon closed this Nov 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

i18n: Proposal to use jekyll-simple-i18n plugin instead of jekyll-multiple-languages #1503

i18n: Proposal to use jekyll-simple-i18n plugin instead of jekyll-multiple-languages #1503

djoate commented Nov 18, 2019 •

edited

Loading

netlify bot commented Nov 18, 2019 •

edited

Loading

dawidpotocki commented Nov 18, 2019 via email •

edited

Loading

jonaharagon commented Nov 22, 2019

i18n: Proposal to use jekyll-simple-i18n plugin instead of jekyll-multiple-languages #1503

i18n: Proposal to use jekyll-simple-i18n plugin instead of jekyll-multiple-languages #1503

Conversation

djoate commented Nov 18, 2019 • edited Loading

Known issues

netlify bot commented Nov 18, 2019 • edited Loading

dawidpotocki commented Nov 18, 2019 via email • edited Loading

jonaharagon commented Nov 22, 2019

djoate commented Nov 18, 2019 •

edited

Loading

netlify bot commented Nov 18, 2019 •

edited

Loading

dawidpotocki commented Nov 18, 2019 via email •

edited

Loading