Initial version for sync_pot.py

Get the latest translation templates from the Kano computer and upload
them into Zanata.

README.md

@@ -1,2 +1,83 @@
 # kano-i18n-sync
-Bi-directionnal synchronization for Kano translation files
+
+Scripts to allow bi-directional synchronization between [Kano
+Linux](https://kano.me/) and the [Zanata](http://zanata.org/) translation
+service. Upload strings to the translation server and pull latest translated
+files from it into your Kano distribution.
+
+We've created Zanata translation projects for the following apps:
+
+* [kano-app-launcher](https://translate.zanata.org/project/view/kano-app-launcher/)
+* [kano-apps](https://translate.zanata.org/project/view/kano-apps/)
+* [kano-dashboard](https://translate.zanata.org/project/view/kano-dashboard/)
+* [kano-feedback](https://translate.zanata.org/project/view/kano-feedback/)
+* [kano-greeter](https://translate.zanata.org/project/view/kano-greeter/)
+* [kano-init](https://translate.zanata.org/project/view/kano-init/)
+* [kano-overworld](https://translate.zanata.org/project/view/kano-overworld/)
+* [kano-peripherals](https://translate.zanata.org/project/view/kano-peripherals/)
+* [kano-profile](https://translate.zanata.org/project/view/kano-profile/)
+* [kano-settings](https://translate.zanata.org/project/view/kano-settings/)
+* [kano-toolset](https://translate.zanata.org/project/view/kano-toolset/)
+* [kano-updated](https://translate.zanata.org/project/view/kano-updated/)
+* [kano-vnc](https://translate.zanata.org/project/view/kano-vnc/)
+* [kano-widgets](https://translate.zanata.org/project/view/kano-widgets/)
+* [make-snakes](https://translate.zanata.org/project/view/make-snakes/)
+* [terminal-quest](https://translate.zanata.org/project/view/terminal-quest/)
+
+
+## Setup
+
+The scripts depend on
+[zanata-python-client](https://github.com/zanata/zanata-python-client) to
+communicate with the Zanata server. To install this executable on Fedora, run:
+
+```
+sudo dnf install python3-zanata-client
+```
+
+[Create yourself an account on
+Zanata](https://translate.zanata.org/account/register) if you haven't done so,
+[generate your API
+key](https://translate.zanata.org/dashboard/settings/client), and copy the
+content of `zanata.ini` that you need to place in a new file named
+`$HOME/.config/zanata.ini`
+
+Now you need to prepare your Kano computer so that the scripts can connect to
+it. To turn on the SSH server on the Kano box, go to the `Advanced` setting and
+enable SSH. From the computer that is going to run the scripts, add a `kano`
+host to your `$HOME/.ssh/config` file, for instance:
+
+```
+Host kano
+  Hostname 192.168.1.15
+  User your_username
+```
+
+Finally, copy your keys to the Kano computer:
+```
+ssh-copy-id kano
+```
+
+Verify you're able to connect to your Kano computer with:
+```
+ssh kano
+```
+
+## Usage
+
+### Upload updated translation templates upon new Kano release
+
+The `sync_pot.py` script connects to the Kano computer to retrieve the latest
+translation templates then takes care of creating the project and new versions
+on Zanata, before uploading the template files to it.
+
+It is intended for the admins for the translation projects on Zanata and should
+be run each time Kano release a new version of their distribution.
+
+The script does not take options, simply invoke it with:
+
+```
+./sync_pot.py
+```
+
+### Pull latest translations from Zanata and update Kano

kano-projects.yaml

@@ -0,0 +1,91 @@
+---
+# - name:
+#   zanata_project_id:
+#   zanata_project_type:
+#   kano_i18n_package_name:
+#   pot_dir:
+- name: kano-app-launcher
+  zanata_project_id: kano-app-launcher
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-app-launcher-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-app-launcher/
+- name: kano-apps
+  zanata_project_id: kano-apps
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-apps-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-apps/
+- name: kano-dashboard
+  zanata_project_id: kano-dashboard
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-dashboard-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-dashboard/
+- name: kano-draw
+  zanata_project_id: kano-draw
+  zanata_project_type: podir
+  kano_i18n_package_name: kano-draw-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-draw/
+- name: kano-feedback
+  zanata_project_id: kano-feedback
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-feedback-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-feedback/
+- name: kano-greeter
+  zanata_project_id: kano-greeter
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-greeter-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-greeter/
+- name: kano-init
+  zanata_project_id: kano-init
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-init-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-init/
+- name: kano-overworld
+  zanata_project_id: kano-overworld
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-overworld-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-overworld/
+- name: kano-peripherals
+  zanata_project_id: kano-peripherals
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-peripherals-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-peripherals/
+- name: kano-profile
+  zanata_project_id: kano-profile
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-profile-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-profile/
+- name: kano-settings
+  zanata_project_id: kano-settings
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-settings-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-settings/
+- name: kano-toolset
+  zanata_project_id: kano-toolset
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-toolset-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-toolset/
+- name: kano-updater
+  zanata_project_id: kano-updater
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-updater-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-updater/
+- name: kano-vnc
+  zanata_project_id: kano-vnc
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-vnc-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-vnc/
+- name: kano-widgets
+  zanata_project_id: kano-widgets
+  zanata_project_type: gettext
+  kano_i18n_package_name: kano-widgets-i18n-orig
+  pot_dir: /mnt/translations/translations/kano-widgets/
+- name: make-snake
+  zanata_project_id: make-snake
+  zanata_project_type: gettext
+  kano_i18n_package_name: make-snake-i18n-orig
+  pot_dir: /mnt/translations/translations/make-snake/
+- name: terminal-quest
+  zanata_project_id: terminal-quest
+  zanata_project_type: gettext
+  kano_i18n_package_name: linux-story-i18n-orig
+  pot_dir: /mnt/translations/translations/linux-story/

sync_pot.py

@@ -0,0 +1,142 @@
+#!/usr/bin/env python
+# Upload latest pot files to zanata
+
+import os
+import paramiko
+import subprocess
+import tempfile
+import yaml
+
+
+def run_command(command):
+    proc = subprocess.Popen(command, stdout=subprocess.PIPE,
+                            stderr=subprocess.PIPE)
+    stdout, stderr = proc.communicate()
+    return stdout.decode(), stderr.decode(), proc.returncode
+
+
+def update_i18n_packages():
+    print("Updating i18n packages to latest version on kano")
+    # FIXME(mandre) Make sure the user has passwordless sudo
+    ssh.exec_command("sudo apt-get update")
+    ssh.exec_command("sudo apt-get install *-i18n-orig")
+
+
+def fetch_package_version(project):
+    print("Fetching package version for %s" % project['name'])
+    command = "dpkg-query --showformat='${Version}' --show %s" % \
+        project['kano_i18n_package_name']
+    stdin, stdout, stderr = ssh.exec_command(command)
+    version = stdout.read().decode()
+    if version:
+        project['version'] = version
+    else:
+        print("Could not fetch version for %s" % project['name'])
+        exit()
+
+
+def ensure_zanata_project(project):
+    print("Check if project %s exists on zanata" % project['name'])
+    command = ["zanata", "project", "info", "--project-id",
+               project['zanata_project_id']]
+    print(" ".join(command))
+    _, _, exit_status = run_command(command)
+    if exit_status != 0:
+        print("Project %s does not exist" % project['name'])
+        create_zanata_project(project)
+
+
+def create_zanata_project(project):
+    print("Creating project %s" % project['name'])
+    command = ["zanata", "project", "create", project['zanata_project_id'],
+               "--project-name", project['name'], "--project-desc",
+               "Unofficial translations for Kano's %s" % project['name'],
+               "--project-type", project['zanata_project_type']]
+    print(" ".join(command))
+    stdout, stderr, exit_status = run_command(command)
+    if exit_status != 0:
+        print("Failed to create project %s" % project['name'])
+        print(stdout)
+        print(stderr)
+        exit()
+
+
+def ensure_zanata_version(project):
+    print("Check if version %s exists for %s on zanata" % (project['version'],
+                                                           project['name']))
+    command = ["zanata", "version", "info", "--project-id",
+               project['zanata_project_id'], "--project-version",
+               project['version']]
+    print(" ".join(command))
+    _, _, exit_status = run_command(command)
+    if exit_status != 0:
+        print("Version %s does not exist for project %s" % (project['version'],
+                                                            project['name']))
+        create_zanata_version(project)
+
+
+def create_zanata_version(project):
+    print("Creating version %s for project %s" % (project['version'],
+                                                  project['name']))
+    command = ["zanata", "version", "create", project['version'],
+               "--project-id", project['zanata_project_id']]
+    print(" ".join(command))
+    stdout, stderr, exit_status = run_command(command)
+    if exit_status != 0:
+        print("Failed to create version %s for %s" % (project['version'],
+                                                      project['name']))
+        print(stdout)
+        print(stderr)
+        exit()
+
+
+def copy_pot_files(remote, local):
+    print("Copying pot files from %s on kano to %s" % (remote, local))
+    scp.chdir(remote)
+    for filename in scp.listdir():
+        print("Copying %s" % filename)
+        scp.get(os.path.join(remote, filename), os.path.join(local, filename))
+
+
+def upload_pot_files(project, potdir):
+    print("Uploading pot files for %s" % project['name'])
+    command = ["zanata", "push", "-f", "--srcdir", potdir,
+               "--project-id", project['zanata_project_id'],
+               "--project-version", project['version'],
+               "--project-type", project['zanata_project_type'],
+               ]
+    print(" ".join(command))
+    stdout, stderr, exit_status = run_command(command)
+    if exit_status != 0:
+        print("Failed to upload pot files to %s" % project['name'])
+        print(stdout)
+        print(stderr)
+
+
+# This assumes there is a kano host in your ~/.ssh/config with public key
+# authentication setup
+ssh = paramiko.SSHClient()
+ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+ssh.connect('kano')
+
+scp = ssh.open_sftp()
+
+update_i18n_packages()
+
+with open("kano-projects.yaml", 'r') as projects_file:
+    try:
+        projects = yaml.safe_load(projects_file)
+    except yaml.YAMLError as e:
+        print(e)
+
+for project in projects:
+    fetch_package_version(project)
+    ensure_zanata_project(project)
+    ensure_zanata_version(project)
+
+    with tempfile.TemporaryDirectory() as tmpdirname:
+        copy_pot_files(project['pot_dir'], tmpdirname)
+        upload_pot_files(project, tmpdirname)
+
+scp.close()
+ssh.close()

zanata.xml

@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<config xmlns="http://zanata.org/namespace/config/">
+  <url>https://translate.zanata.org/</url>
+</config>