1.9 dev (#411)

* refactor priorities * performance report doodling * shutup pycharm * more doodling * doodle tricol * tweak report, fix bug in insert * reorder colors * another metric * sorting engines, trying to catch exc * restructure engine err * restructure engine err * test stuff * test report * test report * test and readme * test and readme * layout * engine recov * pixelfix and exes * readme * keyboard shortcut override * oops * mistakes only deeper * black and readme * readme * fix #421, scale repor * Add shortcut variables (#420) * Add shortcut variables * Address comments and missed shortcuts * fix shortcuts * rename, design report, fix report * make translators work more * relative mid/opening/endgame bounds * win size dbg * more errros * entry * jeff mode * fix * readme * edit THEMES.md * edit katrain/core/ai.py, edit katrain/gui/kivyutils.py * jp (#423) * edit THEMES.md * edit katrain/__main__.py, edit katrain/gui/kivyutils.py and 1 other changes * Update katrain.po (#425) 1.9 first French .po * edit katrain/i18n/locales/fr/LC_MESSAGES/katrain.mo * edit katrain/core/ai.py, edit katrain/gui/popups.py * bluedabadee * fix sgf and black * edit katrain/i18n/locales/fr/LC_MESSAGES/katrain.po * ko * Update zh-TW translation (#426) * cn * cn * Some corrections to Chinese translations (#427) * some corrections in cn and tw * better trans * edit CONTRIBUTIONS.md, edit katrain/i18n/locales/cn/LC_MESSAGES/katrain.mo and 1 other changes * de * tw * player what only * add green background to better value (#428) * edit katrain/gui/popups.py, edit katrain/gui/theme.py * Russian for 1.9 (#432) * edit katrain/i18n/locales/ru/LC_MESSAGES/katrain.po, edit katrain/gui.kv and 1 other changes * sound fix * sound fix * fmt * fix sound * edit THEMES.md, edit CONTRIBUTIONS.md and 1 other changes * sound fix * Do not use ffpyplayer on Windows (#434) * edit katrain/gui/sound.py * no pr * jeff theme * stopsound * sound and theme fix * font fix and more errs * French po corrections (#441) Former translations were misleading. It should be accurate now. * add mouse scroll to browser through node variations. (#442) * add mouse scroll to browser through node variations. * add mouse scroll to browser through node variations. * add mouse scroll to browser through node variations. * edit katrain/popups.kv, edit katrain/__main__.py and 2 other changes * windows binary, black * linux binary * edit INSTALL.md * edit .github/workflows/osxbuild.yaml * edit .github/workflows/osxbuild.yaml * edit .github/workflows/osxbuild.yaml Co-authored-by: Sander Land <sander.land@cognite.com> Co-authored-by: roux g. buciu <11182210+electricRGB@users.noreply.github.com> Co-authored-by: kaorahi <38910552+kaorahi@users.noreply.github.com> Co-authored-by: Dontbtme <66470440+Dontbtme@users.noreply.github.com> Co-authored-by: Liou, Guan-Cheng <Tony-Liou@users.noreply.github.com> Co-authored-by: Ching-yu Lin <60384727+chingyulin@users.noreply.github.com> Co-authored-by: xiaoyifang <105986+xiaoyifang@users.noreply.github.com> Co-authored-by: Alexander Kiselev <satoshi.yoda.register.264@gmail.com> Co-authored-by: Sander Land <Sander@bioeng219-lap.isd.kcl.ac.uk> Co-authored-by: Carton He <cartonhe@gmail.com> Co-authored-by: Sander Land <sander.land@aroni.fritz.box> Co-authored-by: Sander Land <sander.land@futurefacts.nl>
sanderland · Jun 28, 2021 · bf70f2c · bf70f2c
1 parent 5f96697
commit bf70f2c
Show file tree

Hide file tree

Showing 50 changed files with 1,657 additions and 284 deletions.
diff --git a/.github/workflows/osxbuild.yaml b/.github/workflows/osxbuild.yaml
@@ -52,7 +52,7 @@ jobs:
         pushd KaTrain.app/Contents/Resources/
         ln -s ./venv/bin/KaTrain yourapp
         popd
-        ./fix-bundle-metadata.sh KaTrain.app -n KaTrain -v "1.8.0" -a "Sander Land" -o "org.katrain.KaTrain" -i "../../katrain/katrain/img/icon.ico"
+        ./fix-bundle-metadata.sh KaTrain.app -n KaTrain -v "1.9.0" -a "Sander Land" -o "org.katrain.KaTrain" -i "../../katrain/katrain/img/icon.ico"
         ./cleanup-app.sh KaTrain.app -g 1
         ./relocate.sh KaTrain.app
     - name: Create dmg
@@ -67,12 +67,4 @@ jobs:
       with:
         name: KaTrainOSX
         path: osx_app
-    - name: Upload to GitHub Release
-      uses: softprops/action-gh-release@78c309ef59fdb9557cd6574f2e0be552936ed728
-      if: startsWith(github.ref, 'refs/tags/')
-      env:
-        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      with:
-        files: KaTrainOSX/*
-        draft: true
-        fail_on_unmatched_files: true
+
diff --git a/.gitignore b/.gitignore
@@ -1,4 +1,5 @@
 dist_sgf
+sgftest
 
 # additions
 KataGoData

diff --git a/CONTRIBUTIONS.md b/CONTRIBUTIONS.md
@@ -52,7 +52,8 @@ Many thanks to these additional authors:
 * "blamarche" for adding the board coordinates toggle.
 * "pdeblanc" for adding the ancient chinese scoring option, fixing a bug in query termination
 * "LiamHz" for adding the 'back to main branch' keyboard shortcut.
-* "xiaoyifang" for adding the reset analysis option, feature to save options on the loading screen.
+* "xiaoyifang" for adding the reset analysis option, feature to save options on the loading screen, and scrolling through variations.
+* "electricRGB" for help with adding configurable keyboard shortcuts.
 
 ## Translators
 
@@ -65,7 +66,7 @@ Many thanks to the following contributors for translations.
 * Russian: Dmitry Ivankov and Alexander Kiselev
 * Simplified Chinese: Qing Mu with contributions from "Medwin" and Viktor Lin
 * Japanese: "kaorahi"
-* Traditional Chinese: "Tony-Liou"
+* Traditional Chinese: "Tony-Liou" with contributions from Ching-yu Lin
 
 ## Additional thanks to
 

diff --git a/ENGINE.md b/ENGINE.md
@@ -0,0 +1,74 @@
+# KataGo troubleshooting
+
+This page lists common ways in which the provided KataGo fails to work out of the box, and how to resolve these issues.
+If you find your problem is not in here, you can ask on the [Leela Zero & Friends Discord](http://discord.gg/AjTPFpN) (use the #gui channel),
+ providing detailed information about your error.  
+
+
+* [General](#General)
+    * [GPU vs CPU](#CPU)
+* [Windows specific help](#Windows)
+* [MacOS specific help](#Mac)
+* [Linux specific help](#Linux)
+
+
+
+## <a name="General"></a> General
+
+###  <a name="CPU"></a> GPU vs CPU
+
+The standard executables assume you have a compatible graphics card (GPU). 
+If you don't, KataGo will fail to start in ways that are difficult for KaTrain to pick up.  
+
+On Windows and Linux, you should be able to resolve this by:
+
+* Going to general and engine settings (F8)
+* Click 'download katago versions' and wait for downloads to finish.
+* Select a CPU based KataGo version (named 'Eigen' after the library it uses).
+
+Keep in mind that a CPU based engine can be significantly slower, and you may want to set your maximum number of
+visits to a lower number to compensate for this.
+
+###  <a name="Models"></a> KataGo model versions
+
+KataGo models have changed over time, and selecting an older executable with a newer model can lead to errors.
+Of the provided binaries, this is typically the case for the 1.6.1 'bigger boards' binary, which should
+ only be used with the standard 15/20/30/40 block models, and not the newer distributed training models.
+
+
+## <a name="Mac"></a><img src="https://upload.wikimedia.org/wikipedia/commons/8/8a/Apple_Logo.svg" alt="macOs" height="35"/> For macOS users
+
+### Running from source
+
+Make sure you `brew install katago` or set the engine path to your own KataGo binary, as there is no executable included.
+
+### New Macs with M1 architecture
+
+Make sure you `brew install katago` as the provided executable does not work on rosetta.
+
+### Getting more information about errors
+
+On macOS, the .app distributable will not show a console, so you will need install using `pip` to see the console window.
+
+##  <a name="Windows"></a><img src="https://upload.wikimedia.org/wikipedia/commons/5/5f/Windows_logo_-_2012.svg" alt="Windows" height="35"/> For Windows users
+
+### Getting more information about errors
+
+Run DebugKaTrain.exe, which is released in the .zip file distributable in releases. This will show a console window
+ which typically tells you more.
+
+
+## <a name="Linux"></a><img src="https://upload.wikimedia.org/wikipedia/commons/a/ab/Linux_Logo_in_Linux_Libertine_Font.svg" alt="Linux" height="35"/> For Linux users
+
+### libzip compatibility
+
+The most common KataGo issue relates to different libzip versions in the provided executables.
+Although the provided executables should work on the latest versions of Ubuntu, various other versions and distros differ in their libzip version. 
+
+* First, try `sudo apt-get install libzip-dev` to 
+* Next, try installing all packages mentioned [here](INSTALL.md#LinuxTrouble).
+* A final alternative is to go [here](https://github.com/lightvector/KataGo) and compile KataGo yourself.
+
+### Getting more information about errors
+
+Check the terminal output around startup time.
diff --git a/INSTALL.md b/INSTALL.md
@@ -6,6 +6,8 @@
     * [Troubleshooting and installation from sources](#WindowsSources)
 * [Quick install guide for Linux](#LinuxQuick)
     * [Troubleshooting and installation from sources](#LinuxSources)
+* [Configuring Multiple GPUS](#GPU)
+* [Troubleshooting KataGo](#KataGo)
 
 ## <img src="https://upload.wikimedia.org/wikipedia/commons/8/8a/Apple_Logo.svg" alt="macOs" height="35"/> Installation for macOS users
 
@@ -15,8 +17,9 @@ The easiest way to install is probably [brew](https://brew.sh/). Simply run `bre
 
 You can also find downloadable .app files for macOS [here](https://github.com/sanderland/katrain/releases). 
 Simply download, unzip the file, mount the .dmg and drag the .app file to your application folder, everything is included.
+The first time launching the application you may need to [control-click in finder to give permission for the 'unidentified' app to launch](https://support.apple.com/guide/mac-help/open-a-mac-app-from-an-unidentified-developer-mh40616/mac). This is simply a result of Apple charging $99/year to developers to be 'identified'.
 
-Users with the last generation M1 macs with different architecture should then `brew install katago` in addition to this. KaTrain will automatically detect this katago binary.
+Users with the last generation M1 macs with different architecture should then `brew install katago` in addition to this. KaTrain will automatically detect this KataGo binary.
 
 ### <a name="MacCommand"></a>Command line install guide
 
@@ -99,7 +102,7 @@ It assumes you have a working Python 3.6+ installation.
 A binary for KataGo is included, but if you have compiled your own, press F8 to open general settings and change the 
  KataGo executable path to the relevant KataGo v1.4+ binary.
 
-### Troubleshooting and advanced installation from sources
+### <a name="LinuxTrouble"></a>Troubleshooting and advanced installation from sources
 
 You can try to manually install dependencies to resolve some issues relating to missing dependencies,
  e.g. the binary 'wheel' is not provided, KataGo is not starting, or sounds are not working.
@@ -128,7 +131,9 @@ You can now start KaTrain by running `python3 -m katrain`
 
 In case KataGo does not start, an alternative is to go [here](https://github.com/lightvector/KataGo) and compile KataGo yourself.
 
-## Configuring the GPU(s) KataGo uses
+
+
+## <a name="GPU"></a> Configuring the GPU(s) KataGo uses
 
 In most cases KataGo detects your configuration correctly, automatically searching for OpenCL devices and select the highest scoring device. 
 However, if you have multiple GPUs or want to force a specific device you will need to edit the 'analysis_config.cfg' file in the KataGo folder.
@@ -148,8 +153,8 @@ As you can see it scores about twice as high as the Intel UHD chip and KataGo ha
 
 * Open the 'analysis_config.cfg' file in the `katrain/KataGo` folder in your python packages, or local sources.
   If you can't find it, turn on `debug_level=1` in general settings and look for the command that is used to start KataGo.
-* Search for `numNNServerThreadsPerModel` (~line 75), uncomment the line by deleting the # and set the value to 2. The line should read `numNNServerThreadsPerModel = 2`.
-* Search for `openclDeviceToUseThread` (~line 117), uncomment by deleting the # and set the values to the device ID numbers identified in the terminal.
+* Search for `numNNServerThreadsPerModel` (~line 108), uncomment the line by deleting the # and set the value to 2. The line should read `numNNServerThreadsPerModel = 2`.
+* Search for `openclDeviceToUseThread` (~line 164), uncomment by deleting the # and set the values to the device ID numbers identified in the terminal.
   From the example above, we would want to use devices 1 and 2, for the Intel and AMD GPUs, but not device 0 (the CPU). In our case, the lines should read:
 ```
 openclDeviceToUseThread0 = 1
@@ -165,3 +170,8 @@ openclDeviceToUseThread1 = 2
     Using OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) OpenCL 1.2
     Using OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) OpenCL 1.2
 ```
+
+
+## <a name="KataGo"></a> Troubleshooting and advanced KataGo settings
+
+See [here](ENGINE.md) for an overview of how to resolve various issues with KataGo.
diff --git a/THEMES.md b/THEMES.md
@@ -1,21 +1,25 @@
 # Themes
-Version 1.7 brings basic support for themes.
+
+Version 1.7 brings basic support for themes, and 1.9 extends it to include keyboard shortcuts and support for multiple theme files.
 
 ## Creating and editing themes
 
-* Look at the `Theme` class in `katrain/gui/theme.py`.
-* Make a `theme.json` file in your `<home dir>/.katrain` directory and specify any variables from the above class you want to override, e.g. 
+* Look at the `Theme` class in [`katrain/gui/theme.py`](https://github.com/sanderland/katrain/blob/master/katrain/gui/theme.py).
+* Make a `theme<yourthemename>.json` file in your `<home dir>/.katrain` directory and specify any variables from the above class you want to override, e.g. 
  ```json
  {
-  "BACKGROUND_COLOR": [1,0,0,1]
+  "BACKGROUND_COLOR": [1,0,0,1],
+  "KEY_STOP_ANALYSIS": "f10",
+  "MISTAKE_SOUNDS": ["jeff.wav","what.wav"]
 }
   ```
-* All resources (including icons which can not be renamed for now) will be looked up in `<home dir>/.katrain` first, so files with identical names there can be used to override sounds and images.
+* All resources (including icons, which can not be renamed for now) will be looked up in `<home dir>/.katrain` first, so files with identical names there can be used to override sounds and images.
+* If variables are specified in multiple theme files, the *latest* alphabetically takes precedence. That is, each later theme file overwrites the settings from any previous one.
 
 ## Installation
 
 * To install a theme, simply unzip the theme.zip to your .katrain folder. 
-  * On windows you can find it in C:\Users\you\.katrain and on linux in ~/.katrain.
+  * On Windows you can find it in C:\Users\you\\.katrain and on linux in ~/.katrain.
   * When in doubt, the general settings dialog will also show the location.
 * To uninstall a theme, remove theme.json and all relevant images from that folder.
 
@@ -37,3 +41,11 @@ Version 1.7 brings basic support for themes.
 
 ![Preview](https://raw.githubusercontent.com/sanderland/katrain/master/themes/eric-lizzie.png)
 
+
+### Jeff sounds
+
+* This theme makes Jeff comment `Ahhh?` and `What?!` when you make mistakes.
+* Sounds provided by Mikkgo.
+
+[Download](https://github.com/sanderland/katrain/blob/master/themes/jeff-sounds.zip)
+
diff --git a/i18n.py b/i18n.py
@@ -32,9 +32,11 @@
         strings_to_langs[entry.msgid][lang] = entry
         strings_to_keys[entry.msgid][lang] = set(re.findall("{.*?}", entry.msgstr))
         if entry.msgid in lang_to_strings[lang]:
-            print("duplicate", entry.msgid, "in", lang)
+            print("duplicate", entry.msgid, "in", lang, "--> deleting", entry.msgstr)
             errors = True
-        lang_to_strings[lang].add(entry.msgid)
+            po[lang].remove(entry)
+        else:
+            lang_to_strings[lang].add(entry.msgid)
     if todos[lang] and any("todo" in a for a in sys.argv):
         print(f"========== {lang} has {len(todos[lang])} TODO entries ========== ")
         for item in todos[lang]:
@@ -76,24 +78,25 @@
     for msgid, lang_entries in strings_to_langs.items():
         if lang in lang_entries and "TODO" in lang_entries[lang].comment:
             if any(e.msgstr == lang_entries[lang].msgstr for ll, e in lang_entries.items() if ll != lang):
-                todo_comment = (
-                    f"TODO - {lang_entries[DEFAULT_LANG].comment}" if lang_entries[DEFAULT_LANG].comment else "TODO"
-                )  # update todo
-                if (
-                    lang_entries[lang].msgstr != lang_entries[DEFAULT_LANG].msgstr
-                    or lang_entries[lang].comment.replace("\n", " ") != todo_comment
-                ):
-                    print(
-                        [
-                            lang_entries[lang].msgstr,
-                            lang_entries[DEFAULT_LANG].msgstr,
-                            lang_entries[lang].comment,
-                            todo_comment,
-                        ]
-                    )
-                    lang_entries[lang].msgstr = lang_entries[DEFAULT_LANG].msgstr  # update
-                    lang_entries[lang].comment = todo_comment
-                    print(f"{lang}/{msgid} todo entry updated")
+                if lang_entries.get(DEFAULT_LANG):
+                    todo_comment = (
+                        f"TODO - {lang_entries[DEFAULT_LANG].comment}" if lang_entries[DEFAULT_LANG].comment else "TODO"
+                    )  # update todo
+                    if (
+                        lang_entries[lang].msgstr != lang_entries[DEFAULT_LANG].msgstr
+                        or lang_entries[lang].comment.replace("\n", " ") != todo_comment
+                    ):
+                        print(
+                            [
+                                lang_entries[lang].msgstr,
+                                lang_entries[DEFAULT_LANG].msgstr,
+                                lang_entries[lang].comment,
+                                todo_comment,
+                            ]
+                        )
+                        lang_entries[lang].msgstr = lang_entries[DEFAULT_LANG].msgstr  # update
+                        lang_entries[lang].comment = todo_comment
+                        print(f"{lang}/{msgid} todo entry updated")
 
     po[lang].save(pofile[lang])
     mofile = pofile[lang].replace(".po", ".mo")

diff --git a/katrain/KataGo/katago b/katrain/KataGo/katago
diff --git a/katrain/KataGo/katago.exe b/katrain/KataGo/katago.exe