Release on Flathub #1602
Comments
|
Sure. Contributions like this are very welcome. However, we would need to integrate this into our current CI process to have those flatpacks generated automatically. Note that creating installers for Artisan is not trivial. For example, Artisan uses Qt's QWebEngine to render the HTML reports internally and generate PDF exports. Correctly bundling QWebEngine is non-trivial. We wish the builds would be smaller (the universal2 builds for macOS are larger than 1.2GB)! However, there are limits. Artisan uses almost anything from Qt. Not much to keep off there (the macOS build script does a lot removal and still the packages are large). Also the translations/locales add considerable to the size of the packages. Not sure if it makes sense to split them off. How would the correct locale be installed per user? Besides that some users use several locales at the same time (I have my locale in english, but use software often set to German). Regarding the app id: the github home of Artisan has a dash in the name as well. Maybe you can/have-to escape those dashes? |
|
Thank you for your response, and happy to hear you're open to this. Note that the building happens on Flathub infrastructure, ideally based on source code for all components. CI could be useful to ensure it builds correctly, but is not strictly necessary. There is a KDE+Qt + PyQt basis, I will have a look how QtWebEngine integrates into this. Translations/locales can either be bundled in the Flatpak, or be present as a kind of language-add-on packages (there is infrastructure for this). There can be multiple language packages installed, so that should be fine.
|
|
I think the Flatpak packaging is almost finished. Appstream fileInclusion of an appstream metadata file in the artisan repository (see Flathub on Required Metadata). This file is used to show the app in appstores, across different distributions. It contains description, links to screenshots, relevant URLs, and information about releases. 1a. Inclusion in Artisan repository (*)If the appstream file could be included in the Artisan repository, that would be great (and somewhat of a requirement for Flathub). Note that this file needs to be updated with each release (or the new release cannot be released on Flathub). If this is tedious, I can help to think about how to automate updating this, e.g. from Github releases and the release history. 1b. Also for Debian & other Linux distributionsNote that the appstream file may also be useful to include in other Linux packages. For a Debian package, see Debian AppStream Guidelines. Note that the Perhaps this is something to consider later. It is mostly relevant when the deb package is included in a repository that software stores can recognize. I would just like you to know about this, to decide where to put the appstream file. 1c. Question on OARS (*)OARS is about age rating metadata, and there is one question where I did not have the answer for: is any user tracking happening (e.g. a user counter, error reporting, or something similar)? If no, then the Note that the The Appstream file<?xml version="1.0" encoding="UTF-8"?>
<component type="desktop-application">
<id>org.artisan_scope.artisan</id>
<name>Artisan</name>
<summary>Visual scope for coffee roasters</summary>
<metadata_license>CC0-1.0</metadata_license>
<project_license>GPL-3.0-only</project_license>
<supports>
<control>pointing</control>
<control>keyboard</control>
<control>touch</control>
</supports>
<description>
<p>Artisan is an open-source software that helps coffee roasters record, analyze, and control roast profiles. This software can automate the creation of roasting metrics to help make decisions that influence the final coffee flavor.</p>
<p>Notable features</p>
<ul>
<li>we believe in Open-Source, check out our code and contribute</li>
<li>seamless integration into the artisan.plus inventory management service</li>
<li>extensive machine and device support including control</li>
<li>unlimited number of curves</li>
<li>rate-of-rise (RoR), area-under-the-curve (AUC), development-time-ratio (DTR) calculations and projection lines</li>
<li>statistics, roast profile evaluation, Profile Analyzerg, Roast Comparatorg, Profile Transposerg and Roast Simulatorg</li>
<li>roast-, production- and ranking reportsg</li>
<li>automated reproduction of roasts via alarm programsg, replay of eventsg or PID controlg</li>
<li>batch counterg</li>
<li>profile designerg, cupping editor, spider- and wheel graph</li>
<li>user defined buttons and slidersg with programmable actions</li>
<li>many import and export formats like Aillio Roastime, Cropster XLS, Giesen CSV, IKAWA CSV, Probat Pilot, RoastLogger, RoastLog, RoastPath,..</li>
</ul>
<p>The Artisan project runs on donations from individuals and companies recognising the value of Artisan for their work or leisure. Consider supporting this project with your donation, or even better, purchase an artisan.plus membership!</p>
</description>
<developer id="org.artisan-scope">
<name>Artisan developers</name>
</developer>
<url type="bugtracker">https://github.com/artisan-roaster-scope/artisan/issues</url>
<url type="homepage">https://artisan-scope.org/</url>
<url type="donation">https://artisan-scope.org/donate/</url>
<url type="contact">https://artisan-roasterscope.blogspot.com/p/contact-me.html</url>
<url type="faq">https://artisan-scope.org/help/#faq</url>
<url type="vcs-browser">https://github.com/artisan-roaster-scope/artisan</url>
<launchable type="desktop-id">org.artisan_scope.artisan.desktop</launchable>
<screenshots>
<screenshot type="default">
<image>https://github.com/artisan-roaster-scope/artisan/blob/2c94e7d03f51221771a6f5e10c3a55fd1a26e744/wiki/screenshots/artisan-cover.png?raw=true</image>
</screenshot>
<screenshot>
<image>https://github.com/artisan-roaster-scope/artisan/blob/2c94e7d03f51221771a6f5e10c3a55fd1a26e744/wiki/screenshots/artisan-cover2.png?raw=true</image>
</screenshot>
</screenshots>
<content_rating type="oars-1.1">
<content_attribute id="social-info">mild</content_attribute> <!-- TODO check if there is a user-counter or so (e.g. with update check) -->
<content_attribute id="money-purchasing">mild</content_attribute>
</content_rating>
<branding>
<color type="primary" scheme_preference="light">#d8ebf1</color>
<color type="primary" scheme_preference="dark">#0c6aa6</color>
</branding>
<releases>
<release version="2.10.4" date="2024-03-22">
<url type="details">https://github.com/artisan-roaster-scope/artisan/releases/tag/v2.10.4</url>
<description>
<p>This version is a bug fix release.</p>
<ul>
<li>adds metadata to PDF, SVG and PNG exports</li>
<li>fixes regression which broke the PID dialog for MODBUS, S7, TC4 and Kaleido (Issue #1480 and Issue #1515)</li>
<li>fixes regression which broke devices Omega HH309 34 and the just introduced Digi-Sense 20250-07 and Extech 42570 (PR #1481)</li>
<li>fixes regression introduced in v2.8.4 which broke persisting volume changes made in Roast Properties dialog (Discussion #1511)</li>
<li>fixes regression which broke persistence of custom column width of alarm and extra device and table</li>
<li>fixes regression which broke the formating of extra curve data in background event table</li>
<li>fixes regression which broke the automatic opening of completed profiles in ArtisanViewer (Issue #1516)</li>
<li>improves formating of profile and background data tables, custom event button table, alarm table and extra device table</li>
<li>fixes Linux builds that failed to start Artisan v2.10.2 on some configurations</li>
<li>fixes phases bar rendering in case CHARGE is not set</li>
</ul>
<p>IMPORTANT NOTE</p>
<p>The following issues have found in this release and are being worked on:</p>
<ul>
<li>this version does not play well with Probat Series III and Sample Roaster on all supported platforms. Please keep using the v2.8.4 release until v2.10.6 has been released</li>
</ul>
</description>
</release>
</releases>
</component>2. Cleaning up the installation script (*)The Flatpak currently contains some lines to install the package. It does not use pyinstaller, also because some dependencies are already installed in the runtime and base application (it wouldn't make sense to bundle those). This is what I currently do. I would love to have most of these steps as an installer script in the Artisan source tree: something that installs the app in a directory from which it is runnable, without bundling or dependencies. I could try my hand at that, but would need to refactor the build scripts. Please let me know if you think that's a good idea. Sidenote: one thing I encountered is a Python recursion error on import of 3. IconIs the icon available as SVG? That is preferred nowadays. |
- adds app icon as svg - renames debian desktop file - adds appstream metadata
|
I added tyour appstream metadata file under debian/usr/share/metainfo and the app icon as artisan.svg. I also renamed the desktop file according to your instructions. Regarding OARS, the information on the page you link is a little loose. The Artisan app is counting the number of roasts a user does to show a dialog asking for a donation on a regular basis. However, this information is not transmitted anywhere and stays local. Then there is a method to send detailed debug log information via email back to us. But that mechanism is somewhat hidden and has to be triggered by the user. The app is also not sending that email automatically but just generates an email with the log files attached on request by the user. This email then still has to be manually send by the user to somewhere. I assume this does not count as user tracking which we do not plan to introduce at any point. So I removed that social-info line assuming my interpretation and evaluation of those actions is correct. Regarding point 3 I would like to suggest that you make a start with this and send us a PR. This is not totally trivial as we build for different platforms on CI and we would love to keep this going;). This snap7 thing is puzzling. There is not much specific about that lib or our use (but for the fact that they just made a major version step and changed some API and we still have to use the old version on some platforms for reason and thus need to support both APIs). That lib has a platform specific binary part as it is only a Python thin wrapper around the multi-platform native Siemens S7 protocol implementation known as snap7. You could get around this loop by importing it in the artisan.py main script. This is not ideal as we tried (only with partial success) to keep all Artisan specifics in the main.py package. Did you try to move this import higher up in main.py instead of adding it to the artisan.py script? What was the loop exactly about? By copying it to artisan.py you made it escape your compileall action as the artisan.py script is not compiled, right. Hm. |
|
Could those build scripts for flatpacks/flathub also be independent of what we have right now? One more thing: we are about to release Artisan v3 (new major version) next week or latest the week after. This could be the first release on Flathub. We could go for a "manual" flatpack build for this one and it could also be submitted a little after the regular v3 release. Let us know what is possible! |
|
Thanks for your replies, and for adding the appstream metadata file, and icon (the icon still contains embedded PNGs for something, I'll have a look if I can make it fully SVG-based). Regarding OARS, I would judge the optional debugging info sent manually by mail not as user tracking, good you removed the line. I'll have a look at the build scripts. In the meantime, I feel confident enough to submit this in its current form to Flathub, then it can hopefully be simplified later. The snap7 thing appeared to be caused by the importlib monkey patch for MacOS, that recently seems to have been disabled (but not in the latest release 2.10.4, which I'm working with). The build scripts for the Flatpak can indeed be independent of the rest. But I do see quite some duplication across the build scripts, and I would think it is hard to maintain a growing number of them, especially e.g. what is copied to Great to hear about v3, I'll be happy to get that submitted to Flathub. Agree that we can keep the Flathub building aproach as it is now, and gradually improve (by moving some parts to build scripts part of artisan), if that turns out to make sense. |
|
I will update the SVG with a clean one in my next commit. Sorry. Right, this monkey patch was necessary as packaging on macOS failed. At least this one is resolved now. We constantly have to work around those things. Currently the main Python packaging lib introduced several complex incompatibilities one version just with py2app another one with pip/pyinstaller/numpy/scipy. Hard to find a combination of versions per platform which allows to bundle builds together:( |
|
Thank you! No problem :) Ah, that's why there are all these version number specifics in |
|
I'm trying out v2.10.5-beta now, and notice that |
|
I have it working with pydantic, if that is the course of action. The total Flatpak size is 564 MB without and |
|
Pydantic is needed and we plan to make use of pydantic within Artisan in more places in the future. Thanks for your work thus I don't understand why those python packages cannot be just downloaded from pypip via pip as the Artisan build scripts do. In that case there seems to be no need for use rust to compile from source, right? |
|
Thanks for your reply! Good to hear more use of pydantic is planned (for the code), and to know what to aim for. My reasoning was, that often, checks done on development are omitted in production builds, I would have expected pydantic to allow this, some kind of 'dummy' mode where these checks are omitted (unless we want more checks in production builds). But guess that would be more something for upstream. The Flatpak packaging scripts prefer building things from source, I think both because having the source is generally more flexible / debuggable / reproducible, but also just because that's what the packaging scripts do right now (it is not enforced, however). In any case, the issue is tackled. |
|
We use (and plan to use more) pydantic to validate input on read into the app during runtime. Those check we want during runtime not to have to handle such "broken" data within the app. For development checks we use the type checkers mypy and pyright as well as the linters pylint and ruff. Those type annotations used by the type checkers are indeed ignore by the Python runtime. I understand this idea of working from source. However, many packages Artisan is build upon are non-trivial to build from source correctly, like PyQt, numpy, scipy and some others. The chance is high that, without that expert knowledge of those package owners, your builds are somewhat broken, at least on some platforms. Therefore other Python packages and apps like Artisan using those packages load the corresponding binary builds from pypip. Those packages can be retrieved before the "build" (assemble) on Flatpack to avoid any further network communication during the build. This is how the CI Artisan builds work too. |
|
Thanks for the background, good to know! The idea that building from source might lead to a broken package is a something I need to think about. PyQt has a Flatpak BaseApp to work with, so should be fine. Numpy, scipy and matplotlib, as well as dependencies like cryptography are widely built from source in Flatpaks. There are some packages that need native libraries, which I have added where known (when not shipped in the Python package). I'll take a critical look at them again. |
|
Isn't it very inefficiency (energy wise) if each install compiles every component from source, besides that the install takes way longer? |
|
Ah, that would be cumbersome and wasteful, yes. What is installed by the user is in binary form (using OSTree) (served by Flathub), the build process - running on Flathub systems for each app release - builds from sources. |
|
Question on secret storage (for the plus integration, afaik): keyring comes with both If the backends need to be listed for pyinstaller, why not adding them to the hiddenimports in the spec file for each platform? |
|
The automatic default backend selection did never work in builds (on some platforms not even on running from source). This seems to have changed. I just tested builds on macOS, Windows and Ubuntu and on all three platforms the backend selection seemed to worked without issues. Still a scary change as before this worked. I hope someone else can test this a bit. Note that the keyring is only used to manage the password for the access to artisan.plus. I did not test the selection of a different backend by using a configuration on Ubuntu to choose libsecret (whatever this is; I am not a Linux user) over the SecretStorage that seems to be used as default (if installed). |
|
Ouch, that sounds like a bad situation. Good to hear it's working now, and yes, a little scray ... I will test on my Linux setup before the next release. Even though documentation suggests that libsecret works with Flatpak for the sandboxed environment, I could not get it to work without adding permissions to access all secrets (it so happens that all applications I could find that work with passwords do this as well). And then the existing SecretStorage also works (the Python keyring packages actually prefers it). So no change needed here (it will probably at some point in the future, when sandboxing is improved). I'll try to keep an eye on this in the future. |
|
v3 released. What is the status on Flathub? |
|
Thanks for the headsup! I'm on it, will do some testing, and create the PR on Flathub. Since a lot happens to actually install Artisan, I expect some questions, but think that in a week or so it could be available (depending a bit on how much work still is needed, according to the Flathub reviewer). update I see the changes to the beta (which I've been testing) are not big, so I'll do minimal testing before submitting. |
|
Ah, one thing: I'm missing the release info in the metainfo file. This is how the Flatpak finds out the version, so it is pretty important to add the release to this file for each release (sorted by release date, newest on top). Could you still add that? (Strictly speaking, this may be a new release, but I can also get the updated file from Github during the build, without doing a new release.) |
|
Ah, I found a way to do this in the Flatpak repository, so no hurry I guess. You could add this diff, if you like. diff --git a/src/debian/usr/share/metainfo/org.artisan_scope.artisan.metainfo.xml b/src/debian/usr/share/metainfo/org.artisan_scope.artisan.metainfo.xml
index 8303a20c..7950ddc5 100644
--- a/src/debian/usr/share/metainfo/org.artisan_scope.artisan.metainfo.xml
+++ b/src/debian/usr/share/metainfo/org.artisan_scope.artisan.metainfo.xml
@@ -67,6 +67,52 @@
</branding>
<releases>
+ <release version="3.0.0" date="2024-09-01">
+ <url type="details">https://github.com/artisan-roaster-scope/artisan/releases/tag/v3.0.0</url>
+ <description>
+ <p>ADDITIONS</p>
+ <ul>
+ <li>adds support for the execution of roast plans scheduled on artisan.plus</li>
+ <li>adds advanced summary statistics</li>
+ <li>adds support for transparent colors</li>
+ <li>adds {WEIGHTin} placeholder substitute by the current batch size (g) in command actions</li>
+ <li>adds additional button label substitutions, \V, \F and \T reporting the event value, the event value interpreted as temperature in Fahrenheit, and the event value interpreted as temperature in Celsius. The last two are automatically converted to the currently selected temperature unit.</li>
+ <li>adds "Load p-i-d from background" setting to configure the PID to the settings stored in background profile</li>
+ <li>capture and persist between-batch protocol (BBP) data as measured during a roasting session</li>
+ </ul>
+
+ <p>NEW HARDWARE SUPPORT</p>
+ <ul>
+ <li>adds machine support for the BeanGo Cube X</li>
+ <li>adds machine support for MUGMA Roasters</li>
+ <li>adds updated Sivetz fluid bed roasting machines support for the latest machines recording also fan changes</li>
+ <li>adds machine support for iRm roasting machines featuring Mitshubishi PLCs</li>
+ <li>adds Bühler RM20 Simatic Legacy setup supporting older firmware versions not returning the machine state (Issue #1529)</li>
+ <li>adds support for the Phidget RCC0004 server motor controller (Discussion #1546)</li>
+ <li>adds support for the to-be-release Phidget TMP1202 module</li>
+ <li>adds Hottop Command control to activate and deactivate the control function of the machine via event buttons and alarms</li>
+ <li>adds compression toggle, detailed device logging, and origin header to WebSocket communication</li>
+ </ul>
+
+ <p>CHANGES</p>
+ <ul>
+ <li>only reset roasting notes on RESET if profile is loaded (Issue #1521)</li>
+ <li>disable input filtering on device channel of binary or special types like NONE, dummy, and slider values</li>
+ </ul>
+
+ <p>FIXES</p>
+ <ul>
+ <li>fixes regression preventing Artisan v2.10.2 and v2.10.4 to connect successfully to Probat Series III machines via WebSockets incl. the Probat Sample Roaster (Issue #1531)</li>
+ <li>fixes regression in Roast Properties, unable to change/add ground color value (Issue #1520)</li>
+ <li>fixes regressions in table copy functions</li>
+ <li>fixes regression introduced in v2.8.4 that prevented the cupping chart being added properly to roast reports (Discussion #1563)</li>
+ <li>fixes event playback by temperature being blocked by already past background events if playback is turned ON during a roast</li>
+ <li>avoids rendering timestamps as "xx:60" in mouse pointer time/temp/RoR widget</li>
+ <li>ensures that time-axis ticks extend over the full range of readings w.r.t. the loaded background and foreground profiles</li>
+ <li>fixes a regression preventing the correct persistence of default SV values (Issue #1631)</li>
+ </ul>
+ </description>
+ </release>
<release version="2.10.4" date="2024-03-22">
<url type="details">https://github.com/artisan-roaster-scope/artisan/releases/tag/v2.10.4</url>
<description> |
|
There's one thing I'm not fully happy with yet, which I noticed recently: when saving a file, the default directory you're saving to is not something the user recognizes, but a Flatpak-specific Related: flatpak/xdg-desktop-portal#475, portal support in Qt, maybe flatpak/xdg-desktop-portal#997. update because of the autosave function, it could make sense to allow access to the user's homedir, and not only to files (and directories) specifically granted. It could be made to work with file sandboxing, but I'm not fully sure yet. update the specific issue of the default directory being off, seems like it has been fixed by flatpak/xdg-desktop-portal#982, and would work from xdg-desktop-portal 1.18 on. update selecting directories should also work since xdg-desktop-portal 1.7.1 flatpak/xdg-desktop-portal#200 (comment), and the autosave dialog does use the Qt dialog for selecting a directory. update adding permission to access the Documents path, so autosave works now update submitted to Flathub |
|
Sorry for being late, I forgot to update that metainfo.xml. Did it now and corrected the release date (1.9 => 1.8). |
|
Thanks for updating the metainfo file. |
|
Something just happend. I got an invite to join flathub/org.artisan_scope.artisan and did so. Anything further I need to do? |
|
The app is live on Flathub now! Next step would be to get the app verified. |
|
Working on verification... |
|
I did add the verification token to the artisan-scope.org site and clicked "Verify" under the Flathub Developer Portal. It now shows "Unverify", but the Artisan entry on Flathub still shows Unverified. Maybe this needs a moment? There is also a node on the Artisan Flathub page which states "App data review pending". Some entries are ticked already, most are pending. Anything we have to do on this? |
|
Thanks! The listing can take a few hours to propagate, sometimes. Let's see tomorrow. |
|
And verified for me. |
|
Cool! What a success. The linux download page already links to Flatpack. Thanks a lot for all that time you spent on this! Wonderful contribution! We couldn't have done this on our own as we lack the required compentences. Hope you are also a user and maybe you'd like to join the development team or at least the Linux-dev/release team;) Feel free to close this issue if you think this is done. |
|
Thank you for your kind words. I'm a happy (though fairly fresh) user, and would be glad to be involved in Linux-dev/release areas of Artisan. I think my next step would be looking at the installation and bundling process for the various platforms, for which I'll open a new issue. |
|
We are releasing Artisan v3.0.2, a patch release, today. I did update the metainfo.xml in the repository and also added a screenshot of Artisan 3 running on linux to the repository. Just received an email from the Flathub.Bot that the build fails. Not sure what triggered this build. Anyhow, I could not even figure out the reason why and what exactly failed from the provided log. Could you take a look? Thanks! |
|
Thank you for the new release, and adding a Linux screenshot. |
|
Thanks for this information and taking care. There is no hurry on this, this release just fixes a build issue that breaks the new (non-essential) scheduler feature, makes some smaller adjustments and updates some libs. We could have gone with just a build update on Linux and Windows, but we decided that the performance increase of using the ruff-based bidi implementation and that of the new numpy/scipy major versions would be worse a minor release. |
|
Seems to run fine. And nice to see the numpy major version bump (I'll do some more math-related testing then).
|
|
Excellent news! I removed the list of Contributors from the About file as this one was not maintained and to simplify things. Financial contributions are listed in the corresponding release blog posts and code contributions are managed by GitHub. |
|
In case you want or need to join the projects dev team on GitHub, let me know. |
|
Numpy and scipy tests were fine, merged the Flathub PR. |
Artisan is packaged for Linux as AppImage, Debian package and RPM. It's great to have this.
I would like to suggest to also offer a Flatpak package on Flathub, because:
I would be willing to work on this, and have a mostly working Flatpak already. Pending:
org.artisan_scope.artisan(dash becomes underscore, see here)624MB539MBunpacked; note that the deb is 727MB unpacked including Qt) - remove build dependencies; symlinked fonts already present in base system; prune yoctupuce binary libraries; remove QtWebEngine devtools, webdriver and dictionaries;also removed babel and QtWebEngine locales for languages which Artisan is not translated intoOptionally split locales to separate packagesdoesn't do much yet for this app, skip for nowfinish-args(do we need sounds? x11-fallback? ...)The text was updated successfully, but these errors were encountered: