Consider support for Flatpak/snapd

[v6ak]

Both Flatpak and Snapd can be eused alogside of usual package managers. Flatpak seems to be more interesting, because it supports per-user installation. So, if you need a specific app in one VM (e.g., Skype or Spotify), you can install it there without affecting the template and without cloning it. Snapd does not seem to (currently) support it.

For snapd support, it should be relatively easy:

• Perform snaps updates when updating apt/yum/pacman/… packages.
• Check for snap updates when updating apt/yum/pacman/… packages.

For basic support of Flatpak, it is the same:

• Perform Flatpak updates when updating apt/yum/pacman/… packages.
• Check for Flatpak updates when updating apt/yum/pacman/… packages.
• Configure Flatpak to use updates proxy in TemplateVM.

For user installs, it might be a little harder:

• Allow a template-based VM to perform its own updates.
• Split user Flatpak updates and system Flatpak updates.
• Automatically choose user install when running in TemplateBasedVM and system install otherwise.

Related discussion: https://groups.google.com/forum/#!topic/qubes-users/VDs-5y6tUlI

[tapir1]

Any updates towards this milestone? I'm eager to have this.

[maximilize]

I just wrote a simple script which makes snappy installs persistent over reboots in app VM's.
https://gist.github.com/maximilize/5043fc09614438791dce3406cc6b69f7

[marmarek]

@maximilize looking at your script it seems enough to make some directory persistent using bind-dirs. For example /rw/config/qubes-bind-dirs.d/50_snapd.conf:

binds+=( '/var/lib/snapd' '/snap' )

This doesn't handle snap-*.mount files, can you provide example one and when it is created?

[maximilize]

The mount files are created by snap during app installation. Here a script to recreate the systemd mount files:

#!/bin/bash -e

cd /var/lib/snapd/snaps/
while read app revision; do
		cat >/etc/systemd/system/snap-$app-$revision.mount <<EOF
[Unit]
Description=Mount unit for ${app}, revision ${revision}
Before=snapd.service

[Mount]
What=/var/lib/snapd/snaps/${app}_${revision}.snap
Where=/snap/${app}/${revision}
Type=squashfs
Options=nodev,ro,x-gdu.hide

[Install]
WantedBy=multi-user.target
EOF
		sudo systemctl -q enable snap-$app-$revision.mount
		sudo systemctl -q start snap-$app-$revision.mount
done < <(ls *.snap | cut -d'.' -f1 | sed 's/_/ /')

[maximilize]

Here my final solution: https://gist.github.com/maximilize/6070958a08246f7a849379a9d530faa9

It would be great to see a system like this integrated into qubes. With snap, it is possible to install new persistent apps in AppVM's, without the need to touch the template.

[marmarek]

I've packaged it here: https://github.com/QubesOS/qubes-app-linux-snapd-helper
Slightly modified:

• split it into separate files
• use systemd unit features for startup condition, check /usr/bin/snap (not snapd) to handle Fedora too
• add rpm packaging

Debian packaging on the way.
Many thanks @maximilize!
One missing piece is adding menu entries, but that depends on #4152 (then, it's only matter of adding yet another directory - /var/lib/snapd/desktop/applications).

[maximilize]

Awesome, thanks for integrating it so quickly!

[marmarek]

Take a look at it, especially if you're ok with GPLv2+ license (right now only mentioned in rpm file, need to add actual license file).

[maximilize]

Sure, GPLv2 is fine. I already checked your code, it all seems to be good.

[marmarek]

Package uploaded to current-testing repository for R4.0.

[tasket]

This sounds really good. I'd just like to mention that the ability to install in /rw per-user comes with the downside that persistent malware can reside there. This should be mentioned when documenting the feature.

As an alternative, it may be possible to have templates configured for a separate snapd/flatpak volume that is read-only to appvms. (Maybe they could even be shared between templates, at least in snap's case - although I'm not sure if this is a great way to save resources.) Otherwise, simple installation into template root is the better and more Qubes-like policy.

[msm-code]

Note for other people as dense as I was today:

You're supposed to install qubes-snapd-helper package in the template vm, to be able to install snaps in derived vms.

Cheers.

[tasket]

@maximilize qubes-snapd-helper apparently doesn't allow installation into templates (or at least not via the update proxy). Is there any guide for installing snaps into a template instead of appVM /rw?

The reason I ask is that some packages are now only readily available as snaps, so there is a use case where the user simply wants to use snapd because software is not available via default apt or dnf repos. In this case, the software stays in root and is therefore more secure than if it resided in /rw.

[ioleo]

@marmarek is there any documentation how to use this qubes-snapd-helper?
So far I managed to install it in TemplateVM. But I can't get my snaps persistent in AppVM. I'm probably missing some step here.

[jpouellet]

I had a fedora template reliably hang while trying to apply (dnf) updates during some flatpack step. I had to go in and kill some flatpack process for the VM to be able to proceed with updates. Unfortunately I don't remember the details of exactly what the process was and what it was blocked on, and I don't have an old template with the same behavior lying around to reproduce, but figured I'd mention it anyway. It was reproducible at the time though (trying to update would hang every time, until I finally helped it past that step by killing the process).

[marmarek]

Like #4466 ? I've seen that too. I haven't managed to reproduce it reliably, but it looked like waiting for some network data. Which is weird, as templates don't have normal network interface (nor flatpak is configured to use updates proxy), so it should fail immediately with "no route to host" error.

Back to the topic of this issue, flatpak generally should work fine in TemplateBasedVM as long as flatpak install --user is used. In TemplateVM on the other hand, flatpak install --system could be a good way, but it needs to be configured to use updates proxy. I haven't found out how, yet.

I've updated description to include those steps.

[jpouellet]

Like #4466 ?

Yes, IIRC exactly that.

[v6ak]

Back to the topic of this issue, flatpak generally should work fine in TemplateBasedVM as long as `flatpak install --user` is used.

I can confirm it. It works well for me. I see two pitfalls: * You have to add repositories with --user, too. I am not sure if repositories added without --user are used when installing with --user, but even if they are, they obviously won't survive reboot unless they are in the template. * If you accidentally use sudo with --user, you obviously install the app for root, not for user. While this looks quite obvious, it took some time to realize it. So, when you just copy a command from some howto, you likely need to both add `--user` and remove `sudo `.

[ioleo]

@marmarek Never mind my comment above.

For future reference/others - it seems all you need to do is to (for Fedora based VMs):

• sudo dnf install qubes-snapd-helper snapd in Template VM
• shutdown Template VM (and App VM if it was running) so that changes are applied
• snap install <package> in App VM
• shutdown App VM
• now, go to Qube Settings for your App VM --> Applications tab --> click Refresh Applications
• your <package> is now not only persistent, but also registered as any other package and can be selected for the quick launch menu

WARNING My problem was caused by symlink sudo ln -s /var/lib/snapd/snap /snap (in TemplateVM) which I added at some point to make snapd work (before I learned about the helper). If you did like me, simply remove the symlink sudo rm /snap, it conflicts with the helper.

[100111001]

I've used the instruction above to install 2 programs using snap (slack+freeplane):

Unfortunately, it happens on a regulary basis (weekly), that starting these apps results in the following error:
[user@b ~]$ slack
internal error, please report: running "slack" failed: cannot find installed snap "slack" at revision 17: missing file /var/lib/snapd/snap/slack/17/meta/snap.yaml
[user@b ~]$ snap slack
error: unknown command "slack", see 'snap help'.

After that the progs are not startable anymore. The same error is shown. Restart doesn't help.

The workaround is to re-install the snap programs. Unfortunately, with this you lose all settings of the progs.

Any suggestions or ideas?

[100111001]

I noticed that my issue reported above disappears if I disable connections to api.snapcraft.io
Of course after that it's not possible to use snap to refresh or install new applications. But at least the installed apps do not break anymore.
So, existing workaround is:

1. install all needed snaps
2. disable connections to api.snapcraft.io using the build in firewall
3. (to manual refresh/update/install of the apps you can re-enable connections to api.snapcraft.io temporary)

[heinrich-ulbricht]

Following the instructions from @ioleo worked like a charm for Signal Messenger. I successfully installed it to a current Fedora 30-based AppVM. Thank you.

Note that when installing snapd a failure message is being shown but as discussed in #5392 this shouldn't and apparently doesn't matter.

Now let's see if I run into the same issues as @100111001. For now I'll allow connections to api.snapcraft.io.

[100111001]

@heinrich-ulbricht have you experienced the same issue I described? I faced it again on 2 different machines. At the one with access to api.snapcraft.io slacks and other apps stop starting from time to time and I have ro reinstall them (via snap). At the other with blocked access to api.snapcraft.io no so such issue appears.

[heinrich-ulbricht]

@100111001 I had no problems since installing Signal Messenger on my machine. So far it just works.

[100111001]

@100111001 I had no problems since installing Signal Messenger on my machine. So far it just works.

I'd like to try to narrow it down to a root cause. I'm using:

• a fedora-30 template
• snap apps with and without "--classic" argument for snap
• snap --version: snap 2.42.1-1.fc30

@heinrich-ulbricht could you provide your system details?

[v6ak]

I remember similar issues with snapd, although I don't have exact description. As far as I have remember, even `--classic` snaps (namely Powershell) have those issues.

[heinrich-ulbricht]

@100111001

• Fedora 30 (was a Fedora 28 template, then upgraded to Fedora 30)
• classic or not? don't know, using qubes-snapd-helper, followed instructions here Consider support for Flatpak/snapd #2766 (comment)
• snap --version:

snap    2.42-2.fc30
snapd   2.42-2.fc30
series  16
fedora  30
kernel  4.19.81-1.pvops.qubes.x86_64

Since I'm only using one snap so far (Signal Messenger) I might be of limited help here. But we can try :)

[v6ak]

The `--classic` option is required by some snaps and it disables sandboxing. As I can read at https://snapcraft.io/signal-desktop[https://snapcraft.io/signal-desktop] , Signal messenger doesn't require it, so you probably don't use it, unless you have explicitly (and probably knowingly) specified this option.

[ar4s-eth]

@marmarek Never mind my comment above.

For future reference/others - it seems all you need to do is to (for Fedora based VMs):

• sudo dnf install qubes-snapd-helper snapd in Template VM
• shutdown Template VM (and App VM if it was running) so that changes are applied
• snap install <package> in App VM
• shutdown App VM
• now, go to Qube Settings for your App VM --> Applications tab --> click Refresh Applications
• your <package> is now not only persistent, but also registered as any other package and can be selected for the quick launch menu

WARNING My problem was caused by symlink sudo ln -s /var/lib/snapd/snap /snap (in TemplateVM) which I added at some point to make snapd work (before I learned about the helper). If you did like me, simply remove the symlink sudo rm /snap, it conflicts with the helper.

Sorry if this is a stupid question (just getting started with Linux): After installing qubes-snapd helper in the Fedora Template VM (and rebooting), am I supposed to be able to invoke "snap" in the App VM? (command not found). Am I also required to snapd in the template VM?

[marmarek]

Yes, you also need to install snapd in the template.

[Echocage]

@100111001 I have the exact same problem.

This time when I restarted, it was /var/lib/snapd/snaps/spotify_42.snap that was removed. when I try to run it, I see

internal error, please report: running "spotify" failed: cannot find installed snap "spotify" at revision 42: missing file /snap/spotify/42/meta/snap.yaml
and running snap list results in

Name Version Rev Tracking Publisher Notes
spotify 42 latest/stable spotify✓ broken

Everytime I start my VM, I see a random number of

Device Removed /var/lib/snapd/snaps/*.snap is removed

showing which snap install broke this time.

Does anyone have a solution to this problem? This has really be impacting my work, snaps are randomly broken constantly.

Notes: I'm running qubes 4.0.3 with debian appvms, and most of these packages do require --classic to function.

[marmarek]

Do you have /snap as a symlink to /var/lib/snapd/snap? It should look like this:

$ ls -l /snap
lrwxrwxrwx 1 root root 19 Sep 12 13:12 /snap -> /var/lib/snapd/snap

Check at both times when application is working and when it just broke.
Check also if /var/lib/snapd is properly mounted:

$ mount|grep ' /var/lib/snapd '
/dev/xvdb on /var/lib/snapd type ext4 (rw,relatime,discard)

[Echocage]

@marmarek
So the /var/lib/snapd seems to be mounted correctly, but it looks like my ls -l /snap isn't returning the link

user@work:~$ ls -l /snap
total 40
drwxr-xr-x 2 root root 4096 Oct 18 00:50 bin
drwxr-xr-x 4 root root 4096 Oct 18 00:50 core
drwxr-xr-x 3 root root 4096 Oct 12 02:14 core18
drwxr-xr-x 3 root root 4096 Oct 12 02:14 dbeaver-

Any idea why this isn't properly being linked automatically? What can I do to fix this?

It also doesn't look like /snap exists in /var/lib/snapd, I wonder why that is

ls: cannot access '/var/lib/snapd/snap': No such file or directory

Idk if this matters, but I installed snap on the template, is that what's causing this issues? I installed the actual snap packages on the appvm
All of my qubes seem to have this same issue for whatever reason.

[marmarek]

I missed you said Debian VM - there /snap is supposed to be a directory, not a symlink.
Make sure you have qubes-snapd-helper installed in the template.

[Echocage]

I missed you said Debian VM - there /snap is supposed to be a directory, not a symlink.
Make sure you have qubes-snapd-helper installed in the template.

Oh shoot, gotcha. I've got qubes-snapd-helper on the template. Very strange that it's happening on all my cubes. Maybe I'll try creating some fresh qubes from scratch and see if it's still an issue. Thanks anyways mate!

[Echocage]

Does anyone have a solutio

@100111001 I have the exact same problem.

This time when I restarted, it was /var/lib/snapd/snaps/spotify_42.snap that was removed. when I try to run it, I see

internal error, please report: running "spotify" failed: cannot find installed snap "spotify" at revision 42: missing file /snap/spotify/42/meta/snap.yaml
and running snap list results in

Name Version Rev Tracking Publisher Notes
spotify 42 latest/stable spotify✓ broken

Everytime I start my VM, I see a random number of

Device Removed /var/lib/snapd/snaps/*.snap is removed

showing which snap install broke this time.

Does anyone have a solution to this problem? This has really be impacting my work, snaps are randomly broken constantly.

Notes: I'm running qubes 4.0.3 with debian appvms, and most of these packages do require --classic to function.

Does anyone have any solution to this? This happens when I start new containers for both debian and fedora. Packages will almost always break within 1-3 restarts of the appvm. Is there a way that I can stop this from happening?

[100111001]

I haven't checked the suggestion from Marek (#2766 (comment)), yet, as this issue happens now less often. But from time to time my slack breaks the same way. My latest hypothesis is that it's a concurrency issue during start of the AppVM, because sometimes when this error occurred, it could be solved by rebooting the AppVM. As soon as I couldn't resolve this after the 3rd reboot, I re-installed the (slack) snap.

It's not really a solution, but at least 2 workarounds which work for me.

[100111001]

Do you have /snap as a symlink to /var/lib/snapd/snap? It should look like this:

$ ls -l /snap
lrwxrwxrwx 1 root root 19 Sep 12 13:12 /snap -> /var/lib/snapd/snap

Check at both times when application is working and when it just broke.
Check also if /var/lib/snapd is properly mounted:

$ mount|grep ' /var/lib/snapd '
/dev/xvdb on /var/lib/snapd type ext4 (rw,relatime,discard)

Once again it happened that a snap was missing after a restart of the VM everything works again.

Both

$ mount|grep ' /var/lib/snapd '

and

$ ls -l /snap

show the same result not depending on whether snap is working or not.

Looking forward to support in further investigations.

[100111001]

Trying to execute the snap application at command line results in:

cannot perform operation: mount --rbind /dev /tmp/snap.rootfs_Sfyhgj//dev: No such file or directory

After a restart of the AppVM, the snap starts like a charm.

[100111001]

Additional observation: Since it happened again that my snap refused to start, I checked the directory
/var/lib/snapd/snap/my-snap/19
and it was empty. After a AppVm restart it contains all files and the snap is starting.
(Template for the AppVM is fedora 32)

[tlaurion]

Hello there, couple questions on flatpak/snap usage here.

Let's take for granted the usage of the following apps: session-desktop, riseup-vpn, signal-desktop; which distributes themselves as snap images through snap default repositories.

Let's take for granted that only snapd and qubes-snapd-helper are installed in TemplateVM, while those applications are installed as snap install app in target AppVM. (snap install session-desktop riseup-vpn signal-desktop)

• How are the applications updated by snap automatically? Randomly under a AppVM life? So the user launches an AppVM by launching the desired App. At launch, it runs the old version and then at some point it is updated and the user needs to restart the app to take advantage of the newer version? From my little experiment, the user is not made aware he is running an old version when launching it; while the goal here would be to have required apps per AppVM, without awareness of needed updates prior of launching a vulnerable app; unless that app would have been installed in TemplateVM, and that app be upgraded per TemplateVM upgrades. From my understanding, users installing apps in AppVMs will run outdated apps from AppVM launch without being aware of it.
• How does the user know verifies the integrity/validity/authenticity of an app being deployed?
• How does the user know that the applications have been updated prior of usage (where applications being updated in TemplateVMs per normal package deployment is explicit)
• What about flatpak/snap critics here: https://flatkill.org/2020/

Basically: would you recommend to QubesOS users to use snap/flatpak?
This is in link with @DemiMarie comment here: #1939 (comment)

I am still not convinced about flatpak/snap and am looking to have my mind changed. Thanks

EDIT:

• debian-10 snapd is too old to support session-desktop or signal-desktop (2.47+ required, where debian-10)

[andrewdavidwong]

FYI, @micahflee has just published "Qube Apps: a Flatpak-based app store for each qube", which looks very cool and might be of interest to folks here.