Add flatpak support

[ea7ababe]

• Added a flatpak manifest and some other files under the flatpak/ directory:
  • Makefile - has rules to regenerate go-sources.json, modules.txt and js-sources.json
  • app.winboat.Winboat.metainfo.xml - flatpak metadata
  • app.winboat.Winboat.yml - flatpak manifest
  • go-sources.json - guest server dependencies
  • js-sources.json - main app dependencies
  • modules.txt - guest server modules file for go vendor
• Changed execFile* calls to be flatpak-aware, they now use flatpak-spawn when inside flatpak to call external programs (docker/podman and xfreerdp aren't shipped with the bundle).
• Added a CI job to build a flatpak bundle.

This isn't really Flathub-ready because of the use of flatpak-spawn (although, some apps that use it do get accepted, so I dunno), but it would be convenient to at least provide flatpak bundles from this github repository, as the current installation options do not work on, e.g. GNU Guix (appimages don't work there, not without hacks anyway).

[waffles-dev]

Thanks for this - haven't had a chance to go through it all yet but at a glance it looks high quality.

I think we're ok if the flatpak is podman only.

@tulilirockz mentioned a while back that podman-remote might be the way forward for flatpak, not sure exactly what her intention was for that approach though. I don't have any real working knowledge of flatpaks nor flathubs publishing requirements.

[auri-f5bde6]

@tulilirockz mentioned a while back that podman-remote might be the way forward for flatpak, not sure exactly what her intention was for that approach though. I don't have any real working knowledge of flatpaks nor flathubs publishing requirements.

I think, (but I'm not sure) the idea is to ship podman with the app in the flatpak, and expose the host podman socket to flatpak. Then podman inside the flatpak can use podman remote to connect to the socket.

[auri-f5bde6]

Also it might be a good idea to wait for #515 before dealing with flatpak. I think
But nice work!

[ea7ababe]

Food for thought: both docker and podman have HTTP APIs that can be used instead of their cli programs:

• https://docs.docker.com/reference/api/engine/version/v1.52/ (OpenAPI)
• https://docs.podman.io/en/latest/_static/api.html?version=latest (OpenAPI)

It's possible to generate a typescript client for those APIs and use it directly instead of relying on external CLIs:

docker run -it --rm -v "$PWD:/local" \
  openapitools/openapi-generator-cli generate \
  -i https://docs.docker.com/reference/api/engine/version/v1.52.yaml \
  -g typescript-node \
  -o /local/docker-client

Thus avoiding the need to package docker and podman altogether. The docker client will probably work for both docker and podman, as podman API is supposed to be compatible, but this needs testing (I haven't tried it yet).

[waffles-dev]

Also it might be a good idea to wait for #515 before dealing with flatpak. I think But nice work!

TY for having a look Auri - this sets off a chain haha as we want to do a full refactor (and rework the UI) of the config page first. I was waiting for Lev/Tibix to do the first sub-page but I should probably just give it a go.

[auri-f5bde6]

Food for thought: both docker and podman have HTTP APIs that can be used instead of their cli programs:

• https://docs.docker.com/reference/api/engine/version/v1.52/ (OpenAPI)
• https://docs.podman.io/en/latest/_static/api.html?version=latest (OpenAPI)

It's possible to generate a typescript client for those APIs and use it directly instead of relying on external CLIs:

docker run -it --rm -v "$PWD:/local" \
  openapitools/openapi-generator-cli generate \
  -i https://docs.docker.com/reference/api/engine/version/v1.52.yaml \
  -g typescript-node \
  -o /local/docker-client

Thus avoiding the need to package docker and podman altogether. The docker client will probably work for both docker and podman, as podman API is supposed to be compatible, but this needs testing (I haven't tried it yet).

That seemes a lot more work than just exposing the unix socket, and I don't know if it's worth it.

p.s. I can't really speak for whether any changes should be made to the flatpak packaging as I have no idea how it work.

[waffles-dev]

Hey @ea7ababe - are any of these files are automatically generated/updated?

[ea7ababe]

Hey @ea7ababe - are any of these files are automatically generated/updated?

go-sources.json, js-sources.json and modules.txt contain all Go and JS dependencies, and are generated by the rules in the Makefile (it's in the same directory as those files).

They aren't updated automatically though, someone would need to periodically run those rules, and commit their output into the repository - I didn't include the generation step into the CI pipeline.

[ea7ababe]

The metainfo file was written manually, but if you worry about keeping it in sync with the release notes on the github page, then I imagine there would need to be a NEWS file, from which both would be generated.

[waffles-dev]

Hey @ea7ababe - are any of these files are automatically generated/updated?

go-sources.json, js-sources.json and modules.txt contain all Go and JS dependencies, and are generated by the rules in the Makefile (it's in the same directory as those files).

They aren't updated automatically though, someone would need to periodically run those rules, and commit their output into the repository - I didn't include the generation step into the CI pipeline.

Thanks - I'm wondering if we can move it into the CI pipeline/general build process (for local builds) and not have them committed?

[ea7ababe]

I'm wondering if we can move it into the CI pipeline/general build process (for local builds) and not have them committed?

Hey, sorry for the delay. Can you look at the commit above? I've removed the generated files, and added a docker buildx bake -based build process to generate these files and build the flatpak bundle.

Locally one would use it like this:

docker buildx create \
  --name winboat-ctx \
  --buildkitd-flags --allow-insecure-entitlement=security.insecure \
  --driver docker-container --use
docker buildx bake flatpak

And in CI just tick the "With flatpak" box, it uses the same process, essentially.

I've had to add the security.insecure entitlement due to flatpak-builder's use of bwrap and namespaces (relevant issue flatpak/flatpak#3027).

[waffles-dev]

Oh cool - I'll try it when I get time, hopefully this weekend. I was half expecting it to rely on the org.flatpak.Builder flatpak - but I'm a complete newbie when it comes to this side of things.

[ShaunRoselt]

Oh cool - I'll try it when I get time, hopefully this weekend. I was half expecting it to rely on the org.flatpak.Builder flatpak - but I'm a complete newbie when it comes to this side of things.

Have you had time to try it yet?

[waffles-dev]

Oh cool - I'll try it when I get time, hopefully this weekend. I was half expecting it to rely on the org.flatpak.Builder flatpak - but I'm a complete newbie when it comes to this side of things.

Have you had time to try it yet?

not yet - sorry this slipped! been focusing on the things we're aiming to get in for 0.9.1 - I think Tibix/Levev are planning to look at this after that next release goes out (but I could be wrong). I'll try and get a look in soon I promise!

[waffles-dev]

When I run I get

[+] Building 0.0s (1/1) FINISHED                                                                                             docker-container:winboat-ctx
 => [internal] load local bake definitions                                                                                                           0.0s
 => => reading docker-bake.hcl 258B / 258B                                                                                                           0.0s
Your build is requesting privileges for following possibly insecure capabilities:

 - Running privileged containers that can make system changes

In order to not see this message in the future pass "--allow=security.insecure" to grant requested privileges.

Your full command with requested privileges:

docker buildx bake --allow=security.insecure flatpak

Do you want to grant requested privileges and continue?

Not sure how comfortable I feel about that message on dev machine, maybe in a VM or something I'd not care so much...

[ea7ababe]

I was half expecting it to rely on the org.flatpak.Builder flatpak

I initially tried to make it work using org.flatpak.Builder, but it was missing the flatpak-go-mod program, and its not as easy to add things to existing flatpaks, as it is with docker. So I settled on the docker approach instead.

Not sure how comfortable I feel about that message on dev machine, maybe in a VM or something I'd not care so much...

So far, I don't really have any good ideas on how to design a build system, that doesn't use those privileges. Both docker and flatpak-builder want to create an isolated build environment to ensure some level of determinism in the build process, but flatpak-builder can't do it inside of docker without those special privileges.

One idea that comes to mind is to build a custom flatpak for building the app, that includes org.flatpak.Builder as a "base app" and adds other necessary tools on top, and then build winboat flatpak using that custom builder flatpak... but as far as security goes, you don't really gain anything from it. It's just now flatpak-builder is executing inside flatpak's sandbox (which obtains these permissions via org.flatpak.Builder's manifest), instead of docker's. And this process seems more complicated to me.

[waffles-dev]

Hmm, ok. I really wish I knew more about this stuff myself - sorry I'm so new to building flatpaks!

I think this needs to be updated for latest main, given Tibix has moved the build system to bun?

[ea7ababe]

I think this needs to be updated for latest main, given Tibix has moved the build system to bun?

Well, the bad news is that flatpak-node-generator doesn't suport bun.lock yet (flatpak/flatpak-builder-tools#507), and package-lock.json was removed — so I can't rebase just yet. I'll try to add support for it over the weekend.

[ea7ababe]

Oof, I've just realized, that flatpak doesn't even have a bun extension yet. I modified flatpak-node-generator to handle bun.lock, but the app itself would still need to be built and executed by node... which makes handling bun.lock directly somewhat pointless, as node would still need a package-lock.json. I'm afraid we might have to put this away until bun itself is packaged for flatpak.

[ea7ababe]

Actually, sorry, maybe I got confused yesterday. Do I understand correctly, that winboat switched to bun only at build time for its package manager, and not runtime? I'm trying to find anything on bun support in electron itself, and it seems vanilla electron is currently for node only.

Would it be too much of a maintenance burden to support both bun and node for building? Currently, bun can convert bun.lock into yarn.lock, which then can be picked up by yarn or npm. This can be done automatically at build time, without checking the other lock files in.

[waffles-dev]

I'm not sure - that's a question for @TibixDev