Provide Flatpak builds

[cassidyjames]

Describe the feature or problem you’d like to solve

Users of elementary OS and other desktops may have a hard time finding, installing, and trusting this fork of the GitHub Desktop app.

Proposed solution

If Flatpak builds were included, it would be a one-click install on elementary OS and many other Linux distros. Further, if the pipeline were set up with Flathub.org, it would be available in app stores out of the box on some distros and more discoverable on others.

Additional context

Flatpak is just a packaging technology, not a store. You can provide Flatpaks in several ways, with different advantages. The simplest way in CI may be by building a Flatpak bundle, but I don't think that will handle updates. A better long-term solution would likely be to submit the Flatpak to a centralized remote like Flathub.

Flathub supports trusted developers pushing releases to Flathub via CI, though I'm not sure of the specifics. Hopefully someone from the project could chime in here.

I'm coming here from Twitter. 😉

[shiftkey]

👋 thanks for starting this discussion off @cassidyjames!

The first thing that comes to mind is https://github.com/endlessm/electron-installer-flatpak which supports a programmatic API to generate the flatpak from a packaged Electron app using Node (which is how our other packaging works currently).

That's the path I'd like to try first for a couple of reasons:

• it supports passing in options to configure the equivalent of a flatpak manifest, and appears to provide some sane defaults
• it feels very similar to electron-installer-debian, which I've been using in move debian packaging out to dedicated step #263 and felt less magical than electron-builder
• it can follow our existing pattern for generating installers as part of a release

Once I've wrapped #263 I'll drop some instructions in here for how we can follow the same pattern from #263 if someone wants to take a shot at wiring up the packaging step.

[shiftkey]

cc @jfgordon2 who I know contributed some Elementary OS integrations previously

[jfgordon2]

@shiftkey this sounds super interesting, I'd be happy to take a look once you've established a pattern for the debian builder

[shiftkey]

@jfgordon2 I merged #263 and was planning to publish an update during the week, but the key parts to the module can be found in script/package-debian.ts if you wanted to setup an equivalent for flatpak support.

I also annotated the PR itself with additional context: #263 (review)

[gasinvein]

Hello there. I've managed to build the app with flatpak-builder without network access, so it could be submitted to Flathub (offline builds are mandatory there).
flatpak-builder manifest is here, sources are generated with flatpak/flatpak-builder-tools#118.

There are two major obstacles severely complicating build, and since I'm not very familiar with electron and friends, it would be nice if the developers could provide some help with these:

• There is no easy way to point electron-download to a pre-downloaded cache. I believe electron-get handles it better (at least it should respect electron_config_cache env var).
• node-sass must match node abi version (since it's used during build time), while other native modules must match electron's abi version. The tricky part is that since we build offline, node headers can't be downloaded on demand, only pointed to via npm_config_nodedir env var.

[shiftkey]

Hello there. I've managed to build the app with flatpak-builder without network access, so it could be submitted to Flathub (offline builds are mandatory there).

@gasinvein I'd love to hear more about this, especially the offline builds bit, because this feels like a fair bit of additional work to adapt things. Not to say it's impossible, it's just going against how the project tooling has been optimized.

[gasinvein]

@shiftkey By default, flatpak-builder downloads all sources by itself and runs build process without network access. While it's just a default for flatpak-builder, Flathub. the de-facto "flatpak store", require it.
For things like yarn, there are some helpers that read yarn.lock and produce drop-in manifest part for flatpak-builder, which then populates yarn offline mirror before spawning build process. It's simple in general, but become trickier with node packages that do download files during install. Making such packages work usually involves pre-downloading download cache for them.

Also, keep in mind that it's only Flathub requirement, so if you don't plan to submit the app there - building offline isn't necessary.

[shiftkey]

@gasinvein thanks for digging into this. I can fill in some context from my end about how things are setup.

There are two major obstacles severely complicating build, and since I'm not very familiar with electron and friends, it would be nice if the developers could provide some help with these:

For context, the project itself contains two package.json files:

• the one at the root is there for tooling, and any native modules would target the installed node version
• the one under app is for the Electron-specific dependencies, and leverages the app/.npmrc config to target our specific version of Electron

• There is no easy way to point electron-download to a pre-downloaded cache. I believe electron-get handles it better (at least it should respect electron_config_cache env var).

I believe electron-download is only used by electron-chromedriver these days, but electron-download does support an environment variable override to control where it downloads the files from: https://www.npmjs.com/package/electron-download

You can set the ELECTRON_MIRROR or NPM_CONFIG_ELECTRON_MIRROR environment variable or mirror opt variable to use a custom base URL for grabbing Electron zips.

• node-sass must match node abi version (since it's used during build time), while other native modules must match electron's abi version. The tricky part is that since we build offline, node headers can't be downloaded on demand, only pointed to via npm_config_nodedir env var.

Ideally our Electron-specific native modules should use prebuild to obtain the native modules they need (based on the ABI version) from GitHub releases, so they shouldn't need to fall back to downloading headers and building from source.

It's been a while since I've checked those though - if you're seeing specific problem packages that don't seem to behave this way we can look into whether there are upgrades pending. Alternatively, if there's a cheat sheet of tools to install and commands to run against your repository to see these problems myself, I can dig into them later this week when I have some more time.

[gasinvein]

I believe electron-download is only used by electron-chromedriver these days, but electron-download does support an environment variable override to control where it downloads the files from: https://www.npmjs.com/package/electron-download

If I understood it right, electron-packager 13.x.x, used by GitHub Desktop, uses electron-download.

You can set the ELECTRON_MIRROR or NPM_CONFIG_ELECTRON_MIRROR environment variable or mirror opt variable to use a custom base URL for grabbing Electron zips.

That's the problem - we can control only from where it'll download electron, no way to make it use existing local files without downloading anything.

commands to run against your repository to see these problems myself

I've already circumvent these problems, so the app can be built and ran like any other flatpak. It's just that applied workarounds are ugly and fragile, so it'll be hard to maintain.
I guess that completely replacing electron-download with electron-get could help.

[shiftkey]

If I understood it right, electron-packager 13.x.x, used by GitHub Desktop, uses electron-download.

Looks like this got ported in v14.0.0 - we should be on this version too based on the last tagged release:

desktop/package.json

Line 172 in 4b32c1e

"electron-packager": "^14.2.1",

That's the problem - we can control only from where it'll download electron, no way to make it use existing local files without downloading anything.

Let's see if we can get spectron and electron-chromedriver (which I believe is the last vestige of electron-download that's relevant to us) to also switch over to @electron/get if that's easier to interoperate with (I think that's what the core team wants to see projects using anyway).

EDIT: looks like the first upgrade has landed on master and will go out in the latest electron-chromedriver release electron/chromedriver@3c22216#diff-b9cfc7f2cdf78a7f4b91a753d10865a2. Then it would just be a matter of a new spectron upgrade to go with it.

[gasinvein]

@shiftkey Thanks for clarification. Still, I'm pretty sure something it triggering electron (not chromedriver) download via electron-download. Can it be somehow caused by this?

desktop/package.json

Line 129 in 4b32c1e

"@types/electron-packager": "^13.0.0",

[shiftkey]

I've been using the lockfile as that makes explicit what packages were being resolved.

I don't believe it's that package, because the version we have doesn't depend on either electron-download or @electron/get:

https://github.com/shiftkey/desktop/blob/527d9668c08b2ac4510389e3587ca8527d8b635e/yarn.lock#L289-L294

I believe it's this package:

https://github.com/shiftkey/desktop/blob/527d9668c08b2ac4510389e3587ca8527d8b635e/yarn.lock#L3496-L3502