[query] [electron-updater] Why is a subset of linux distros supported for autoupdates?

[xyloflake]

So as the title states, I was looking for a solution to make auto-update work on platforms like arch linux. I read a comment made by @develar that it is by intention - that some platforms are not supported. However I failed to find any issue that mentioned the "why". I'd like to know why it is unsupported.

Thanks in advance!

[xyloflake]

What are the problems associated with implementing it for some distros is another thing I'd like to know :)

[mmaietta]

I'm not familiar as to why more aren't supported. When I joined the crew, originally it was only AppImage. I implemented deb and rpm autoupdates after a significant testing period. For other platforms like arch linux, someone will need to do the implementation (happy to provide guidance) and test the different package managers that could be utilized.

Basically, for implementation, all that is needed are the spawn cmds for the other platforms to be supported. Here's how deb updates were implemented

electron-builder/packages/electron-updater/src/DebUpdater.ts

Lines 30 to 40 in 28c7864

	protected doInstall(options: InstallOptions): boolean {
	const sudo = this.wrapSudo()
	// pkexec doesn't want the command to be wrapped in " quotes
	const wrapper = /pkexec/i.test(sudo) ? "" : `"`
	const cmd = ["dpkg", "-i", options.installerPath, "||", "apt-get", "install", "-f", "-y"]
	this.spawnSyncLog(sudo, [`${wrapper}/bin/bash`, "-c", `'${cmd.join(" ")}'${wrapper}`])
	if (options.isForceRunAfter) {
	this.app.relaunch()
	}
	return true
	}

[xyloflake]

I'm not familiar as to why more aren't supported. When I joined the crew, originally it was only AppImage. I implemented deb and rpm autoupdates after a significant testing period. For other platforms like arch linux, someone will need to do the implementation (happy to provide guidance) and test the different package managers that could be utilized.

Basically, for implementation, all that is needed are the spawn cmds for the other platforms to be supported. Here's how deb updates were implemented

electron-builder/packages/electron-updater/src/DebUpdater.ts

Lines 30 to 40 in 28c7864

	protected doInstall(options: InstallOptions): boolean {
	const sudo = this.wrapSudo()
	// pkexec doesn't want the command to be wrapped in " quotes
	const wrapper = /pkexec/i.test(sudo) ? "" : `"`
	const cmd = ["dpkg", "-i", options.installerPath, "||", "apt-get", "install", "-f", "-y"]
	this.spawnSyncLog(sudo, [`${wrapper}/bin/bash`, "-c", `'${cmd.join(" ")}'${wrapper}`])
	if (options.isForceRunAfter) {
	this.app.relaunch()
	}
	return true
	}

I would be super happy to implement for arch linux, am I allowed to do that? I thought of making a PR but was afraid of it being rejected because of what @develar said.

Even before the above instructions were posted, I had figured it out on how to implement it :)

I also have another patch for a repo I work on. It's for electron-builder 24.13.3 since that's the stable version, and it reduces the size by around 7MB! For larger codebases, this could result in an even more compressed installer, which is awesome. However, I've found some anomalies in the codebase that are bugging me before I make the PR. Could you check them out?

Thanks a ton! Can't wait to hear your thoughts! 🚀

[mmaietta]

I'm not opposed to supporting arch linux if you're willing to put up a PR! We'll just need to flag it as a Beta Feature like the deb/rpm autoupdaters.
I'm not sure what you're referring to that develar would reject. Would you mind providing a link to that conversation?

I'm not sure I follow w.r.t. the anomalies. The v25 CI builds are passing correctly.

Re: the node-gyp errors, I'm happy to take a look if you can put together a minimum reproducible repo? (Or do I just use the muffon repo itself with npm run package:all?)

[xyloflake]

I'm not opposed to supporting arch linux if you're willing to put up a PR! We'll just need to flag it as a Beta Feature like the deb/rpm autoupdaters.

Super cool! Will make a PR after patching 24.13.3 in the muffon repo and will test out in the betas if the autoupdates work well or not for arch! Sounds alright?

I'm not sure what you're referring to that develar would reject. Would you mind providing a link to that conversation?

Yes! Sure. Here is the issue in which develar states "Some platforms are intentionally not supported" - #3844

Just like you stated earlier; at that time only appImage was supported.

I'm not sure I follow w.r.t. the anomalies. The v25 CI builds are passing correctly.

Ah, the build info is outdated as of now. What I was trying to reference is

Electron builder internally resets the compression configuration option to normal for Windows, albeit without explicit rationale. Consequently, this patch intervenes to prevent the reset to normal exclusively for Windows systems, as the aforementioned algorithm cannot operate optimally on other platforms.

As in why it's reset to normal without any reason? It works fine without that... Also, only Windows is affected by this "normal" thing iirc.

Re: the node-gyp errors, I'm happy to take a look if you can put together a minimum reproducible repo? (Or do I just use the muffon repo itself with npm run package:all?)

Oh forget that please it may be outdated as of now because I see the CI/CD pipeline getting through fine. At the time of writing that, the CI/CD pipeline also failed 😄.

muffon does not have the error as we don't use electron-builder v25/v24 alphas for stability purposes. I'll check over the weekend if the issue persists or not and will notify based upon that!

Thank you so much for the support! Is there any community chat server for electron-builder? I'd suggest something like Slack/Discord/Matrix/etc. for chatting about and casually asking about stuff 😅

[mmaietta]

Very interesting comment regarding using package managers. I could understand the use case there; I guess I interpreted that electron apps were designed to allow distribution outside of package managers or App Stores.

Related, I discovered that there's an electron-builder channel within the Electron discord community https://discord.com/channels/745037351163527189/746375188358365204

[xyloflake]

Very interesting comment regarding using package managers. I could understand the use case there; I guess I interpreted that electron apps were designed to allow distribution outside of package managers or App Stores.

Were designed or were not designed? I feel like you wanted to say NOT designed?

Related, I discovered that there's an electron-builder channel within the Electron discord community https://discord.com/channels/745037351163527189/746375188358365204

Cool, but I think this deserves a dedicated discord server since doing all of development and asking questions might be a pain in one single channel.

Can I also have your discord and reach out so I can ask some more questions if you don't mind?

Thank you so much for the reply! Can you uhh maybe address the compression regarding concerns as well?

[mmaietta]

I interpreted that electron apps were designed to allow distribution outside of package managers or App Stores.

I say "were designed" because that's why I think auto-update support should be allowed for other linux distros. Under-the-hood, the auto-updaters are just calling the package managers anyhow.

--

That's a fair reason. I used to have the Github Discussions page but wasn't receiving enough notifications or had the hours to be able to monitor it. I removed it in favor of just having support tickets as Github Issues for traceability.

--

Re:

As in why it's reset to normal without any reason? It works fine without that... Also, only Windows is affected by this "normal" thing iirc.

I haven't the slightest clue as to why that logic exists. The commit is over 7 years old and was only part of a refactor, so the core reason is obfuscated to me. My best guess is that maximum compression causes the update differential to vary more widely, causing (almost) the entire app to be downloaded on each update instead of optimized for differential downloading.

[xyloflake]

I interpreted that electron apps were designed to allow distribution outside of package managers or App Stores.

I say "were designed" because that's why I think auto-update support should be allowed for other linux distros. Under-the-hood, the auto-updaters are just calling the package managers anyhow.

Okay, I got what you mean now! Yup, that definitely makes sense. So I'll be pushing over a PR most probably over weekend. Might be a bit late since I've got school and stuff...

That's a fair reason. I used to have the Github Discussions page but wasn't receiving enough notifications or had the hours to be able to monitor it. I removed it in favor of just having support tickets as Github Issues for traceability.

That's a very good reason imho. However discord also has a forum feature that makes having supports tickets like GitHub issues possible right from discord! Also, I meant to say that there should be a discord server for specifically contributing to electron-builder monorepo as it may need answers to quite a lot of questions people might have while contributing; just like the legacy stuff about the compression we are referring to.

Re:

As in why it's reset to normal without any reason? It works fine without that... Also, only Windows is affected by this "normal" thing iirc.

I haven't the slightest clue as to why that logic exists. The commit is over 7 years old and was only part of a refactor, so the core reason is obfuscated to me. My best guess is that maximum compression causes the update differential to vary more widely, causing (almost) the entire app to be downloaded on each update instead of optimized for differential downloading.

I think that the specific decision of "max compression but entire app is downloaded on autoupdates" should be passed on to the end developer(?). Since this is all legacy stuff, can we run some tests and get a rough idea of if that can be removed or not? If that can be removed, we can get some actual compression on the installers.

Explicit optimization benchmark:
On muffon, before the custom compression type "ultra" was introduced, we used "store". This got us faster builds and no significant size difference between "store" and "maximum". The installers were about 91.3MB in size. After the implementation of "ultra": we now have 64MB installer binaries.

[xyloflake]

My best guess is that maximum compression causes the update differential to vary more widely, causing (almost) the entire app to be downloaded on each update instead of optimized for differential downloading.

I think we should run some real world testing before coming to that conclusion? I don't think the thing about differential builds is correct... The entire app is downloaded anyways (?).

[xyloflake]

@mmaietta how do I test stuff for electron-updater? Do I have to rebuild on CI for every code change?

[xyloflake]

@mmaietta I really tried hard to patch it but I'm having a VERY hard time testing the updates locally. Do you have any tips for testing auto updates locally? I literally have to package the application again and again for literally testing 1 line of code change and then wait for 12 minutes for build and finally figure out that I can't test the auto updates because I get checkForUpdatesAndNotify called, downloadPromise is null when I run the application.

Edit: I found out by reading the documentation -

Note that in order to develop/test UI/UX of updating without packaging the application you need to have a file named dev-app-update.yml in the root of your project, which matches your publish setting from electron-builder config (but in yaml format). But it is not recommended, better to test auto-update for installed application (especially on Windows). Minio is recommended as a local server for testing updates.

[xyloflake]

Also, one more question, how do you determine if the deb, rpm or pacman is to be used?

[xyloflake]

@mmaietta you haven't been replying since days. Are you okay?

[mmaietta]

Been on vacation 🙂

Since this is all legacy stuff, can we run some tests and get a rough idea of if that can be removed or not? If that can be removed, we can get some actual compression on the installers.
I think we should run some real world testing before coming to that conclusion? I don't think the thing about differential builds is correct... The entire app is downloaded anyways (?).

If you'd like to run some tests, please feel free to do so. The entire app is not supposed to be getting downloaded anyways unless the diff is genuinely that significant. There's been optimizations in the newer versions of electron-builder that optimizes the asar file ordering to improve differential updates as well.

how do I test stuff for electron-updater? Do I have to rebuild on CI for every code change?

This is the zsh alias set up on my mac for moving electron-builder/updater changes directly into a local project to avoid the CI/CD flow

alias resync="rsync -upaRv --include='*.js' --include='*.d.ts' --include='*.nsi' --include='*.json' --include='*/' --include='*.py*' --include='*.tiff' --exclude='*'  ~/Development/electron-builder/packages/./* node_modules/"

Call cmd resync from the project directory that is at the same level as electron-builder repo folder.

I literally have to package the application again and again for literally testing 1 line of code change and then wait for 12 minutes for build

For electron-updater code, this is the only route unfortunately. There's no way to test it via unit test due to the need for sudo.

Also, one more question, how do you determine if the deb, rpm or pacman is to be used?

This needs an updater added here:

electron-builder/packages/electron-updater/src/main.ts

Lines 41 to 50 in 553c737

	switch (fileType) {
	case "deb":
	_autoUpdater = new (require("./DebUpdater").DebUpdater)()
	break
	case "rpm":
	_autoUpdater = new (require("./RpmUpdater").RpmUpdater)()
	break
	default:
	break
	}

And your updater target added here:

electron-builder/packages/app-builder-lib/src/targets/FpmTarget.ts

Lines 280 to 282 in 553c737

	private supportsAutoUpdate(target: string) {
	return ["deb", "rpm"].includes(target)
	}