steamos-extension-clean-games hangs at boot on SteamOS 3.9

[Soromeister]

As the title suggests, I have installed only this extension to see what it's about but I am unable to boot anymore.

[Soromeister]

As a matter of fact, it refuses to boot if I only have steamos-extension-loader-installer even.

[sovietotaku]

I can confirm this. The SteamOS 3.9. stops booting if you enable systemd-sysext and install the extensions from this repository.

There is only one solution to the problem right now, and that is to disable the systemd extensions:

1. So, you've messed up the system. Create a SteamOS recovery drive and boot from it. You will also need a physical keyboard connected to Steam Deck!
2. Open the console and run steamos-chroot --disk /dev/nvme0n1 --partset A
3. Then check if you booted into the correct partset. Check that the extensions directory exists. For example, like this: ls -alh /var/lib | grep extensions. If everything is OK, continue. If the directory does not exist, press Ctrl+D on your keyboard and run the command steamos-chroot --disk /dev/nvme0n1 --partset A
4. Run the following command: systemctl disable systemd-sysext
5. Restart Steam Deck, boot from the SSD, log in to the desktop, and open the console. Run in sequence:
   sudo systemctl enable --now systemd-sysext (because without this you will not be able to disable the extensions)
   sudo systemctl disable steamos-extension-loader-installer.service
sudo systemctl disable steamos-extension-loader (just in case)
   sudo systemctl disable systemd-sysext
sudo rm -rf /var/lib/extensions
   Now restart your Deck again.
6. You have now removed the extensions and disabled the systemd extension service. Work normally. Also, do not install systemd extensions until Valve fixes them!
   Optional: Contact Valve technical support to request that they fix this functionality.

[MiningMarsh]

@sovietotaku Do you know if this a problem with sysext in general on SteamOS, or only if you install one of these extensions? I'm going to spend some time this weekend debugging to see if it's something in the extensions causing the problem, but if this is an issue with sysext I'm not sure how I should move forward.

[MiningMarsh]

@Soromeister
In addition to @sovietotaku's excellent instructions, you can remove the file placed at /etc/atomic-update.d/steamos-extension-loader.conf.

This is the configuration file that tells SteamOS not to wipe the installer services when updating. If you remove it and update, you should more-or-less guarantee a clean install again.

This is the second time a SteamOS update has broken these extensions. I'm going to spend some time exploring ways to make uninstallation easier, so users are not stranded again trying to repair their install. This might be difficult due to how extensions are designed; I'm not sure how long it'll take or if I can find a suitable method.

One immediate solution that comes to mind is a flashable live USB that does nothing but disable extensions (you could boot this off an SD card). That's fairly inelegant, though.

[MiningMarsh]

I have added a warning to the README until this is resolved.

[MiningMarsh]

@Soromeister @sovietotaku

Last night I tested both the latest stable and beta branches on my deck OLED and both seemed functional. I could not test preview, as even after I removed all the extension files, for some reason my deck did not want to switch to the preview branch. It would reboot, everything would work fine, and then it would just still be on the beta branch for some reason.

On the beta branch, my deck says the OS is at version 3.7.20. I'm afraid I might not be able to replicate this issue for testing.

I generally stay on the stable branch and test everything against it, so I don't see these issues with the beta/previews as quickly as users do.

[bentemple]

So, I updated to the preview channel, and just like you @MiningMarsh the first time it just bumped to beta. I then tried again ( from beta ) and it let me update to the preview track. That being said, I only have 3.8.1 available, but I updated to it, and the steamos-extensions all seem to be working just fine for me on this version.

So 2 questions would be:

1. Is 3.9 a typo and it should be 3.8? because 3.8 isn't even released yet.
2. Is this bug still a problem on the latest 3.8.1 for you all? if not, it seems likely to me that it was a bug on Valve's side they fixed. (and in general it still feels that way, since this is just leveraging a built-in systemd system.)

I do like the idea of making it easier to remove the entire systemd-installer service. I'm considering making a decky plugin to manage all of these extensions, but a "bricked" OS because of an OS bug is maybe less something I would target fixing with that uninstaller for the first pass.

[bentemple]

I wonder if how broad the atomic config is is what is causing the issues here:

/etc/systemd/system/**
/etc/systemd/user/**
/etc/steamos-extension-loader

Feels like we could probably just reduce this down to:

/etc/atomic-update.conf.d/steamos-extension-loader.conf
/etc/systemd/system/steamos-extension-loader.service
/etc/systemd/system/sysinit.target.wants/steamos-extension-loader.service
/etc/systemd/system/multi-user.target.wants/steamos-extension-loader-installer.service
/etc/systemd/system/timers.target.wants/steamos-extension-loader-system-cleaner.timer
/etc/steamos-extension-loader

And optionally extensions could maybe define their own units to be kept by adding or removing them from this config dynamically?

Not retaining them should, at worst, mean that the user would need to reboot a second time after a system update to get them back to enabled. (if they don't add to this list)

[MiningMarsh]

I wonder if how broad the atomic config is is what is causing the issues here:

/etc/systemd/system/**
/etc/systemd/user/**
/etc/steamos-extension-loader

Feels like we could probably just reduce this down to:

/etc/atomic-update.conf.d/steamos-extension-loader.conf
/etc/systemd/system/steamos-extension-loader.service
/etc/systemd/system/sysinit.target.wants/steamos-extension-loader.service
/etc/systemd/system/multi-user.target.wants/steamos-extension-loader-installer.service
/etc/systemd/system/timers.target.wants/steamos-extension-loader-system-cleaner.timer
/etc/steamos-extension-loader

And optionally extensions could maybe define their own units to be kept by adding or removing them from this config dynamically?

Not retaining them should, at worst, mean that the user would need to reboot a second time after a system update to get them back to enabled. (if they don't add to this list)

This certainly could be the culprit. The loader both enables and starts all units based on the preset files, the way it is setup wouldn't even require an extra reboot I don't think. It would just mean the units would always be re-added after the update, so they shouldn't even need their own atomic file.

I think this would be a safe change to make.

[bentemple]

I'll make some PRs. I've been meaning to bring in my GHA and such as well anyway