-
-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Contribution] qvm-upgrade-template
(easy in-place upgrades for Debian and Fedora templates)
#8605
Comments
qvm-upgrade-template
(easy in-place upgrades for Debian and Fedora templates)
I'm willing to help further (including documentation and improving the code), but am unclear how to enlist a reviewer/mentor. |
We talked about this during the summit. @marmarta is the go-to person for this and she has some ideas on how to go about it. If I recall correctly the first goal is to expose the fact that there are template upgrades available in the updater and the second part would be to actually make it possible for the user to inplace upgrade them. Exposing it via the UI is even better than a cli utility, but it'll certainly need a backend. And a I think issue can be merged into #6904. |
I'm not sure if they're really the same. #6904 seems to be about trying to help the user migrate from one customized template to another, whereas this is about scripting the in-place upgrade steps for templates. They could end up achieving a result that is close enough to count as effectively the same for most users, but there are subtle differences. Also, this is an issue for a specific package contribution. (At least, that's what appears to be so far.) |
Well, it depends on how the developers decide to handle things. If this truly ends up being a contrib package, then I'm not sure whether the general GUI can be based on it, since contrib packages are optional and not installed by default. (Users have to opt-in.) Also, it remains to be seen whether the devs even agree with this approach as a long-term solution.
It would be convenient, but it's not strictly necessary. We can just add the
S: blocked
|
Oh I love this! I have recently made a PR to expose "hey! this template is stale!!!" information in the GUI (in the updater widget); it would be absolutely amazing to have those script as a tool to use in the Template Manager tool. I think a qubes-template-upgrade script is a way to go, sure - I can put GUI wrappers around it, but a working CLI script is 1. really wanted by a lot of people 2. easier to wrap in GUI than GUI is to wrap around in CLI :D |
The GUI tool will want machine-readable output, including progress information, so that it can display what is going on and has happened without complex parsing. In particular, the GUI tool is likely to be run in dom0, so it needs to treat the script output as untrusted. |
@DemiMarie @marmarta I can add JSON outputs and a command line option for that output, if that'd speed things along. Do you think JSON would be suitable, or do you have another format in mind? I've also quickly added a "help" option when no arguments are provided to the scripts. Open to further suggestions and any additional features |
JSON isn't really suitable for progress information (it's just a single object, not stream of updates). Progress could be reported by simply printing BTW few comments to the scripts: the Fedora script creates cache block device, but doesn't use it; the Debian script opens gnome-terminal (to start the template?) but also doesn't use it. And also, avoid using |
Implemented per @marmarek's suggestion [1]. This wouldn't have worked on minimal templates (or if somebody removed qubes-core-agent-passwordless-root package). [1]: QubesOS/qubes-issues#8605 (comment)
Since this script is non-interactive, there's no need to start a gnome-terminal for the user to type commands. Implemented per Marek's suggestion [1]. This was replaced with `qvm-start`. [1]: QubesOS/qubes-issues#8605 (comment)
There was a missing option to actually use the cache. This was pointed out by Marek in [1]. [1]: QubesOS/qubes-issues#8605 (comment)
I took peek at the scripts and applied those improvements. Let's get this nice addition rolling 💪 |
Have updated the codes to correct this issue: #8725 (comment) |
May be worth merging these or adding as an option for future major releases: https://github.com/QubesOS/qubes-dist-upgrade/blob/main/scripts/upgrade-template-standalone.sh |
per discussions with @deeplow #8605 (comment) and @marmarek #8725 I've overhauled the scripts, merged into one script for use with both fedora- and debian-based templates and completed testing. The new repo and script can be found here https://github.com/kennethrrosen/qubes-template-upgrade/blob/07cd12f419f4715170aed3d4198683fabad4700a/qvm-template-upgrade |
You are still using On the top of the script, use New templates have the version set in their feature For the same reason, template type should not be provided by the user, but gathered via the qube feature User may have more than the default repositories, they need to be upgraded also, glob the |
Thanks @ben-grande ; and for all your work on "qusal." I've incorporated some of those changes https://github.com/kennethrrosen/qubes-template-upgrade/blob/main/qvm-template-upgrade but I'm not clear on how to alter the script to gather the |
Happy to see some movement here :)
I'm guessing this is via qubes-features. Something like:
|
Then could the potential answer be have the user put their desired template name in, have the script decide the next logical distribution (based on |
Thanks!
qvm-run has not
Does not seem like a good idea to detect based on template name, many users have templates without the named mentioning |
I think we're on the same page. I had attempted to account for the various template names one might have. Your solution is more robust. I've updated the script to correct then |
I've tested the script on several templates and various naming conventions; further testers would be appreciated. But the script works as expected. Thanks again, @ben-grande for your review and pointers, and @deeplow for your gut-checks. Debian templates before I'd eventually hope this gets worked into a gui, but for now I've used Latest version is in the main branch: https://github.com/kennethrrosen/qubes-template-upgrade/tree/main |
Still doesn't support Debian Debian supports both |
Good catch. Reverted. |
I've also created a small GUI following QubesOS style to accompany the script for those who don't want to use the shell: https://github.com/kennethrrosen/qubes-template-upgrade/blob/main/qubes-template-upgrade-gui |
Per contributor guidelines, I've also converted the script to Python. |
|
I was taking a look at this the other day and I think its architecture pretty much fits into what we're trying to achieve. We could:
Advantages with this approach are many:
Later steps could include the integration with the GUI updater, but that is out of scope for the ticket. But it could make use of the |
Without the ingenious gumption and go-getter attitude of @kennethrrosen, we wouldn't be here today. However, I decided to do some exploration on the feasibility of doing this instead via vmupdate (what the GUI updater uses), as Marek suggested. This branch has a very simple PoC that adds a new After doing that initial work, I can confirm that ultimately this vmupdate is probably what we want to use. Here are a few reasons:
I though this would be a pretty trivial (re)implementation done in a weekend. However, after spending a significant part of today attempting to add dnf_api support, I didn't get anywhere close to a fedora dist upgrade, since the API doesn't map well at all to the respective commands (kudos to @piotrbartman for the work that's already there!). So I guess I may need some help to get the APT / DNF APIs working. What's missing?
I'm happy to spend a few more days working on this over the next month or so, if this feels like a worthy pursuit. But I do not want to suck out the contribution energy out going on here. And I know doing with vmupdate requires a lot more setup (qubes-builderv2) and is significantly more complex (the code changes will be less, probably, but the code to understand is a lot more). Would folks be OK with this alternative or should we explore both in parallel and see which one is the most viable / suitable? |
This project in its current form could be a beneficial stopgap for users intimidated by the user-led upgrade-in-place procedure. So I'm happy to maintain the package as a contrib in testing while also working with @deeplow to work this into vmupdate for a much-later release. We could also poll forum members as to whether the additional upgrader and GUI would be welcome in the meanwhile as we explore the more integrated approach. |
How to file a helpful issue
The problem you're addressing (if any)
Per official documentation, it is recommended that users download latest templates through
qvm-template
before "switching" everything from a pre-existing template to the newer template.qvm-prefs
allows for a somewhat seamless transition of preferences. But inheritance and persistence is a bit more complicated for the non-technical user.This makes upgrading-in-place a very attractive option for new and experienced users alike. Fedrora documentation and Debian documentation suggest this process is for the "advanced" user. This should not be the case.
The solution you'd like
Include as a qubes command the option to "upgrade-in-place" a template, perhaps
qvm-upgrade-debian
andqvm-upgrade-fedora
. I've provided basic Bash scripts for doing so, with an option for upgrading multiple templates of the same distro at once:For Fedora-based templates.
For Debian-based templates.
At present, both scripts were tested successfully for upgrading Fedora-37 to Fedora-38 and Debian-11 to Debian-12. These scripts might also benefit from
qvm-clone
commands to create .bak templates and a help/usage function.The value to a user, and who that user might be
Such scripts would lower the barrier-to-entry for non-technical users interested in adopting Qubes. Furthermore, it would lower the amount of time spent on upgrading individual templates a user may have created based on the defaults, becoming a value-add. It would also limit bandwidth on current Qubes documentation.
The text was updated successfully, but these errors were encountered: