Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Foreman/Satellite el8toel9 upgrade #1181

Merged
merged 4 commits into from
May 2, 2024
Merged

Foreman/Satellite el8toel9 upgrade #1181

merged 4 commits into from
May 2, 2024

Conversation

evgeni
Copy link
Member

@evgeni evgeni commented Feb 19, 2024 •
edited
Loading

  • move common Satellite upgrade code to common so it can be used by 7to8 and 8to9 upgrades
  • add 8to9 specific code to allow Foreman/Satellite upgrades from EL8 to EL9
  • refresh collation version if pulp-ansible (and its custom collation) is present
  • refactor satellite_upgrade_services to use SystemdServicesTasks

@github-actions GitHub Actions
Copy link

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

  • review please @oamg/developers to notify leapp developers of the review request
  • /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build. If you need a different version of leapp from PR#42, use /packit test oamg/leapp#42

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

  • /packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
  • /packit test --labels beaker-minimal-8.9to9.3,kernel-rt-8.9to9.3 to schedule kernel-rt and beaker-minimal test sets for 8.9->9.3 upgrade path

[Deprecated] To launch on-demand regression testing public members of oamg organization can leave the following comment:

  • /rerun to schedule basic regression tests using this pr build and latest upstream leapp build as artifacts
  • /rerun 42 to schedule basic regression tests using this pr build and leapp*PR42* as artifacts
  • /rerun-sst to schedule sst tests using this pr build and latest upstream leapp build as artifacts
  • /rerun-sst 42 to schedule sst tests using this pr build and leapp*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

@evgeni evgeni force-pushed the foreman-el9 branch 19 times, most recently from c145a77 to 67133b5 Compare February 22, 2024 08:05
@evgeni evgeni marked this pull request as ready for review February 22, 2024 08:12
@evgeni evgeni requested a review from a team February 22, 2024 08:12
fernflower
fernflower reviewed Mar 1, 2024
View reviewed changes
Copy link
Member

@fernflower fernflower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that this is not a ready to go pr (at least given the fact SATELLITE_VERSIONs are not set in stone yet). I will move it to draft so that no one on a merging spree gets the wrong idea :)

@fernflower fernflower marked this pull request as draft March 1, 2024 12:35
@evgeni
Copy link
Member Author

evgeni commented Mar 1, 2024

I see that this is not a ready to go pr (at least given the fact SATELLITE_VERSIONs are not set in stone yet). I will move it to draft so that no one on a merging spree gets the wrong idea :)

While not ready for Satellite consumption, this can today upgrade upstream Foreman just fine, so I'd be interested to get it in as-is, if you'd be OK with that.

@evgeni
Copy link
Member Author

evgeni commented Apr 26, 2024

No worries, rebases are cheap :)

@evgeni evgeni force-pushed the foreman-el9 branch 5 times, most recently from cd7f1bb to d074f3c Compare April 26, 2024 08:12
@lpramuk
Copy link

lpramuk commented May 1, 2024

I have successfully tested this PR for both Satellite and Capsule using:
leapp-0.17.0-100.20240430083919859552.master.3.gb21450a.el8
leapp-repository-0.20.0-0.20240426090422205877.pr1181.30.gc00d93a3.el8

@pirat89 pirat89 dismissed their stale review May 2, 2024 07:24

dropping the blocker based on above comments. thanks!

fernflower
fernflower previously approved these changes May 2, 2024
View reviewed changes
Copy link
Member

@fernflower fernflower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@fernflower
Copy link
Member

@evgeni Could you please squash commits into logical blocks (should not be one commit) and add description to each of those squashed commits' messages?
Happy to merge it afterwards.

@evgeni
Copy link
Member Author

evgeni commented May 2, 2024

Will do.
Also just realized new PES data was updated, so I'll check if we still need the workaround for RHEL-32759 (my expectation is not)

evgeni added 2 commits May 2, 2024 12:12
@evgeni
Move common Satellite Upgrade code to "common"
ac6777f 
This allows the re-use of the code in the el8toel9 upgrade.
@evgeni
Add el8toel9 upgrade facts for Satellite
73c12d9 
This adds the el8toel9 specific fact scanner/generator for Satellite
upgrades. The result of this actor is what drives the actual upgrade
actions.
evgeni added 2 commits May 2, 2024 12:39
@evgeni
Refresh collation version if pulp-ansible is present
ecc1bb7 
When migrating to a new OS, we REINDEX all databases.
pulp_ansible ships with an own collation (using ICU), which needs a
version refresh after the REINDEX has been done.
@evgeni
Refactor satellite_upgrade_services to use SystemdServicesTasks
b6d09c1 
We used to just delete the symlinks in /etc/systemd, but with the new
systemd actors this doesn't work anymore as they will restore the
pre-delete state because they by default aim at having source and
target systems match in terms of services. By using SystemdServicesTasks
we can explicitly turn those services off and inform all interested
parties about this.
@evgeni
Copy link
Member Author

evgeni commented May 2, 2024

It's not needed \o/

fernflower
fernflower approved these changes May 2, 2024
View reviewed changes
Copy link
Member

@fernflower fernflower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@fernflower fernflower merged commit bad2fb2 into oamg:master May 2, 2024
15 checks passed
@evgeni evgeni deleted the foreman-el9 branch May 2, 2024 11:50
@evgeni
Copy link
Member Author

evgeni commented May 2, 2024

Thanks everyone!

@pirat89 pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label May 2, 2024
pirat89 added a commit to pirat89/leapp-repository that referenced this pull request Aug 16, 2024
@pirat89
# Release 0.21.0
3ff50a5 
## Packaging
- .. names of packages, dependencies, changes in provided capabilities....

## Upgrade handling
### Fixes
- Add missing RHUI GCP config info for RHEL for SAP (oamg#1253)
- Fix creation of the post upgrade report about changes in states of systemd services (oamg#1210)
- Fix detection of valid sshd config with internal-sftp subsystem in Leapp (oamg#1212)
- Fix evaluation of PES data (oamg#1194)
- Fix failing "update-ca-trust" command caused by missing util-linux package (oamg#1169)
- Fix handling of versions in RHUI configuration for ELS and SAP upgrades (oamg#1240)
- Fix the parsing of the lscpu output (oamg#1184, oamg#1208)
- Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (oamg#1178)
- Fix upgrade on aarch64 via RHUI on AWS (oamg#1240)
- Handle a false positive GPG check error when TargetUserSpaceInfo is missing  (oamg#1269)
- Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (oamg#1205)
- Update the default kernel cmdline (oamg#1193, oamg#1216)
- Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (oamg#1211)
- Wait for the storage initialization when /usr is on separate file system - covering SAN (oamg#1218, oamg#1219)
- [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (oamg#1243)
- [IPU 7 -> 8] Fix detection of bootable device on RAID (oamg#1260)
- [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (oamg#1270)

### Enhancements
- [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (oamg#1245, oamg#1246)
- Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (oamg#1199)
- Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (oamg#1197)
- Improve set_systemd_services_states logging (oamg#1213)
- Include leapp command execution and defined leapp envars inside leapp.db - (oamg#1152)
- Introduce experimental upgrades in 'live' mode for the testing (oamg#1248)
- Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (oamg#1241)
- Several minor improvements in messages printed in console output (oamg#1173, oamg#1214, oamg#1274)
- Several minor improvements in report and error messages (oamg#1207, oamg#1217, oamg#1234, oamg#1235, oamg#1242)
- Sort lists in dnf-plugin-data for easier overview (oamg#1231)
- [IPU 7 -> 8] Allow upgrade of content from ELS repositories (oamg#1198)
- [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (oamg#1206)
- [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (oamg#1195)
- [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (oamg#1249)
- [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (oamg#1181)
- [IPU 9 -> 10] Introduced number of changes to enable experimental IPU 9 -> 10 (oamg#1169)
- [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (oamg#1268)
- [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (oamg#1169, oamg#1273)

## Additional changes interesting for devels
- drop unused `packager` field from gpg-signatures.json (oamg#1233)
- [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible
- [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo
@pirat89 pirat89 mentioned this pull request Aug 16, 2024
Merged
pirat89 added a commit that referenced this pull request Aug 16, 2024
@pirat89
# Release 0.21.0 (#1282)
03c257b 
## Packaging
- Start building for EL 9 in the upstream repository on COPR (#1169)

## Upgrade handling
### Fixes
- Add missing RHUI GCP config info for RHEL for SAP (#1253)
- Fix creation of the post upgrade report about changes in states of systemd services (#1210)
- Fix detection of valid sshd config with internal-sftp subsystem in Leapp (#1212)
- Fix evaluation of PES data (#1194)
- Fix failing "update-ca-trust" command caused by missing util-linux package (#1169)
- Fix handling of versions in RHUI configuration for ELS and SAP upgrades (#1240)
- Fix the parsing of the lscpu output (#1184, #1208)
- Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (#1178)
- Fix upgrade on aarch64 via RHUI on AWS (#1240)
- Handle a false positive GPG check error when TargetUserSpaceInfo is missing  (#1269)
- Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (#1205)
- Update the default kernel cmdline (#1193, #1216)
- Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (#1211)
- Wait for the storage initialization when /usr is on separate file system - covering SAN (#1218, #1219)
- [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (#1243)
- [IPU 7 -> 8] Fix detection of bootable device on RAID (#1260)
- [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (#1270)

### Enhancements
- [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (#1245, #1246)
- Update leapp data files (#1280)
- Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (#1199)
- Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (#1197)
- Improve set_systemd_services_states logging (#1213)
- Include leapp command execution and defined leapp envars inside leapp.db - (#1152)
- Introduce experimental upgrades in 'live' mode for the testing (#1248)
- Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (#1241)
- Several minor improvements in messages printed in console output (#1173, #1214, #1274)
- Several minor improvements in report and error messages (#1207, #1217, #1234, #1235, #1242)
- Sort lists in dnf-plugin-data for easier overview (#1231)
- [IPU 7 -> 8] Allow upgrade of content from ELS repositories (#1198)
- [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (#1206)
- [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (#1195)
- [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (#1249)
- [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (#1181)
- [IPU 9 -> 10] Introduced number of changes to enable IPU 9 -> 10 for testing (#1169)
- [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (#1268)
- [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (#1169, #1273)

## Additional changes interesting for devels
- drop unused `packager` field from gpg-signatures.json (#1233)
- [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible
- [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo
yuravk pushed a commit to yuravk/leapp-repository that referenced this pull request Aug 20, 2024
@pirat89 @yuravk
# Release 0.21.0 (oamg#1282)
3eb4ade 
## Packaging
- Start building for EL 9 in the upstream repository on COPR (oamg#1169)

## Upgrade handling
### Fixes
- Add missing RHUI GCP config info for RHEL for SAP (oamg#1253)
- Fix creation of the post upgrade report about changes in states of systemd services (oamg#1210)
- Fix detection of valid sshd config with internal-sftp subsystem in Leapp (oamg#1212)
- Fix evaluation of PES data (oamg#1194)
- Fix failing "update-ca-trust" command caused by missing util-linux package (oamg#1169)
- Fix handling of versions in RHUI configuration for ELS and SAP upgrades (oamg#1240)
- Fix the parsing of the lscpu output (oamg#1184, oamg#1208)
- Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (oamg#1178)
- Fix upgrade on aarch64 via RHUI on AWS (oamg#1240)
- Handle a false positive GPG check error when TargetUserSpaceInfo is missing  (oamg#1269)
- Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (oamg#1205)
- Update the default kernel cmdline (oamg#1193, oamg#1216)
- Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (oamg#1211)
- Wait for the storage initialization when /usr is on separate file system - covering SAN (oamg#1218, oamg#1219)
- [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (oamg#1243)
- [IPU 7 -> 8] Fix detection of bootable device on RAID (oamg#1260)
- [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (oamg#1270)

### Enhancements
- [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (oamg#1245, oamg#1246)
- Update leapp data files (oamg#1280)
- Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (oamg#1199)
- Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (oamg#1197)
- Improve set_systemd_services_states logging (oamg#1213)
- Include leapp command execution and defined leapp envars inside leapp.db - (oamg#1152)
- Introduce experimental upgrades in 'live' mode for the testing (oamg#1248)
- Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (oamg#1241)
- Several minor improvements in messages printed in console output (oamg#1173, oamg#1214, oamg#1274)
- Several minor improvements in report and error messages (oamg#1207, oamg#1217, oamg#1234, oamg#1235, oamg#1242)
- Sort lists in dnf-plugin-data for easier overview (oamg#1231)
- [IPU 7 -> 8] Allow upgrade of content from ELS repositories (oamg#1198)
- [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (oamg#1206)
- [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (oamg#1195)
- [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (oamg#1249)
- [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (oamg#1181)
- [IPU 9 -> 10] Introduced number of changes to enable IPU 9 -> 10 for testing (oamg#1169)
- [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (oamg#1268)
- [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (oamg#1169, oamg#1273)

## Additional changes interesting for devels
- drop unused `packager` field from gpg-signatures.json (oamg#1233)
- [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible
- [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo

(cherry picked from commit 03c257b)
yuravk pushed a commit to yuravk/leapp-repository that referenced this pull request Aug 20, 2024
@pirat89 @yuravk
# Release 0.21.0 (oamg#1282)
3ef49b8 
## Packaging
- Start building for EL 9 in the upstream repository on COPR (oamg#1169)

## Upgrade handling
### Fixes
- Add missing RHUI GCP config info for RHEL for SAP (oamg#1253)
- Fix creation of the post upgrade report about changes in states of systemd services (oamg#1210)
- Fix detection of valid sshd config with internal-sftp subsystem in Leapp (oamg#1212)
- Fix evaluation of PES data (oamg#1194)
- Fix failing "update-ca-trust" command caused by missing util-linux package (oamg#1169)
- Fix handling of versions in RHUI configuration for ELS and SAP upgrades (oamg#1240)
- Fix the parsing of the lscpu output (oamg#1184, oamg#1208)
- Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (oamg#1178)
- Fix upgrade on aarch64 via RHUI on AWS (oamg#1240)
- Handle a false positive GPG check error when TargetUserSpaceInfo is missing  (oamg#1269)
- Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (oamg#1205)
- Update the default kernel cmdline (oamg#1193, oamg#1216)
- Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (oamg#1211)
- Wait for the storage initialization when /usr is on separate file system - covering SAN (oamg#1218, oamg#1219)
- [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (oamg#1243)
- [IPU 7 -> 8] Fix detection of bootable device on RAID (oamg#1260)
- [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (oamg#1270)

### Enhancements
- [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (oamg#1245, oamg#1246)
- Update leapp data files (oamg#1280)
- Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (oamg#1199)
- Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (oamg#1197)
- Improve set_systemd_services_states logging (oamg#1213)
- Include leapp command execution and defined leapp envars inside leapp.db - (oamg#1152)
- Introduce experimental upgrades in 'live' mode for the testing (oamg#1248)
- Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (oamg#1241)
- Several minor improvements in messages printed in console output (oamg#1173, oamg#1214, oamg#1274)
- Several minor improvements in report and error messages (oamg#1207, oamg#1217, oamg#1234, oamg#1235, oamg#1242)
- Sort lists in dnf-plugin-data for easier overview (oamg#1231)
- [IPU 7 -> 8] Allow upgrade of content from ELS repositories (oamg#1198)
- [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (oamg#1206)
- [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (oamg#1195)
- [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (oamg#1249)
- [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (oamg#1181)
- [IPU 9 -> 10] Introduced number of changes to enable IPU 9 -> 10 for testing (oamg#1169)
- [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (oamg#1268)
- [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (oamg#1169, oamg#1273)

## Additional changes interesting for devels
- drop unused `packager` field from gpg-signatures.json (oamg#1233)
- [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible
- [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo

(cherry picked from commit 03c257b)
@yuravk yuravk mentioned this pull request Aug 22, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abadger abadger abadger left review comments

@fernflower fernflower fernflower approved these changes

@pirat89 pirat89 pirat89 left review comments

Assignees
No one assigned
Labels
changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant enhancement New feature or request
Projects
None yet
Milestone
8.10/9.5
Development

Successfully merging this pull request may close these issues.
6 participants
@evgeni @fernflower @mkluson @pirat89 @lpramuk @abadger