Fix detection when a bootable partition in on RAID #1260

mscherer · 2024-07-03T12:39:52Z

On my server, leapp preupgrade fail with the following error:

Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/site-packages/leapp/repository/actor_definition.py", line 74, in _do_run
    actor_instance.run(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/leapp/actors/__init__.py", line 289, in run
    self.process(*args)
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scangrubdevpartitionlayout/actor.py", line 18, in process
    scan_layout_lib.scan_grub_device_partition_layout()
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scangrubdevpartitionlayout/libraries/scan_layout.py", line 91, in scan_grub_device_partition_layout
    dev_info = get_partition_layout(device)
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scangrubdevpartitionlayout/libraries/scan_layout.py", line 79, in get_partition_layout
    part_start = int(part_info[2]) if len(part_info) == len(part_all_attrs) else int(part_info[1])
ValueError: invalid literal for int() with base 10: '*'

This is caused by the following line:

/dev/sda1   *        2048     1026047      512000   fd  Linux raid autodetect

I have my server on EL7 with / using a Linux RAID so len(part_info) != len(part_all_attrs), hence why it try to convert '*' to int.

github-actions · 2024-07-03T12:40:04Z

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

review please @oamg/developers to notify leapp developers of the review request
/packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
However, here are additional useful commands for packit:

/packit test to re-run manually the default tests
/packit retest-failed to re-run failed tests manually
/packit test oamg/leapp#42 to run tests with leapp builds for the leapp PR#42 (default is latest upstream - master - build)

Note that first time contributors cannot run tests automatically - they need to be started by a reviewer.

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

/packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
/packit test --labels beaker-minimal-8.10to9.4,kernel-rt-8.10to9.4 to schedule kernel-rt and beaker-minimal test sets for 8.10->9.4 upgrade path

See other labels for particular jobs defined in the .packit.yaml file.

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

fernflower · 2024-07-03T12:43:16Z

Could you please add a unit test?

mscherer · 2024-07-03T12:54:14Z

I modified the existing one to deal with that case, so do you prefer a separate test just for that ?

mscherer · 2024-07-03T13:05:06Z

I found a not too ugly way to write the test (eg, not cut and paste), hope that work.

...ystem_upgrade/el7toel8/actors/scangrubdevpartitionlayout/tests/test_scan_partition_layout.py

pirat89 · 2024-07-03T13:45:09Z

@mscherer unit test is now reproducing original issue. so it seems it's not fixed :) btw, in case you would like to test locally, you can try (in the main dir) TEST_CONTAINER=rhel7 make test_container or you can use test_container_no_lint skip linters. so you do not need to wait for us to retrigger unit-tests execution every time. see make help for more if interested.

mscherer · 2024-07-03T14:09:02Z

I am trying to find a VM to run the test without bothering you too much (it is just full of small papercuts)

mscherer · 2024-07-03T15:00:22Z

And now I managed to get a VM, it seems to run fine.

pirat89 · 2024-07-04T09:37:26Z

/packit copr-build

fernflower · 2024-07-04T11:48:51Z

Thanks for the unit test!

mscherer · 2024-07-09T13:13:10Z

Is there something waiting for me ? From a quick look at the CI, the 3 tests fail for different reasons and they do not look like related to the patch itself (unless I am mistaken)

pirat89 · 2024-07-10T21:10:23Z

Hi @mscherer , actually it's now problem on our side. Most likely we will ping you during the next week, asking for rebase when tests are fixed, so we can see that there is not introduced another regression. the code looks good to me, just want to be sure that it will be final solution and we will not need to investigate similar issue again soon. And we are also little bit busy last week.

Downstream ticket with basically same issue has been reported today:

https://issues.redhat.com/browse/RHEL-47038

Thank you for the contribution!!

pirat89 · 2024-07-16T22:38:08Z

Hi @mscherer, the test execution in master branch should be fixed now. Can you please rebase your branch?

On my server, leapp preupgrade fail with the following error: Traceback (most recent call last): File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python2.7/site-packages/leapp/repository/actor_definition.py", line 74, in _do_run actor_instance.run(*args, **kwargs) File "/usr/lib/python2.7/site-packages/leapp/actors/__init__.py", line 289, in run self.process(*args) File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scangrubdevpartitionlayout/actor.py", line 18, in process scan_layout_lib.scan_grub_device_partition_layout() File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scangrubdevpartitionlayout/libraries/scan_layout.py", line 91, in scan_grub_device_partition_layout dev_info = get_partition_layout(device) File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scangrubdevpartitionlayout/libraries/scan_layout.py", line 79, in get_partition_layout part_start = int(part_info[2]) if len(part_info) == len(part_all_attrs) else int(part_info[1]) ValueError: invalid literal for int() with base 10: '*' This is caused by the following line: /dev/sda1 * 2048 1026047 512000 fd Linux raid autodetect I have my server on EL7 with / using a Linux RAID so len(part_info) != len(part_all_attrs), hence why it try to convert '*' to int.

mscherer · 2024-07-17T19:44:48Z

@pirat89 done

pirat89 · 2024-07-18T07:57:06Z

/packit copr-build

pirat89

lgtm and all tests passed. Thank you for your contribution!

## Packaging - .. names of packages, dependencies, changes in provided capabilities.... ## Upgrade handling ### Fixes - Add missing RHUI GCP config info for RHEL for SAP (oamg#1253) - Fix creation of the post upgrade report about changes in states of systemd services (oamg#1210) - Fix detection of valid sshd config with internal-sftp subsystem in Leapp (oamg#1212) - Fix evaluation of PES data (oamg#1194) - Fix failing "update-ca-trust" command caused by missing util-linux package (oamg#1169) - Fix handling of versions in RHUI configuration for ELS and SAP upgrades (oamg#1240) - Fix the parsing of the lscpu output (oamg#1184, oamg#1208) - Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (oamg#1178) - Fix upgrade on aarch64 via RHUI on AWS (oamg#1240) - Handle a false positive GPG check error when TargetUserSpaceInfo is missing (oamg#1269) - Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (oamg#1205) - Update the default kernel cmdline (oamg#1193, oamg#1216) - Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (oamg#1211) - Wait for the storage initialization when /usr is on separate file system - covering SAN (oamg#1218, oamg#1219) - [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (oamg#1243) - [IPU 7 -> 8] Fix detection of bootable device on RAID (oamg#1260) - [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (oamg#1270) ### Enhancements - [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (oamg#1245, oamg#1246) - Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (oamg#1199) - Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (oamg#1197) - Improve set_systemd_services_states logging (oamg#1213) - Include leapp command execution and defined leapp envars inside leapp.db - (oamg#1152) - Introduce experimental upgrades in 'live' mode for the testing (oamg#1248) - Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (oamg#1241) - Several minor improvements in messages printed in console output (oamg#1173, oamg#1214, oamg#1274) - Several minor improvements in report and error messages (oamg#1207, oamg#1217, oamg#1234, oamg#1235, oamg#1242) - Sort lists in dnf-plugin-data for easier overview (oamg#1231) - [IPU 7 -> 8] Allow upgrade of content from ELS repositories (oamg#1198) - [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (oamg#1206) - [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (oamg#1195) - [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (oamg#1249) - [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (oamg#1181) - [IPU 9 -> 10] Introduced number of changes to enable experimental IPU 9 -> 10 (oamg#1169) - [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (oamg#1268) - [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (oamg#1169, oamg#1273) ## Additional changes interesting for devels - drop unused `packager` field from gpg-signatures.json (oamg#1233) - [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible - [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo

## Packaging - Start building for EL 9 in the upstream repository on COPR (#1169) ## Upgrade handling ### Fixes - Add missing RHUI GCP config info for RHEL for SAP (#1253) - Fix creation of the post upgrade report about changes in states of systemd services (#1210) - Fix detection of valid sshd config with internal-sftp subsystem in Leapp (#1212) - Fix evaluation of PES data (#1194) - Fix failing "update-ca-trust" command caused by missing util-linux package (#1169) - Fix handling of versions in RHUI configuration for ELS and SAP upgrades (#1240) - Fix the parsing of the lscpu output (#1184, #1208) - Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (#1178) - Fix upgrade on aarch64 via RHUI on AWS (#1240) - Handle a false positive GPG check error when TargetUserSpaceInfo is missing (#1269) - Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (#1205) - Update the default kernel cmdline (#1193, #1216) - Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (#1211) - Wait for the storage initialization when /usr is on separate file system - covering SAN (#1218, #1219) - [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (#1243) - [IPU 7 -> 8] Fix detection of bootable device on RAID (#1260) - [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (#1270) ### Enhancements - [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (#1245, #1246) - Update leapp data files (#1280) - Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (#1199) - Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (#1197) - Improve set_systemd_services_states logging (#1213) - Include leapp command execution and defined leapp envars inside leapp.db - (#1152) - Introduce experimental upgrades in 'live' mode for the testing (#1248) - Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (#1241) - Several minor improvements in messages printed in console output (#1173, #1214, #1274) - Several minor improvements in report and error messages (#1207, #1217, #1234, #1235, #1242) - Sort lists in dnf-plugin-data for easier overview (#1231) - [IPU 7 -> 8] Allow upgrade of content from ELS repositories (#1198) - [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (#1206) - [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (#1195) - [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (#1249) - [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (#1181) - [IPU 9 -> 10] Introduced number of changes to enable IPU 9 -> 10 for testing (#1169) - [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (#1268) - [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (#1169, #1273) ## Additional changes interesting for devels - drop unused `packager` field from gpg-signatures.json (#1233) - [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible - [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo

## Packaging - Start building for EL 9 in the upstream repository on COPR (oamg#1169) ## Upgrade handling ### Fixes - Add missing RHUI GCP config info for RHEL for SAP (oamg#1253) - Fix creation of the post upgrade report about changes in states of systemd services (oamg#1210) - Fix detection of valid sshd config with internal-sftp subsystem in Leapp (oamg#1212) - Fix evaluation of PES data (oamg#1194) - Fix failing "update-ca-trust" command caused by missing util-linux package (oamg#1169) - Fix handling of versions in RHUI configuration for ELS and SAP upgrades (oamg#1240) - Fix the parsing of the lscpu output (oamg#1184, oamg#1208) - Fix the upgrade of systems using RHUI on AWS after changes in RHUI client package (oamg#1178) - Fix upgrade on aarch64 via RHUI on AWS (oamg#1240) - Handle a false positive GPG check error when TargetUserSpaceInfo is missing (oamg#1269) - Target by default always "GA" channel repositories unless a different channel is specified for the leapp execution (oamg#1205) - Update the default kernel cmdline (oamg#1193, oamg#1216) - Update the device driver deprecation data, fixing invalid fields for some AMD CPUs (oamg#1211) - Wait for the storage initialization when /usr is on separate file system - covering SAN (oamg#1218, oamg#1219) - [IPU 7 -> 8] Drop enforced tomcat removal for satellite when upgrading to RHEL 8.10 (oamg#1243) - [IPU 7 -> 8] Fix detection of bootable device on RAID (oamg#1260) - [IPU 8 -> 9] Inhibit the upgrade to RHEL 9.5 on ARM architecture due to incompatibility of the RHEL 8 bootloader and RHEL 9.5 kernel (oamg#1270) ### Enhancements - [IPU 8 -> 9] Introduce upgrade path 8.10 -> 9.5 (oamg#1245, oamg#1246) - Update leapp data files (oamg#1280) - Apply solutions for leftover rpms for all major upgrade paths - including experimental actors (oamg#1199) - Do not terminate the upgrade dracut module execution anymore if /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed exists (oamg#1197) - Improve set_systemd_services_states logging (oamg#1213) - Include leapp command execution and defined leapp envars inside leapp.db - (oamg#1152) - Introduce experimental upgrades in 'live' mode for the testing (oamg#1248) - Load obsoleted GPG keys from gpg-signatures.json file instead of hardcoding them (oamg#1241) - Several minor improvements in messages printed in console output (oamg#1173, oamg#1214, oamg#1274) - Several minor improvements in report and error messages (oamg#1207, oamg#1217, oamg#1234, oamg#1235, oamg#1242) - Sort lists in dnf-plugin-data for easier overview (oamg#1231) - [IPU 7 -> 8] Allow upgrade of content from ELS repositories (oamg#1198) - [IPU 7 -> 8] Inhibit the upgrade when Legacy GRUB is detected (oamg#1206) - [IPU 7 -> 8] Inhibit the upgrade when embedding area is small to prevent failed bootloader update (oamg#1195) - [IPU 8 -> 9] Enable EL 8 > 9 upgrades on Alibaba cloud (oamg#1249) - [IPU 8 -> 9] Enable EL 8 to 9 upgrade of Satellite/Foreman server (oamg#1181) - [IPU 9 -> 10] Introduced number of changes to enable IPU 9 -> 10 for testing (oamg#1169) - [IPU 9 -> 10] Prevent upgrading if NetworkManager is configured with dhcp=dhclient (oamg#1268) - [IPU 9 -> 10] Update URLs in reports to reflect the next planned major upgrade path (oamg#1169, oamg#1273) ## Additional changes interesting for devels - drop unused `packager` field from gpg-signatures.json (oamg#1233) - [IPU 9 -> 10] make system_upgrade/common leapp repo Python 3.12 compatible - [IPU 9 -> 10] introduced system_upgrade/el9toel10 leapp repo (cherry picked from commit 03c257b)

mscherer force-pushed the bug/fix_root_on_raid branch 2 times, most recently from 9311442 to 0f0a0a5 Compare July 3, 2024 13:03

pirat89 reviewed Jul 3, 2024

View reviewed changes

...ystem_upgrade/el7toel8/actors/scangrubdevpartitionlayout/tests/test_scan_partition_layout.py Outdated Show resolved Hide resolved

mscherer force-pushed the bug/fix_root_on_raid branch from 0f0a0a5 to c26ea69 Compare July 3, 2024 13:13

mscherer force-pushed the bug/fix_root_on_raid branch from c26ea69 to b3e5804 Compare July 3, 2024 13:51

pirat89 added the bug Something isn't working label Jul 10, 2024

pirat89 added this to the 8.10/9.5 milestone Jul 16, 2024

pirat89 changed the title ~~[WIP] Fix detection when a bootable partition in on RAID~~ Fix detection when a bootable partition in on RAID Jul 16, 2024

mscherer force-pushed the bug/fix_root_on_raid branch from b3e5804 to da43a29 Compare July 17, 2024 19:44

pirat89 approved these changes Jul 20, 2024

View reviewed changes

pirat89 merged commit 7cd469b into oamg:master Jul 20, 2024
17 checks passed

pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Jul 20, 2024

pirat89 mentioned this pull request Aug 16, 2024

# Release 0.21.0 #1282

Merged

yuravk mentioned this pull request Aug 22, 2024

ELevate 0.21.0 AlmaLinux/leapp-repository#124

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix detection when a bootable partition in on RAID #1260

Fix detection when a bootable partition in on RAID #1260

mscherer commented Jul 3, 2024 •

edited

Loading

github-actions bot commented Jul 3, 2024

fernflower commented Jul 3, 2024

mscherer commented Jul 3, 2024

mscherer commented Jul 3, 2024

pirat89 commented Jul 3, 2024 •

edited

Loading

mscherer commented Jul 3, 2024

mscherer commented Jul 3, 2024

pirat89 commented Jul 4, 2024

fernflower commented Jul 4, 2024

mscherer commented Jul 9, 2024

pirat89 commented Jul 10, 2024 •

edited

Loading

pirat89 commented Jul 16, 2024 •

edited

Loading

mscherer commented Jul 17, 2024

pirat89 commented Jul 18, 2024

pirat89 left a comment

Fix detection when a bootable partition in on RAID #1260

Fix detection when a bootable partition in on RAID #1260

Conversation

mscherer commented Jul 3, 2024 • edited Loading

github-actions bot commented Jul 3, 2024

Thank you for contributing to the Leapp project!

fernflower commented Jul 3, 2024

mscherer commented Jul 3, 2024

mscherer commented Jul 3, 2024

pirat89 commented Jul 3, 2024 • edited Loading

mscherer commented Jul 3, 2024

mscherer commented Jul 3, 2024

pirat89 commented Jul 4, 2024

fernflower commented Jul 4, 2024

mscherer commented Jul 9, 2024

pirat89 commented Jul 10, 2024 • edited Loading

pirat89 commented Jul 16, 2024 • edited Loading

mscherer commented Jul 17, 2024

pirat89 commented Jul 18, 2024

pirat89 left a comment

Choose a reason for hiding this comment

mscherer commented Jul 3, 2024 •

edited

Loading

pirat89 commented Jul 3, 2024 •

edited

Loading

pirat89 commented Jul 10, 2024 •

edited

Loading

pirat89 commented Jul 16, 2024 •

edited

Loading