Skip to content

Conversation

SaswatPadhi
Copy link
Contributor

@SaswatPadhi SaswatPadhi commented May 5, 2025

When booting an in-vm kernel with qubes-vm-kernel-support in Archlinux, the initramfs complains about missing gptfix binary (invoked in qubes_cow_setup.sh) and times out waiting for dmroot to be available. After adding gptfix it further complains about missing swapon. Adding those two binaries resolves the errors.

@marmarek
Copy link
Member

marmarek commented May 7, 2025

Is there a /usr/sbin -> /usr/bin symlink inside the initrd? gptfix is called via full /usr/sbin/gptfix

@SaswatPadhi
Copy link
Contributor Author

lsinitcpio /boot/initramfs-linux.img | grep gptfix is empty, indicating that gptfix is entirely missing. Same for swapon.
Upon extracting the image, I do see that there is /usr/sbin -> /usr/bin symlink, but /usr/bin seems to be missing the two files.

As documented on ArchWiki [1],
the `lvm2` hook must appear after the `block`
and before the `filesystems` hook.
Otherwise, LVM might attempt to activate logical volumes
before the underlying block devices are available.

[1]: https://wiki.archlinux.org/title/Install_Arch_Linux_on_LVM#Adding_mkinitcpio_hooks
@SaswatPadhi
Copy link
Contributor Author

@marmarek I also fixed the mkinitcpio hooks order as part of this initramfs fix.

Prior to this change, mkinitcpio -P generated a warning that /bin/sh may not be available for qubes_cow_setup.sh, since we place the qubes hook even before the base hook.

We should really be placing the lvm2 and qubes hooks right after the block hook though.

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.60%. Comparing base (2f550d1) to head (d1af3f7).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #124      +/-   ##
==========================================
+ Coverage   60.56%   60.60%   +0.04%     
==========================================
  Files           6        6              
  Lines         951      952       +1     
  Branches      104      104              
==========================================
+ Hits          576      577       +1     
  Misses        285      285              
  Partials       90       90              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@marmarek marmarek added the openqa-pending PR to be tested in the next OpenQA run label May 25, 2025
@qubesos-bot
Copy link

qubesos-bot commented May 26, 2025

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025052803-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025031804-4.3&flavor=update

  • system_tests_whonix

    • whonix_torbrowser: unnamed test (unknown)
    • whonix_torbrowser: Failed (test died)
      # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
  • system_tests_pvgrub_salt_storage

    • TC_40_PVGrub_archlinux: test_000_standalone_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_40_PVGrub_archlinux: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_41_HVMGrub_archlinux: test_000_standalone_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_41_HVMGrub_archlinux: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_42_PVHGrub_archlinux: test_000_standalone_vm (failure)
      AssertionError: b'6.14.7-arch2-1' != b'6.14.7.arch2-1'

    • TC_42_PVHGrub_archlinux: test_010_template_based_vm (failure)
      AssertionError: b'6.14.7-arch2-1' != b'6.14.7.arch2-1'

  • system_tests_dispvm

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: unnamed test (unknown)

    • gui_keyboard_layout: Failed (test died)
      # Test died: no candidate needle with tag(s) 'work-xterm, work-xter...

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: qr/wHeLC-\d+-/...

  • system_tests_audio

  • system_tests_audio@hw1

  • system_tests_qwt_win10_seamless@hw13

    • windows_install: Failed (test died)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...
  • system_tests_qwt_win11@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Notepad' matc...

Failed tests

47 failures
  • system_tests_whonix

    • whonix_torbrowser: unnamed test (unknown)
    • whonix_torbrowser: Failed (test died)
      # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
  • system_tests_pvgrub_salt_storage

    • TC_40_PVGrub_archlinux: test_000_standalone_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_40_PVGrub_archlinux: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_41_HVMGrub_archlinux: test_000_standalone_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_41_HVMGrub_archlinux: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_42_PVHGrub_archlinux: test_000_standalone_vm (failure)
      AssertionError: b'6.14.7-arch2-1' != b'6.14.7.arch2-1'

    • TC_42_PVHGrub_archlinux: test_010_template_based_vm (failure)
      AssertionError: b'6.14.7-arch2-1' != b'6.14.7.arch2-1'

  • system_tests_dispvm

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: unnamed test (unknown)

    • gui_keyboard_layout: Failed (test died)
      # Test died: no candidate needle with tag(s) 'work-xterm, work-xter...

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: qr/wHeLC-\d+-/...

  • system_tests_audio

  • system_tests_audio@hw1

  • system_tests_qwt_win10_seamless@hw13

    • windows_install: Failed (test died)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...
  • system_tests_qwt_win11@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Notepad' matc...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/132953#dependencies

14 fixed
  • system_tests_whonix

    • whonixcheck: fail (unknown)
      Whonixcheck for sys-whonix failed...

    • whonixcheck: unnamed test (unknown)

  • system_tests_suspend

    • suspend: unnamed test (unknown)
    • suspend: Failed (test died)
      # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
  • system_tests_basic_vm_qrexec_gui

  • system_tests_qrexec

  • system_tests_kde_gui_interactive

    • clipboard_and_web: unnamed test (unknown)

    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

    • clipboard_and_web: wait_serial (wait serial expected)
      # wait_serial expected: "lspci; echo 2E8vz-\$?-"...

  • system_tests_guivm_vnc_gui_interactive

    • gui_filecopy: unnamed test (unknown)
    • gui_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'files-work' matched...
  • system_tests_audio

  • system_tests_whonix@hw7

    • whonixcheck: fail (unknown)
      Whonixcheck for sys-whonix failed...

    • whonixcheck: unnamed test (unknown)

Unstable tests

  • system_tests_update

    update2/Failed (1/5 times with errors)
    • job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...
  • system_tests_update@hw1

    update2/Failed (1/5 times with errors)
    • job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...
  • system_tests_update@hw7

    update2/Failed (1/5 times with errors)
    • job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...
  • system_tests_update@hw13

    update2/Failed (1/5 times with errors)
    • job 139051 # Test died: command 'script -c 'qubes-vm-update --force-update --l...

Performance Tests

Performance degradation:

12 performance degradations
  • debian-12-xfce_exec: 8.16 🔺 ( previous job: 7.12, degradation: 114.57%)
  • debian-12-xfce_exec-data-duplex: 77.81 🔺 ( previous job: 70.34, degradation: 110.61%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 8146.00 :small_red_triangle: ( previous job: 10795.00, degradation: 75.46%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 3672.00 :small_red_triangle: ( previous job: 4826.00, degradation: 76.09%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 225016.00 :small_red_triangle: ( previous job: 250795.00, degradation: 89.72%)
  • fedora-41-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2243.00 :small_red_triangle: ( previous job: 3599.00, degradation: 62.32%)
  • fedora-41-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 125272.00 :small_red_triangle: ( previous job: 170062.00, degradation: 73.66%)
  • fedora-41-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 312821.00 :small_red_triangle: ( previous job: 369868.00, degradation: 84.58%)
  • fedora-41-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 85316.00 :small_red_triangle: ( previous job: 179949.00, degradation: 47.41%)
  • fedora-41-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 14720.00 :small_red_triangle: ( previous job: 17567.00, degradation: 83.79%)
  • fedora-41-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2277.00 :small_red_triangle: ( previous job: 5672.00, degradation: 40.14%)
  • fedora-41-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1229.00 :small_red_triangle: ( previous job: 1953.00, degradation: 62.93%)

Remaining performance tests:

52 tests
  • archlinux_exec: 7.60
  • archlinux_exec-root: 34.51
  • archlinux_socket: 8.49
  • archlinux_socket-root: 8.22
  • archlinux_exec-data-simplex: 63.36
  • archlinux_exec-data-duplex: 74.70
  • archlinux_exec-data-duplex-root: 89.51
  • archlinux_socket-data-duplex: 153.01
  • debian-12-xfce_exec-root: 30.60 🔺 ( previous job: 28.65, degradation: 106.80%)
  • debian-12-xfce_socket: 9.02 🔺 ( previous job: 8.60, degradation: 104.89%)
  • debian-12-xfce_socket-root: 8.48 🟢 ( previous job: 8.52, improvement: 99.48%)
  • debian-12-xfce_exec-data-simplex: 73.67 🔺 ( previous job: 71.62, degradation: 102.86%)
  • debian-12-xfce_exec-data-duplex-root: 82.17 🟢 ( previous job: 82.72, improvement: 99.34%)
  • debian-12-xfce_socket-data-duplex: 153.72 🟢 ( previous job: 156.96, improvement: 97.94%)
  • fedora-41-xfce_exec: 9.47 🔺 ( previous job: 9.27, degradation: 102.21%)
  • fedora-41-xfce_exec-root: 60.32 🟢 ( previous job: 61.51, improvement: 98.06%)
  • fedora-41-xfce_socket: 8.47 🟢 ( previous job: 8.63, improvement: 98.22%)
  • fedora-41-xfce_socket-root: 8.85 🔺 ( previous job: 8.71, degradation: 101.58%)
  • fedora-41-xfce_exec-data-simplex: 72.35 🟢 ( previous job: 75.53, improvement: 95.78%)
  • fedora-41-xfce_exec-data-duplex: 69.41 🟢 ( previous job: 71.56, improvement: 96.99%)
  • fedora-41-xfce_exec-data-duplex-root: 106.78 🟢 ( previous job: 109.13, improvement: 97.85%)
  • fedora-41-xfce_socket-data-duplex: 158.42 🔺 ( previous job: 150.61, degradation: 105.19%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 425213.00 :small_red_triangle: ( previous job: 446963.00, degradation: 95.13%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 210684.00 :green_circle: ( previous job: 129298.00, improvement: 162.94%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 330885.00 :green_circle: ( previous job: 294295.00, improvement: 112.43%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 156622.00 :green_circle: ( previous job: 95454.00, improvement: 164.08%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 100964.00 :green_circle: ( previous job: 79803.00, improvement: 126.52%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 7625.00 :green_circle: ( previous job: 6149.00, improvement: 124.00%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 397489.00 :green_circle: ( previous job: 382273.00, improvement: 103.98%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 441877.00 :green_circle: ( previous job: 437636.00, improvement: 100.97%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 173763.00 :small_red_triangle: ( previous job: 184752.00, degradation: 94.05%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 104796.00 :green_circle: ( previous job: 62195.00, improvement: 168.50%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 10288.00 :green_circle: ( previous job: 6479.00, improvement: 158.79%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7704.00 :green_circle: ( previous job: 7669.00, improvement: 100.46%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 4606.00 :small_red_triangle: ( previous job: 4903.00, degradation: 93.94%)
  • fedora-41-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 375699.00 :green_circle: ( previous job: 368309.00, improvement: 102.01%)
  • fedora-41-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 188017.00 :green_circle: ( previous job: 162081.00, improvement: 116.00%)
  • fedora-41-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 321649.00 :green_circle: ( previous job: 318716.00, improvement: 100.92%)
  • fedora-41-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 84839.00 :small_red_triangle: ( previous job: 87940.00, degradation: 96.47%)
  • fedora-41-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 83778.00 :green_circle: ( previous job: 82694.00, improvement: 101.31%)
  • fedora-41-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 8618.00 :green_circle: ( previous job: 8485.00, improvement: 101.57%)
  • fedora-41-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1341.00 :green_circle: ( previous job: 542.00, improvement: 247.42%)
  • fedora-41-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 408483.00 :green_circle: ( previous job: 373957.00, improvement: 109.23%)
  • fedora-41-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 304199.00 :small_red_triangle: ( previous job: 334687.00, degradation: 90.89%)
  • fedora-41-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 73610.00 :green_circle: ( previous job: 61534.00, improvement: 119.62%)
  • fedora-41-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 85971.00 :green_circle: ( previous job: 80283.00, improvement: 107.08%)
  • fedora-41-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 2749.00 :green_circle: ( previous job: 2215.00, improvement: 124.11%)
  • fedora-41-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8058.00 :green_circle: ( previous job: 7540.00, improvement: 106.87%)
  • fedora-41-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1110.00 :small_red_triangle: ( previous job: 1130.00, degradation: 98.23%)
  • fedora-41-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 344133.00 :green_circle: ( previous job: 324737.00, improvement: 105.97%)
  • fedora-41-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 87393.00 :green_circle: ( previous job: 79021.00, improvement: 110.59%)
  • fedora-41-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8193.00 :green_circle: ( previous job: 7867.00, improvement: 104.14%)

This was referenced May 26, 2025
@marmarek
Copy link
Member

marmarek commented May 28, 2025

I assume you made also some changes to the grub config? The one I get after just grub-mkconfig looks insufficient...
Nevermind, it's just about package installation order...

marmarek added a commit to marmarek/qubes-core-admin that referenced this pull request May 28, 2025
marmarek added a commit to marmarek/qubes-core-admin that referenced this pull request May 29, 2025
@marmarek marmarek merged commit 4ae0573 into QubesOS:main May 29, 2025
2 of 3 checks passed
marmarek added a commit to marmarek/qubes-core-admin that referenced this pull request May 29, 2025
marmarek added a commit to marmarek/qubes-core-admin that referenced this pull request May 29, 2025
Test for QubesOS/qubes-linux-utils#124

Exclude pvgrub2 test, as Arch is also using zstd.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

openqa-pending PR to be tested in the next OpenQA run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants