Wait to preload with qrexec timeout property

[ben-grande]

Tends to fail on "race_more" as requesting 3 Whonix preloaded
disposables seems to be too much for OpenQA.

For: QubesOS/qubes-issues#1512

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.68%. Comparing base (36f3474) to head (5d5c135).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #764      +/-   ##
==========================================
- Coverage   70.74%   70.68%   -0.06%     
==========================================
  Files          61       61              
  Lines       13809    13809              
==========================================
- Hits         9769     9761       -8     
- Misses       4040     4048       +8     

Flag	Coverage Δ
unittests	70.68% <ø> (-0.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[qubesos-bot]

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025121419-4.3&flavor=pull-requests

Test run included the following:

• Add option to wait for GUI session qubes-app-linux-input-proxy#42 (QubesOS/qubes-app-linux-input-proxy@cd6d205)
• Add Arch packaging qubes-core-admin-client#332 (QubesOS/qubes-core-admin-client@40487bc)
• Fix devices serialization to not use prohibited chars qubes-core-admin-client#399 (QubesOS/qubes-core-admin-client@28b79d7)
• Add method to solely create disposable qubes-core-admin-client#401 (QubesOS/qubes-core-admin-client@96e818a)
• Deserialize device in events only if there is a handler registered qubes-core-admin-client#402 (QubesOS/qubes-core-admin-client@322aa48)
• tests: log more info on failure #736 (a3fc21c)
• Disposable performance: Add drive information to HCL, more details in spec, more tests for line graph #744 (5c352a7)
• Wait for user session for preloaded disposables #757 (4094238)
• Switch autostart event in favor of only start #758 (b474909)
• Deserialize device from qrexec arg properly #759 (6137b62)
• Wait to preload with qrexec timeout property #764 (5d5c135)
• Allow GUIVM clients to know if GUIVM has session qubes-core-agent-linux#625 (QubesOS/qubes-core-agent-linux@420e90f)
• network: reload firewall rules on DNS change qubes-core-agent-linux#626 (QubesOS/qubes-core-agent-linux@08f9d2a)
• Query GUIVM session or GUID from client in GUIVM qubes-core-qrexec#217 (QubesOS/qubes-core-qrexec@88935ab)
• Update to upstream version 4.24 qubes-desktop-linux-i3#54 (QubesOS/qubes-desktop-linux-i3@0e8b9b6)
• Delay connecting with gui-daemon until after starting Xorg qubes-gui-agent-linux#251 (QubesOS/qubes-gui-agent-linux@a981e36)
• Re-dock tray icons when tray manager is restarted qubes-gui-daemon#173 (QubesOS/qubes-gui-daemon@d37169c)
• Hide disks that other udev rules marked as hidden from udisks qubes-linux-utils#133 (QubesOS/qubes-linux-utils@17f8800)
• Allow GUIVM clients to know if GUIVM has session qubes-mgmt-salt-dom0-virtual-machines#88 (QubesOS/qubes-mgmt-salt-dom0-virtual-machines@4c3007f)
• Wait for GUI session on the server side qubes-notification-proxy#14 (QubesOS/qubes-notification-proxy@6da96ee)

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025111104-4.3&flavor=update

• system_tests_dispvm

  • TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error + timed out)
    qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...

• system_tests_qwt_win10_seamless@hw13

  • windows_clipboard_and_filecopy: unnamed test (unknown)

• system_tests_gui_interactive_preloaded

  • collect_logs: wait_serial (wait serial expected)
    # wait_serial expected: qr/Pw7Zf-\d+-/...

  • collect_logs: Failed (test died + timed out)
    # Test died: command 'curl --form upload=@journalctl.log --form upn...

  • collect_logs: wait_serial (wait serial expected)
    # wait_serial expected: qr/fQRNV-\d+-/...

• system_tests_dispvm_perf@hw7

  • TC_00_DispVMPerf_fedora-42-xfce: test_400_dom0_dispvm_api (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

  • TC_00_DispVMPerf_fedora-42-xfce: test_900_reader (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

• system_tests_basic_vm_qrexec_gui_ext4

  • TC_20_NonAudio_whonix-gateway-18-pool: test_400_long_window_title (error + cleanup)
    raise TimeoutError from exc_val... TimeoutError

• system_tests_basic_vm_qrexec_gui_xfs

  • TC_00_Basic: test_100_qvm_create (failure)
    AssertionError: libvirt event impl drain timeout

Failed tests

12 failures

• system_tests_dispvm

  • TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error + timed out)
    qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...

• system_tests_qwt_win10_seamless@hw13

  • windows_clipboard_and_filecopy: unnamed test (unknown)
  • windows_clipboard_and_filecopy: Failed (test died)
    # Test died: no candidate needle with tag(s) 'windows-Edge-address-...

• system_tests_qwt_win11@hw13

  • windows_install: Failed (test died)
    # Test died: Install failed with code 1 at qubesos/tests/windows_in...

• system_tests_gui_interactive_preloaded

  • collect_logs: wait_serial (wait serial expected)
    # wait_serial expected: qr/Pw7Zf-\d+-/...

  • collect_logs: Failed (test died + timed out)
    # Test died: command 'curl --form upload=@journalctl.log --form upn...

  • collect_logs: wait_serial (wait serial expected)
    # wait_serial expected: qr/fQRNV-\d+-/...

• system_tests_dispvm_perf@hw7

  • TC_00_DispVMPerf_fedora-42-xfce: test_400_dom0_dispvm_api (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

  • TC_00_DispVMPerf_fedora-42-xfce: test_900_reader (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

  • TC_00_DispVMPerf_whonix-workstation-18: test_900_reader (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

• system_tests_basic_vm_qrexec_gui_ext4

  • TC_20_NonAudio_whonix-gateway-18-pool: test_400_long_window_title (error + cleanup)
    raise TimeoutError from exc_val... TimeoutError

• system_tests_basic_vm_qrexec_gui_xfs

  • TC_00_Basic: test_100_qvm_create (failure)
    AssertionError: libvirt event impl drain timeout

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/158999#dependencies

18 fixed

• system_tests_extra

  • system_tests: Fail (unknown)
    Tests qubes.tests.extra failed (exit code 1), details reported sepa...

  • system_tests: Failed (test died)
    # Test died: Some tests failed at qubesos/tests/system_tests.pm lin...

  • TC_00_QVCTest_debian-13-xfce: test_010_screenshare (failure + cleanup)
    AssertionError: 2.4614345149565264 not less than 2.0

  • TC_00_QVCTest_whonix-gateway-18: test_010_screenshare (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • TC_00_QVCTest_whonix-workstation-18: test_010_screenshare (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

• system_tests_guivm_gui_interactive

  • gui_keyboard_layout: unnamed test (unknown)
  • gui_keyboard_layout: Failed (test died)
    # Test died: no candidate needle with tag(s) 'work-xterm, work-xter...

• system_tests_dispvm

  • TC_20_DispVM_whonix-workstation-18: test_030_edit_file (failure)
    AssertionError: Timeout waiting for editor window

  • TC_20_DispVM_whonix-workstation-18: test_100_open_in_dispvm (failure)
    AssertionError: './open-file test.txt' failed with ./open-file test...

• system_tests_guivm_vnc_gui_interactive

  • gui_filecopy: unnamed test (unknown)
  • gui_filecopy: Failed (test died)
    # Test died: no candidate needle with tag(s) 'disp-text-editor' mat...

• system_tests_qwt_win10@hw13

  • windows_install: wait_serial (wait serial expected)
    # wait_serial expected: qr/kauF4-\d+-/...

  • windows_install: Failed (test died + timed out)
    # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

• system_tests_qwt_win10_seamless@hw13

  • windows_clipboard_and_filecopy: unnamed test (unknown)

• system_tests_qwt_win11@hw13

  • windows_install: wait_serial (wait serial expected)
    # wait_serial expected: qr/iDVvW-\d+-/...

• system_tests_dispvm_perf@hw7

  • TC_00_DispVMPerf_debian-13-xfce: test_411_dom0_dispvm_preload_gui_concurrent_api (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

  • TC_00_DispVMPerf_debian-13-xfce: test_900_reader (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

  • TC_00_DispVMPerf_whonix-workstation-18: test_409_dom0_dispvm_preload_gui_api (failure)
    AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

Unstable tests

Details

Performance Tests

Performance degradation:

11 performance degradations

• debian-13-xfce_exec: 7.99 🔻 ( previous job: 6.42, degradation: 124.42%)
• dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 307771.00 🔻 ( previous job: 491827.00, degradation: 62.58%)
• fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 119277.00 🔻 ( previous job: 137220.00, degradation: 86.92%)
• fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 307500.00 🔻 ( previous job: 355690.00, degradation: 86.45%)
• fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 1992.00 🔻 ( previous job: 3964.00, degradation: 50.25%)
• fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 33249.00 🔻 ( previous job: 49472.00, degradation: 67.21%)
• fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 17538.00 🔻 ( previous job: 75326.00, degradation: 23.28%)
• fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 119920.00 🔻 ( previous job: 140939.00, degradation: 85.09%)
• fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 271511.00 🔻 ( previous job: 349176.00, degradation: 77.76%)
• fedora-42-xfce_dom0-vm-api (mean:0.037): 0.45 🔻 ( previous job: 0.39, degradation: 114.29%)
• fedora-42-xfce_dom0-vm-gui-api (mean:0.056): 0.67 🔻 ( previous job: 0.54, degradation: 123.70%)

Remaining performance tests:

84 tests

• debian-13-xfce_exec-root: 27.12 🟢 ( previous job: 27.48, improvement: 98.68%)
• debian-13-xfce_socket: 8.15 🟢 ( previous job: 8.68, improvement: 93.91%)
• debian-13-xfce_socket-root: 8.24 🟢 ( previous job: 8.40, improvement: 98.05%)
• debian-13-xfce_exec-data-simplex: 68.10 🔻 ( previous job: 67.70, degradation: 100.60%)
• debian-13-xfce_exec-data-duplex: 69.13 🟢 ( previous job: 69.20, improvement: 99.90%)
• debian-13-xfce_exec-data-duplex-root: 79.25 🟢 ( previous job: 86.64, improvement: 91.46%)
• debian-13-xfce_socket-data-duplex: 133.83 🟢 ( previous job: 136.31, improvement: 98.18%)
• fedora-42-xfce_exec: 9.17 🟢 ( previous job: 9.23, improvement: 99.33%)
• fedora-42-xfce_exec-root: 59.89 🟢 ( previous job: 59.89, improvement: 100.00%)
• fedora-42-xfce_socket: 8.07 🟢 ( previous job: 8.23, improvement: 98.02%)
• fedora-42-xfce_socket-root: 8.48 🔻 ( previous job: 8.31, degradation: 102.00%)
• fedora-42-xfce_exec-data-simplex: 69.42 🔻 ( previous job: 66.18, degradation: 104.90%)
• fedora-42-xfce_exec-data-duplex: 65.14 🟢 ( previous job: 71.51, improvement: 91.10%)
• fedora-42-xfce_exec-data-duplex-root: 99.91 🔻 ( previous job: 96.52, degradation: 103.51%)
• fedora-42-xfce_socket-data-duplex: 138.43 🔻 ( previous job: 137.78, degradation: 100.47%)
• whonix-gateway-18_exec: 8.51 🟢 ( previous job: 8.52, improvement: 99.93%)
• whonix-gateway-18_exec-root: 131.84 🔻 ( previous job: 129.05, degradation: 102.16%)
• whonix-gateway-18_socket: 8.05 🟢 ( previous job: 8.26, improvement: 97.46%)
• whonix-gateway-18_socket-root: 8.26 🔻 ( previous job: 7.76, degradation: 106.54%)
• whonix-gateway-18_exec-data-simplex: 69.13 🟢 ( previous job: 73.13, improvement: 94.54%)
• whonix-gateway-18_exec-data-duplex: 70.60 🟢 ( previous job: 71.98, improvement: 98.08%)
• whonix-gateway-18_exec-data-duplex-root: 147.92 🔻 ( previous job: 146.68, degradation: 100.85%)
• whonix-gateway-18_socket-data-duplex: 141.12 🟢 ( previous job: 142.71, improvement: 98.89%)
• whonix-workstation-18_exec: 8.93 🔻 ( previous job: 8.85, degradation: 100.86%)
• whonix-workstation-18_exec-root: 140.56 🟢 ( previous job: 149.84, improvement: 93.80%)
• whonix-workstation-18_socket: 9.17 🔻 ( previous job: 9.12, degradation: 100.53%)
• whonix-workstation-18_socket-root: 8.09 🟢 ( previous job: 8.75, improvement: 92.49%)
• whonix-workstation-18_exec-data-simplex: 66.99 🟢 ( previous job: 67.02, improvement: 99.96%)
• whonix-workstation-18_exec-data-duplex: 71.06 🔻 ( previous job: 69.25, degradation: 102.61%)
• whonix-workstation-18_exec-data-duplex-root: 140.98 🟢 ( previous job: 146.32, improvement: 96.35%)
• whonix-workstation-18_socket-data-duplex: 132.74 🟢 ( previous job: 139.15, improvement: 95.39%)
• dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 439470.00 🟢 ( previous job: 365230.00, improvement: 120.33%)
• dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 220335.00 🔻 ( previous job: 231269.00, degradation: 95.27%)
• dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 103750.00 🔻 ( previous job: 110619.00, degradation: 93.79%)
• dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 123452.00 🔻 ( previous job: 130397.00, degradation: 94.67%)
• dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 43982.00 🟢 ( previous job: 39923.00, improvement: 110.17%)
• dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 7643.00 🟢 ( previous job: 3264.00, improvement: 234.16%)
• dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 12138.00 🟢 ( previous job: 12000.00, improvement: 101.15%)
• dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 3874.00 🟢 ( previous job: 2099.00, improvement: 184.56%)
• dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 326354.00 🟢 ( previous job: 103611.00, improvement: 314.98%)
• dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 433295.00 🟢 ( previous job: 432938.00, improvement: 100.08%)
• dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 186794.00 🟢 ( previous job: 143368.00, improvement: 130.29%)
• dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 105175.00 🟢 ( previous job: 103977.00, improvement: 101.15%)
• dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 9359.00 🔻 ( previous job: 10236.00, degradation: 91.43%)
• dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7828.00 🔻 ( previous job: 8300.00, degradation: 94.31%)
• dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 5154.00 🟢 ( previous job: 3936.00, improvement: 130.95%)
• fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 393609.00 🟢 ( previous job: 346866.00, improvement: 113.48%)
• fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 53519.00 🟢 ( previous job: 26931.00, improvement: 198.73%)
• fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 77268.00 🔻 ( previous job: 85360.00, degradation: 90.52%)
• fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 9402.00 🟢 ( previous job: 8945.00, improvement: 105.11%)
• fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 757.00 🟢 ( previous job: 457.00, improvement: 165.65%)
• fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 370783.00 🔻 ( previous job: 371835.00, degradation: 99.72%)
• fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 129102.00 🟢 ( previous job: 108858.00, improvement: 118.60%)
• fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 371308.00 🟢 ( previous job: 351871.00, improvement: 105.52%)
• fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 2033.00 🟢 ( previous job: 1909.00, improvement: 106.50%)
• fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 7790.00 🔻 ( previous job: 8365.00, degradation: 93.13%)
• fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1050.00 🟢 ( previous job: 517.00, improvement: 203.09%)
• fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 364215.00 🟢 ( previous job: 361328.00, improvement: 100.80%)
• fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 52199.00 🟢 ( previous job: 46842.00, improvement: 111.44%)
• fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 85067.00 🟢 ( previous job: 79581.00, improvement: 106.89%)
• fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2409.00 🟢 ( previous job: 2110.00, improvement: 114.17%)
• fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8603.00 🟢 ( previous job: 7708.00, improvement: 111.61%)
• fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1150.00 🟢 ( previous job: 861.00, improvement: 133.57%)
• debian-13-xfce_dom0-dispvm-api (mean:6.966): 83.59 🟢 ( previous job: 84.68, improvement: 98.72%)
• debian-13-xfce_dom0-dispvm-gui-api (mean:7.833): 94.00 🟢 ( previous job: 98.62, improvement: 95.31%)
• debian-13-xfce_dom0-dispvm-preload-2-api (mean:4.105): 49.26
• debian-13-xfce_dom0-dispvm-preload-4-api (mean:3.781): 45.37
• debian-13-xfce_dom0-dispvm-preload-2-gui-api (mean:4.169): 50.02
• debian-13-xfce_dom0-dispvm-preload-4-gui-api (mean:3.893): 46.72
• debian-13-xfce_dom0-vm-api (mean:0.033): 0.40 🟢 ( previous job: 0.59, improvement: 67.01%)
• debian-13-xfce_dom0-vm-gui-api (mean:0.043): 0.51 🟢 ( previous job: 0.58, improvement: 88.60%)
• fedora-42-xfce_dom0-dispvm-gui-api (mean:8.345): 100.14 🟢 ( previous job: 107.00, improvement: 93.59%)
• fedora-42-xfce_dom0-dispvm-preload-2-api (mean:4.55): 54.59
• fedora-42-xfce_dom0-dispvm-preload-4-api (mean:4.062): 48.74
• fedora-42-xfce_dom0-dispvm-preload-2-gui-api (mean:4.693): 56.32
• fedora-42-xfce_dom0-dispvm-preload-4-gui-api (mean:4.113): 49.36
• whonix-workstation-18_dom0-dispvm-api (mean:9.005): 108.06 🟢 ( previous job: 117.52, improvement: 91.95%)
• whonix-workstation-18_dom0-dispvm-gui-api (mean:9.589): 115.07 🟢 ( previous job: 130.38, improvement: 88.26%)
• whonix-workstation-18_dom0-dispvm-preload-2-api (mean:5.419): 65.03
• whonix-workstation-18_dom0-dispvm-preload-4-api (mean:4.765): 57.18
• whonix-workstation-18_dom0-dispvm-preload-2-gui-api (mean:5.567): 66.80
• whonix-workstation-18_dom0-dispvm-preload-4-gui-api (mean:4.657): 55.88
• whonix-workstation-18_dom0-vm-api (mean:0.038): 0.46 🟢 ( previous job: 0.60, improvement: 76.49%)
• whonix-workstation-18_dom0-vm-gui-api (mean:0.041): 0.49 🔻 ( previous job: 0.45, degradation: 108.46%)