Skip to content

Conversation

@marmarek
Copy link
Member

@marmarek marmarek commented Jan 20, 2025

@codecov
Copy link

codecov bot commented Jan 20, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.26%. Comparing base (4e4fa79) to head (e22c531).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #332   +/-   ##
=======================================
  Coverage   75.26%   75.26%           
=======================================
  Files          52       52           
  Lines        8975     8975           
=======================================
  Hits         6755     6755           
  Misses       2220     2220           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@qubesos-bot
Copy link

qubesos-bot commented Jan 21, 2025

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025121419-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025111104-4.3&flavor=update

  • system_tests_dispvm

    • TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error + timed out)
      qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
  • system_tests_qwt_win10_seamless@hw13

  • system_tests_gui_interactive_preloaded

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/Pw7Zf-\d+-/...

    • collect_logs: Failed (test died + timed out)
      # Test died: command 'curl --form upload=@journalctl.log --form upn...

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/fQRNV-\d+-/...

  • system_tests_dispvm_perf@hw7

    • TC_00_DispVMPerf_fedora-42-xfce: test_400_dom0_dispvm_api (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

    • TC_00_DispVMPerf_fedora-42-xfce: test_900_reader (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_20_NonAudio_whonix-gateway-18-pool: test_400_long_window_title (error + cleanup)
      raise TimeoutError from exc_val... TimeoutError
  • system_tests_basic_vm_qrexec_gui_xfs

Failed tests

12 failures
  • system_tests_dispvm

    • TC_20_DispVM_debian-13-xfce: test_017_preload_autostart (error + timed out)
      qubes.exc.QubesVMShutdownTimeoutError: Domain shutdown timed out: '...
  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Edge-address-...
  • system_tests_qwt_win11@hw13

    • windows_install: Failed (test died)
      # Test died: Install failed with code 1 at qubesos/tests/windows_in...
  • system_tests_gui_interactive_preloaded

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/Pw7Zf-\d+-/...

    • collect_logs: Failed (test died + timed out)
      # Test died: command 'curl --form upload=@journalctl.log --form upn...

    • collect_logs: wait_serial (wait serial expected)
      # wait_serial expected: qr/fQRNV-\d+-/...

  • system_tests_dispvm_perf@hw7

    • TC_00_DispVMPerf_fedora-42-xfce: test_400_dom0_dispvm_api (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf.py --dvm=test-ins...

    • TC_00_DispVMPerf_fedora-42-xfce: test_900_reader (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

    • TC_00_DispVMPerf_whonix-workstation-18: test_900_reader (failure)
      AssertionError: '/usr/lib/qubes/tests/dispvm_perf_reader.py --templ...

  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_20_NonAudio_whonix-gateway-18-pool: test_400_long_window_title (error + cleanup)
      raise TimeoutError from exc_val... TimeoutError
  • system_tests_basic_vm_qrexec_gui_xfs

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/158999#dependencies

18 fixed
  • system_tests_extra

    • system_tests: Fail (unknown)
      Tests qubes.tests.extra failed (exit code 1), details reported sepa...

    • system_tests: Failed (test died)
      # Test died: Some tests failed at qubesos/tests/system_tests.pm lin...

    • TC_00_QVCTest_debian-13-xfce: test_010_screenshare (failure + cleanup)
      AssertionError: 2.4614345149565264 not less than 2.0

    • TC_00_QVCTest_whonix-gateway-18: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

    • TC_00_QVCTest_whonix-workstation-18: test_010_screenshare (failure)
      AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • system_tests_guivm_gui_interactive

    • gui_keyboard_layout: unnamed test (unknown)
    • gui_keyboard_layout: Failed (test died)
      # Test died: no candidate needle with tag(s) 'work-xterm, work-xter...
  • system_tests_dispvm

    • TC_20_DispVM_whonix-workstation-18: test_030_edit_file (failure)
      AssertionError: Timeout waiting for editor window

    • TC_20_DispVM_whonix-workstation-18: test_100_open_in_dispvm (failure)
      AssertionError: './open-file test.txt' failed with ./open-file test...

  • system_tests_guivm_vnc_gui_interactive

    • gui_filecopy: unnamed test (unknown)
    • gui_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'disp-text-editor' mat...
  • system_tests_qwt_win10@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/kauF4-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_qwt_win10_seamless@hw13

  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/iDVvW-\d+-/...
  • system_tests_dispvm_perf@hw7

Unstable tests

Details

Performance Tests

Performance degradation:

11 performance degradations
  • debian-13-xfce_exec: 7.99 🔻 ( previous job: 6.42, degradation: 124.42%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 307771.00 🔻 ( previous job: 491827.00, degradation: 62.58%)
  • fedora-42-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 119277.00 🔻 ( previous job: 137220.00, degradation: 86.92%)
  • fedora-42-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 307500.00 🔻 ( previous job: 355690.00, degradation: 86.45%)
  • fedora-42-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 1992.00 🔻 ( previous job: 3964.00, degradation: 50.25%)
  • fedora-42-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 33249.00 🔻 ( previous job: 49472.00, degradation: 67.21%)
  • fedora-42-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 17538.00 🔻 ( previous job: 75326.00, degradation: 23.28%)
  • fedora-42-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 119920.00 🔻 ( previous job: 140939.00, degradation: 85.09%)
  • fedora-42-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 271511.00 🔻 ( previous job: 349176.00, degradation: 77.76%)
  • fedora-42-xfce_dom0-vm-api (mean:0.037): 0.45 🔻 ( previous job: 0.39, degradation: 114.29%)
  • fedora-42-xfce_dom0-vm-gui-api (mean:0.056): 0.67 🔻 ( previous job: 0.54, degradation: 123.70%)

Remaining performance tests:

84 tests
  • debian-13-xfce_exec-root: 27.12 🟢 ( previous job: 27.48, improvement: 98.68%)
  • debian-13-xfce_socket: 8.15 🟢 ( previous job: 8.68, improvement: 93.91%)
  • debian-13-xfce_socket-root: 8.24 🟢 ( previous job: 8.40, improvement: 98.05%)
  • debian-13-xfce_exec-data-simplex: 68.10 🔻 ( previous job: 67.70, degradation: 100.60%)
  • debian-13-xfce_exec-data-duplex: 69.13 🟢 ( previous job: 69.20, improvement: 99.90%)
  • debian-13-xfce_exec-data-duplex-root: 79.25 🟢 ( previous job: 86.64, improvement: 91.46%)
  • debian-13-xfce_socket-data-duplex: 133.83 🟢 ( previous job: 136.31, improvement: 98.18%)
  • fedora-42-xfce_exec: 9.17 🟢 ( previous job: 9.23, improvement: 99.33%)
  • fedora-42-xfce_exec-root: 59.89 🟢 ( previous job: 59.89, improvement: 100.00%)
  • fedora-42-xfce_socket: 8.07 🟢 ( previous job: 8.23, improvement: 98.02%)
  • fedora-42-xfce_socket-root: 8.48 🔻 ( previous job: 8.31, degradation: 102.00%)
  • fedora-42-xfce_exec-data-simplex: 69.42 🔻 ( previous job: 66.18, degradation: 104.90%)
  • fedora-42-xfce_exec-data-duplex: 65.14 🟢 ( previous job: 71.51, improvement: 91.10%)
  • fedora-42-xfce_exec-data-duplex-root: 99.91 🔻 ( previous job: 96.52, degradation: 103.51%)
  • fedora-42-xfce_socket-data-duplex: 138.43 🔻 ( previous job: 137.78, degradation: 100.47%)
  • whonix-gateway-18_exec: 8.51 🟢 ( previous job: 8.52, improvement: 99.93%)
  • whonix-gateway-18_exec-root: 131.84 🔻 ( previous job: 129.05, degradation: 102.16%)
  • whonix-gateway-18_socket: 8.05 🟢 ( previous job: 8.26, improvement: 97.46%)
  • whonix-gateway-18_socket-root: 8.26 🔻 ( previous job: 7.76, degradation: 106.54%)
  • whonix-gateway-18_exec-data-simplex: 69.13 🟢 ( previous job: 73.13, improvement: 94.54%)
  • whonix-gateway-18_exec-data-duplex: 70.60 🟢 ( previous job: 71.98, improvement: 98.08%)
  • whonix-gateway-18_exec-data-duplex-root: 147.92 🔻 ( previous job: 146.68, degradation: 100.85%)
  • whonix-gateway-18_socket-data-duplex: 141.12 🟢 ( previous job: 142.71, improvement: 98.89%)
  • whonix-workstation-18_exec: 8.93 🔻 ( previous job: 8.85, degradation: 100.86%)
  • whonix-workstation-18_exec-root: 140.56 🟢 ( previous job: 149.84, improvement: 93.80%)
  • whonix-workstation-18_socket: 9.17 🔻 ( previous job: 9.12, degradation: 100.53%)
  • whonix-workstation-18_socket-root: 8.09 🟢 ( previous job: 8.75, improvement: 92.49%)
  • whonix-workstation-18_exec-data-simplex: 66.99 🟢 ( previous job: 67.02, improvement: 99.96%)
  • whonix-workstation-18_exec-data-duplex: 71.06 🔻 ( previous job: 69.25, degradation: 102.61%)
  • whonix-workstation-18_exec-data-duplex-root: 140.98 🟢 ( previous job: 146.32, improvement: 96.35%)
  • whonix-workstation-18_socket-data-duplex: 132.74 🟢 ( previous job: 139.15, improvement: 95.39%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 439470.00 🟢 ( previous job: 365230.00, improvement: 120.33%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 220335.00 🔻 ( previous job: 231269.00, degradation: 95.27%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 103750.00 🔻 ( previous job: 110619.00, degradation: 93.79%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 123452.00 🔻 ( previous job: 130397.00, degradation: 94.67%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 43982.00 🟢 ( previous job: 39923.00, improvement: 110.17%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 7643.00 🟢 ( previous job: 3264.00, improvement: 234.16%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 12138.00 🟢 ( previous job: 12000.00, improvement: 101.15%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 3874.00 🟢 ( previous job: 2099.00, improvement: 184.56%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 326354.00 🟢 ( previous job: 103611.00, improvement: 314.98%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 433295.00 🟢 ( previous job: 432938.00, improvement: 100.08%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 186794.00 🟢 ( previous job: 143368.00, improvement: 130.29%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 105175.00 🟢 ( previous job: 103977.00, improvement: 101.15%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 9359.00 🔻 ( previous job: 10236.00, degradation: 91.43%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 7828.00 🔻 ( previous job: 8300.00, degradation: 94.31%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 5154.00 🟢 ( previous job: 3936.00, improvement: 130.95%)
  • fedora-42-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 393609.00 🟢 ( previous job: 346866.00, improvement: 113.48%)
  • fedora-42-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 53519.00 🟢 ( previous job: 26931.00, improvement: 198.73%)
  • fedora-42-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 77268.00 🔻 ( previous job: 85360.00, degradation: 90.52%)
  • fedora-42-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 9402.00 🟢 ( previous job: 8945.00, improvement: 105.11%)
  • fedora-42-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 757.00 🟢 ( previous job: 457.00, improvement: 165.65%)
  • fedora-42-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 370783.00 🔻 ( previous job: 371835.00, degradation: 99.72%)
  • fedora-42-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 129102.00 🟢 ( previous job: 108858.00, improvement: 118.60%)
  • fedora-42-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 371308.00 🟢 ( previous job: 351871.00, improvement: 105.52%)
  • fedora-42-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 2033.00 🟢 ( previous job: 1909.00, improvement: 106.50%)
  • fedora-42-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 7790.00 🔻 ( previous job: 8365.00, degradation: 93.13%)
  • fedora-42-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1050.00 🟢 ( previous job: 517.00, improvement: 203.09%)
  • fedora-42-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 364215.00 🟢 ( previous job: 361328.00, improvement: 100.80%)
  • fedora-42-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 52199.00 🟢 ( previous job: 46842.00, improvement: 111.44%)
  • fedora-42-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 85067.00 🟢 ( previous job: 79581.00, improvement: 106.89%)
  • fedora-42-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 2409.00 🟢 ( previous job: 2110.00, improvement: 114.17%)
  • fedora-42-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8603.00 🟢 ( previous job: 7708.00, improvement: 111.61%)
  • fedora-42-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1150.00 🟢 ( previous job: 861.00, improvement: 133.57%)
  • debian-13-xfce_dom0-dispvm-api (mean:6.966): 83.59 🟢 ( previous job: 84.68, improvement: 98.72%)
  • debian-13-xfce_dom0-dispvm-gui-api (mean:7.833): 94.00 🟢 ( previous job: 98.62, improvement: 95.31%)
  • debian-13-xfce_dom0-dispvm-preload-2-api (mean:4.105): 49.26
  • debian-13-xfce_dom0-dispvm-preload-4-api (mean:3.781): 45.37
  • debian-13-xfce_dom0-dispvm-preload-2-gui-api (mean:4.169): 50.02
  • debian-13-xfce_dom0-dispvm-preload-4-gui-api (mean:3.893): 46.72
  • debian-13-xfce_dom0-vm-api (mean:0.033): 0.40 🟢 ( previous job: 0.59, improvement: 67.01%)
  • debian-13-xfce_dom0-vm-gui-api (mean:0.043): 0.51 🟢 ( previous job: 0.58, improvement: 88.60%)
  • fedora-42-xfce_dom0-dispvm-gui-api (mean:8.345): 100.14 🟢 ( previous job: 107.00, improvement: 93.59%)
  • fedora-42-xfce_dom0-dispvm-preload-2-api (mean:4.55): 54.59
  • fedora-42-xfce_dom0-dispvm-preload-4-api (mean:4.062): 48.74
  • fedora-42-xfce_dom0-dispvm-preload-2-gui-api (mean:4.693): 56.32
  • fedora-42-xfce_dom0-dispvm-preload-4-gui-api (mean:4.113): 49.36
  • whonix-workstation-18_dom0-dispvm-api (mean:9.005): 108.06 🟢 ( previous job: 117.52, improvement: 91.95%)
  • whonix-workstation-18_dom0-dispvm-gui-api (mean:9.589): 115.07 🟢 ( previous job: 130.38, improvement: 88.26%)
  • whonix-workstation-18_dom0-dispvm-preload-2-api (mean:5.419): 65.03
  • whonix-workstation-18_dom0-dispvm-preload-4-api (mean:4.765): 57.18
  • whonix-workstation-18_dom0-dispvm-preload-2-gui-api (mean:5.567): 66.80
  • whonix-workstation-18_dom0-dispvm-preload-4-gui-api (mean:4.657): 55.88
  • whonix-workstation-18_dom0-vm-api (mean:0.038): 0.46 🟢 ( previous job: 0.60, improvement: 76.49%)
  • whonix-workstation-18_dom0-vm-gui-api (mean:0.041): 0.49 🔻 ( previous job: 0.45, degradation: 108.46%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants