tests: adjust tests for Dovecot 2.4

[marmarek]

Dovecot 2.4 (used in Debian trixie) changed its configuration
significantly. Have two sections for the old and the new version.

QubesOS/qubes-issues#8841

[qubesos-bot]

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025080414-4.3&flavor=templates

Test run included the following:

• tests: adjust tests for Dovecot 2.4 #106 (89a97bd)
• tests: adjust tests for Dovecot 2.4 qubes-app-linux-split-gpg2#28 (QubesOS/qubes-app-linux-split-gpg2@1ed66d1)
• Don't use sudo to make it compatible with minimal template qubes-app-linux-usb-proxy#49 (QubesOS/qubes-app-linux-usb-proxy@340679d)
• Set default desktop background on Debian too qubes-artwork#51 (QubesOS/qubes-artwork@22f45be)
• debian: add hwdata to dependencies qubes-core-admin-client#378 (QubesOS/qubes-core-admin-client@5024b5b)
• tests: disable pvgrub test on debian-13 qubes-core-admin#709 (QubesOS/qubes-core-admin@db4ddeb)
• debian: include gir1.2-gtklayershell-0.1 in runtime deps too qubes-desktop-linux-menu#62 (QubesOS/qubes-desktop-linux-menu@4641be7)
• Adjust sys-gui packages for Debian 13 qubes-mgmt-salt-dom0-virtual-machines#78 (QubesOS/qubes-mgmt-salt-dom0-virtual-machines@494ab38)

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025061004-4.3&flavor=update

• system_tests_extra

  • TC_00_QVCTest_debian-13-minimal: test_010_screenshare (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • TC_00_QVCTest_debian-13-minimal: test_020_webcam (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

• system_tests_network

  • VmNetworking_debian-13-minimal: test_040_inter_vm (error + cleanup)
    raise TimeoutError from exc_val... TimeoutError

• system_tests_guivm_gui_interactive

  • simple_gui_apps: unnamed test (unknown)
  • simple_gui_apps: Failed (test died)
    # Test died: no candidate needle with tag(s) 'work-evince, work-atr...

Failed tests

5 failures

• system_tests_extra

  • TC_00_QVCTest_debian-13-minimal: test_010_screenshare (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

  • TC_00_QVCTest_debian-13-minimal: test_020_webcam (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

• system_tests_network

  • VmNetworking_debian-13-minimal: test_040_inter_vm (error + cleanup)
    raise TimeoutError from exc_val... TimeoutError

• system_tests_guivm_gui_interactive

  • simple_gui_apps: unnamed test (unknown)
  • simple_gui_apps: Failed (test died)
    # Test died: no candidate needle with tag(s) 'work-evince, work-atr...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/142375#dependencies

9 fixed

• system_tests_extra

  • TC_10_Thunderbird_whonix-workstation-17: test_000_send_receive_default (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

  • TC_10_Thunderbird_whonix-workstation-17: test_010_send_receive_inline_signed_only (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

  • TC_10_Thunderbird_whonix-workstation-17: test_020_send_receive_inline_with_attachment (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

  • TC_00_QVCTest_whonix-workstation-17: test_010_screenshare (failure)
    AssertionError: 1 != 0 : Timeout waiting for /dev/video0 in test-in...

• system_tests_splitgpg

  • TC_10_Thunderbird_whonix-workstation-17: test_000_send_receive_default (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

  • TC_10_Thunderbird_whonix-workstation-17: test_010_send_receive_inline_signed_only (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

  • TC_10_Thunderbird_whonix-workstation-17: test_020_send_receive_inline_with_attachment (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

• system_tests_audio

  • TC_20_AudioVM_Pulse_debian-12-xfce: test_252_audio_playback_audiovm_switch_hvm (failure)
    AssertionError: only silence detected, no useful audio data

  • TC_20_AudioVM_Pulse_whonix-workstation-17: test_225_audio_rec_unmuted_hvm (failure)
    AssertionError: too short audio, expected 10s, got 7.95233560090703...

Unstable tests

Performance Tests

Performance degradation:

No issues

Remaining performance tests:

No remaining performance tests