Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qmemman use memory hotplug #506

Merged
merged 6 commits into from
Oct 11, 2023
Merged

Conversation

marmarek
Copy link
Member

If VM supports memory hotplug, start it with maxmem=memory and then use
memory hotplug to (potentially) increase its memory. Qmemman almost had all
the needed parts - especially mem_set function already adjusts maxmem too.
The missing part was letting it know actual maxmem - do that via xenstore
(as qmemman uses xenstore heavily already).

The use of memory hotplug can be enabled (or disabled) explicitly via
'memory-hotplug' feature, or detected based on
'memory-hotplug-supported' file in dom0-provided kernel dir, or
'supported-feature.memory-hotplug' for VM-provided kernel.

Fixes QubesOS/qubes-issues#7956

@codecov
Copy link

codecov bot commented Dec 26, 2022

Codecov Report

Merging #506 (eda55dd) into master (c965d85) will increase coverage by 1.17%.
The diff coverage is 12.77%.

@@            Coverage Diff             @@
##           master     #506      +/-   ##
==========================================
+ Coverage   67.14%   68.31%   +1.17%     
==========================================
  Files          55       56       +1     
  Lines       11131    11157      +26     
==========================================
+ Hits         7474     7622     +148     
+ Misses       3657     3535     -122     
Flag Coverage Δ
unittests 68.31% <12.77%> (+1.17%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
qubes/qmemman/algo.py 90.90% <100.00%> (+90.90%) ⬆️
qubes/qmemman/domainstate.py 100.00% <100.00%> (ø)
qubes/tools/qmemmand.py 0.00% <0.00%> (ø)
qubes/vm/qubesvm.py 52.47% <68.42%> (+0.71%) ⬆️
qubes/qmemman/systemstate.py 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

qubes/vm/qubesvm.py Outdated Show resolved Hide resolved
qubes/vm/qubesvm.py Outdated Show resolved Hide resolved
Copy link
Contributor

@DemiMarie DemiMarie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammer nit

qubes/vm/qubesvm.py Outdated Show resolved Hide resolved
@marmarek marmarek force-pushed the qmemman-cleanup branch 2 times, most recently from 516b3e6 to 6b3584e Compare December 27, 2022 05:52
@qubesos-bot
Copy link

qubesos-bot commented Dec 28, 2022

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2023101004-4.2&flavor=pull-requests

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2023071104-4.2&flavor=update

  • system_tests_basic_vm_qrexec_gui

    • TC_20_AudioVM_Pulse_fedora-38-xfce: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...

    • TC_20_NonAudio_whonix-workstation-17: test_130_qrexec_filemove_disk_full (error)
      subprocess.CalledProcessError: Command 'qubes.WaitForSession' retur...

  • system_tests_pvgrub_salt_storage

    • TC_41_HVMGrub_fedora-38-xfce: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_splitgpg

  • system_tests_usbproxy

  • system_tests_qrexec

  • system_tests_devices

    • TC_00_List_debian-12-xfce: test_001_list_loop_mounted (failure)
      AssertionError: Device test-inst-vm:loop0 (/tmp/test.img) should no...
  • system_tests_vm_qrexec_gui_pipewire

    • pipewire_install: Failed (test died)
      # Test died: command '(set -o pipefail; qubesctl --skip-dom0 --max-...
  • system_tests_basic_vm_qrexec_gui_btrfs

    • TC_20_AudioVM_Pulse_fedora-38-xfce-pool: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_basic_vm_qrexec_gui_xfs

    • TC_20_AudioVM_Pulse_fedora-38-xfce-pool: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_basic_vm_qrexec_gui@hw1

    • TC_20_AudioVM_Pulse_fedora-38-xfce: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_20_AudioVM_Pulse_fedora-38-xfce-pool: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_dispvm

    • TC_20_DispVM_debian-12-xfce: test_030_edit_file (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • TC_20_DispVM_debian-12-xfce: test_100_open_in_dispvm (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • TC_20_DispVM_fedora-38-xfce: test_010_simple_dvm_run (failure)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError

  • system_tests_gui_tools@hw1

    • qubesmanager_templatemanager: unnamed test (unknown)
    • qubesmanager_templatemanager: Failed (test died)
      # Test died: no candidate needle with tag(s) 'template-change-vm-te...
  • system_tests_gui_interactive@hw1

    • simple_gui_apps: unnamed test (unknown)

    • simple_gui_apps: Failed (test died)
      # Test died: no candidate needle with tag(s) 'menu-vm-work' matched...

    • simple_gui_apps: unnamed test (unknown)

Failed tests

35 failures
  • system_tests_basic_vm_qrexec_gui

    • TC_20_AudioVM_Pulse_fedora-38-xfce: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...

    • TC_20_NonAudio_whonix-workstation-17: test_130_qrexec_filemove_disk_full (error)
      subprocess.CalledProcessError: Command 'qubes.WaitForSession' retur...

  • system_tests_pvgrub_salt_storage

    • [unstable] TC_41_HVMGrub_fedora-38-xfce: test_000_standalone_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

    • TC_41_HVMGrub_fedora-38-xfce: test_010_template_based_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

  • system_tests_splitgpg

  • system_tests_usbproxy

  • system_tests_qrexec

  • system_tests_network_ipv6

    • [unstable] VmIPv6Networking_fedora-38-xfce: test_540_ipv6_inter_vm (error)
      qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_devices

    • TC_00_List_debian-12-xfce: test_001_list_loop_mounted (failure)
      AssertionError: Device test-inst-vm:loop0 (/tmp/test.img) should no...
  • system_tests_vm_qrexec_gui_pipewire

    • pipewire_install: Failed (test died)
      # Test died: command '(set -o pipefail; qubesctl --skip-dom0 --max-...
  • system_tests_basic_vm_qrexec_gui_btrfs

    • TC_20_AudioVM_Pulse_fedora-38-xfce-pool: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_basic_vm_qrexec_gui_xfs

    • TC_20_AudioVM_Pulse_fedora-38-xfce-pool: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...

    • [unstable] TC_20_AudioVM_PipeWire_debian-12-xfce-pool: test_226_audio_playback_pipewire (failure)
      AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...

  • system_tests_basic_vm_qrexec_gui@hw1

    • TC_20_AudioVM_Pulse_fedora-38-xfce: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_basic_vm_qrexec_gui_ext4

    • TC_20_AudioVM_Pulse_fedora-38-xfce-pool: test_223_audio_play_hvm (failure)
      AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
  • system_tests_dispvm

    • TC_20_DispVM_debian-12-xfce: test_030_edit_file (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • TC_20_DispVM_debian-12-xfce: test_100_open_in_dispvm (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • TC_20_DispVM_fedora-38-xfce: test_010_simple_dvm_run (failure)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError

    • [unstable] TC_20_DispVM_whonix-workstation-17: test_030_edit_file (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

    • [unstable] TC_20_DispVM_whonix-workstation-17: test_100_open_in_dispvm (failure + cleanup)
      AssertionError: Timeout while waiting for disp[0-9]* window to show

  • system_tests_gui_tools@hw1

    • qubesmanager_templatemanager: unnamed test (unknown)
    • qubesmanager_templatemanager: Failed (test died)
      # Test died: no candidate needle with tag(s) 'template-change-vm-te...
  • system_tests_gui_interactive@hw1

    • simple_gui_apps: unnamed test (unknown)

    • simple_gui_apps: Failed (test died)
      # Test died: no candidate needle with tag(s) 'menu-vm-work' matched...

    • simple_gui_apps: unnamed test (unknown)

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/77326#dependencies

23 fixed
  • system_tests_network

  • system_tests_splitgpg

  • system_tests_manager

    • VMSettingsTest: test_26_default_dispvm (failure)
      self.assertTrue(self.vm.property_... AssertionError: False is not true
  • system_tests_qrexec

  • system_tests_network_ipv6

    • VmIPv6Networking_fedora-38: test_040_inter_vm (error)
      qubes.exc.QubesMemoryError: Not enough memory to start domain 'test...

    • VmIPv6Networking_fedora-38: test_203_fake_ip_inter_vm_allow (error)
      qubes.exc.QubesMemoryError: Not enough memory to start domain 'test...

  • system_tests_dispvm

  • system_tests_whonix

  • system_tests_guivm_gui_interactive

    • update_templates: Failed (test died)
      # Test died: command 'curl --form upload=@template-fedora-38-xfce-v...

Unstable tests

  • system_tests_suspend

    suspend/Failed (1/5 times with errors)
    • job 83088 # Test died: command 'true' timed out at /usr/lib/os-autoinst/autot...
    suspend/wait_serial (1/5 times with errors)
    • job 83088 # wait_serial expected: qr/p5~T5-\d+-/...
  • system_tests_basic_vm_qrexec_gui

    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (1/5 times with errors)
    • job 80971 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    TC_20_AudioVM_Pulse_debian-12-xfce/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 82899 qubes.exc.QubesVMError: Failed to attach audio input from dom0 to t...
    TC_20_AudioVM_Pulse_fedora-38-xfce/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 82367 AssertionError: frequency 3366.354981298878 not in specified range,...
    TC_20_NonAudio_whonix-gateway-17/test_300_bug_1028_gui_memory_pinning (1/5 times with errors)
    • job 82367 AssertionError: Dom0 window doesn't match VM window content, saved ...
  • system_tests_pvgrub_salt_storage

    TC_41_HVMGrub_fedora-38-xfce/test_000_standalone_vm (3/5 times with errors)
    • job 81879 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    • job 82004 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    • job 82379 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_usbproxy

    TC_20_USBProxy_core3_whonix-workstation-17/test_070_attach_not_installed_front (1/5 times with errors)
    • job 81882 qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...
  • system_tests_network_ipv6

    VmIPv6Networking_debian-12-xfce/test_113_reattach_after_provider_kill (1/5 times with errors)
    • job 82377 raise exceptions.TimeoutError() from exc... TimeoutError
    VmIPv6Networking_fedora-38-xfce/test_540_ipv6_inter_vm (1/5 times with errors)
    • job 80991 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • system_tests_network_updates

    TC_00_Dom0Upgrade_debian-12-xfce/test_001_update_check (1/3 times with errors)
    • job 82378 self.assertFalse(self.app.domains[... AssertionError: '1' is not false
    TC_00_Dom0Upgrade_fedora-38-xfce/test_001_update_check (1/3 times with errors)
    • job 82378 self.assertFalse(self.app.domains[... AssertionError: '1' is not false
  • system_tests_basic_vm_qrexec_gui_btrfs

    TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_220_audio_play_pulseaudio (1/5 times with errors)
    • job 81808 AssertionError: Command 'paplay --format=float32le --rate=44100 --c...
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 81995 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
  • system_tests_basic_vm_qrexec_gui_xfs

    TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_221_audio_rec_muted_pulseaudio (1/5 times with errors)
    • job 82370 AssertionError: source-output for VM test-inst-vm1 not found
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_226_audio_playback_pipewire (2/5 times with errors)
    • job 82370 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    • job 82902 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
    TC_20_NonAudio_whonix-gateway-17-pool/test_300_bug_1028_gui_memory_pinning (1/5 times with errors)
    • job 81871 AssertionError: Dom0 window doesn't match VM window content, saved ...
  • system_tests_basic_vm_qrexec_gui@hw1

    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (1/5 times with errors)
    • job 80971 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    TC_20_AudioVM_Pulse_debian-12-xfce/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 82899 qubes.exc.QubesVMError: Failed to attach audio input from dom0 to t...
    TC_20_AudioVM_Pulse_fedora-38-xfce/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 82367 AssertionError: frequency 3366.354981298878 not in specified range,...
    TC_20_NonAudio_whonix-gateway-17/test_300_bug_1028_gui_memory_pinning (1/5 times with errors)
    • job 82367 AssertionError: Dom0 window doesn't match VM window content, saved ...
  • system_tests_basic_vm_qrexec_gui_zfs

    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (2/5 times with errors)
    • job 81872 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    • job 82357 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop4 device to...
    TC_00_Basic/test_200_shutdown_event_race (1/5 times with errors)
    • job 82357 AssertionError: libvirt event impl drain timeout
    TC_00_Basic/test_202_udev_block_exclude_default (1/5 times with errors)
    • job 81872 raise exceptions.TimeoutError() from exc... TimeoutError
    TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 81872 AssertionError: frequency 3379.6121459556007 not in specified range...
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_227_audio_rec_muted_pipewire (1/5 times with errors)
    • job 82889 subprocess.CalledProcessError: Command 'pkill pw-record' returned n...
  • system_tests_basic_vm_qrexec_gui_ext4

    TC_30_Gui_daemon/test_000_clipboard (1/5 times with errors)
    • job 80996 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
    TC_00_Basic/test_120_start_standalone_with_cdrom_dom0 (1/5 times with errors)
    • job 81994 AssertionError: 1 != 0 : b'Timeout waiting for dom0:loop12 device t...
    TC_20_NonAudio_whonix-workstation-17-pool/test_130_qrexec_filemove_disk_full (1/5 times with errors)
    • job 82901 raise exceptions.TimeoutError() from exc... TimeoutError
    TC_20_AudioVM_Pulse_fedora-38-xfce-pool/test_222_audio_rec_unmuted_pulseaudio (1/5 times with errors)
    • job 82369 AssertionError: frequency 3252.494577006508 not in specified range,...
    TC_20_AudioVM_PipeWire_debian-12-xfce-pool/test_226_audio_playback_pipewire (1/5 times with errors)
    • job 81870 AssertionError: Command 'pw-play --format=f32 --rate=44100 --channe...
  • system_tests_dispvm

    TC_20_DispVM_whonix-gateway-17/test_010_simple_dvm_run (1/5 times with errors)
    TC_20_DispVM_whonix-workstation-17/test_010_simple_dvm_run (1/5 times with errors)
    • job 81874 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError
    TC_20_DispVM_whonix-gateway-17/test_020_gui_app (1/5 times with errors)
    TC_20_DispVM_fedora-38-xfce/test_030_edit_file (2/5 times with errors)
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 82011 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_whonix-gateway-17/test_030_edit_file (1/5 times with errors)
    TC_20_DispVM_whonix-workstation-17/test_030_edit_file (2/5 times with errors)
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 82011 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_fedora-38-xfce/test_100_open_in_dispvm (1/5 times with errors)
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    TC_20_DispVM_whonix-gateway-17/test_100_open_in_dispvm (1/5 times with errors)
    TC_20_DispVM_whonix-workstation-17/test_100_open_in_dispvm (2/5 times with errors)
    • job 81874 AssertionError: Timeout while waiting for disp[0-9]* window to show
    • job 82011 AssertionError: Timeout while waiting for disp[0-9]* window to show
  • system_tests_whonix

    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/Failed (1/5 times with errors)
    • job 82385 # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
  • system_tests_whonix@hw1

    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/ (1/5 times with errors)
    whonix_torbrowser/Failed (1/5 times with errors)
    • job 82385 # Test died: no candidate needle with tag(s) 'anon-whonix-tor-brows...
  • system_tests_suspend@hw1

    suspend/Failed (1/5 times with errors)
    • job 83088 # Test died: command 'true' timed out at /usr/lib/os-autoinst/autot...
    suspend/wait_serial (1/5 times with errors)
    • job 83088 # wait_serial expected: qr/p5~T5-\d+-/...

@marmarek marmarek force-pushed the qmemman-cleanup branch 2 times, most recently from beed5ae to 59ced43 Compare December 28, 2022 20:10
@marmarek
Copy link
Member Author

This may require some more work on the kernel side. VM started with initial 400MB (the default) cannot reach 4GB, it stops around 2924MB.

@marmarek
Copy link
Member Author

marmarek commented Aug 6, 2023

This may require some more work on the kernel side. VM started with initial 400MB (the default) cannot reach 4GB, it stops around 2924MB.

Should be fixed with QubesOS/qubes-core-libvirt#47

@marmarek marmarek force-pushed the qmemman-cleanup branch 2 times, most recently from 1280872 to b47cc52 Compare August 7, 2023 09:35
@marmarek marmarek marked this pull request as ready for review October 9, 2023 23:00
If VM supports memory hotplug, start it with maxmem=memory and then use
memory hotplug to (potentially) increase its memory. The hotplug maxmem
needs to be communicated to libvirt->libxl via <maxMemory> element, so
Xen will reserve appropriate amount of memory for internal accounting
(this currently is a patch in qubes package, not available upstream
yet). Qmemman almost had all the needed parts - especially mem_set
function already adjusts maxmem too. The missing part was letting it
know actual maxmem - do that via xenstore (as qmemman uses xenstore
heavily already).

The use of memory hotplug can be enabled (or disabled) explicitly via
'memory-hotplug' feature, or detected based on
'memory-hotplug-supported' file in dom0-provided kernel dir, or
'supported-feature.memory-hotplug' for VM-provided kernel.

Fixes QubesOS/qubes-issues#7956
This helps avoiding cyclic imports - some places (especially
qubes.vm.qubesvm) needs only some of the parts).
There is no sense operating on fractional bytes here.
In practice, it should be rounded to whole pages or even more, but that
require few more changes.
At least some tests are prerequisite for touching balancing algorithm.
When memory hotplug is used, Xen allocates metadata for the future
hotplugged memory at the VM start time. Make sure there is enough free
memory for that.
@marmarek marmarek merged commit 8ce2ef5 into QubesOS:master Oct 11, 2023
3 of 4 checks passed
@marmarek marmarek deleted the qmemman-cleanup branch January 29, 2024 02:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants