KVM HyperV Enlightment for Improved Windows Server 2008+ Performance #2870

kiwiflyer · 2018-10-01T22:47:24Z

Description

Windows has support for several paravirt features that it will use when running on Hyper-V, Microsoft's hypervisor. These features are called enlightenments. Many of the features are similar to paravirt functionality that exists with Linux on KVM (virtio, kvmclock, PV EOI, etc.)

Nowadays QEMU/KVM can also enable support for several Hyper-V enlightenments. When enabled, Windows VMs running on KVM will use many of the same paravirt optimizations they would use when running on Hyper-V.

A number of years ago, a PR was introduced that added a good portion of the code to enable this feature set, but it was never completed. This PR enables the existing features. The previous patch set detailed in #1013 also included the tests.

By selecting Windows PV, the enlightenment additions will be applied to the libvirt configuration. This is support on Windows Server 2008 and beyond, so all currently supported versions of Windows Server.

In our testing, we've seen benchmark improvements of around 20-25% running on Centos 7 hosts and it is also supported on Centos/RHEL 6.5 and later. Testing on Ubuntu would be appreciated.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

How Has This Been Tested?

We've tested this in a lab and QA environment. I'll post some screenshots related to the libvirt xml changes and also provide some performance metric information over the next 24 hours.

Checklist:

I have read the CONTRIBUTING document.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
##Testing:
I have added tests to cover my changes. (not required as the previous PR included the tests)
All relevant new and existing integration tests have passed.
A full integration testsuite with all test that can run on my environment has passed.

rafaelweingartner · 2018-10-02T02:59:12Z

...ypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java

-         *    features.addHyperVFeature(hyv);
-         * }
-         */
+        //for rhel 6.5 and above, hyperv enlightenment feature is added


Thanks for the PR @kiwiflyer!

Would you mind extracting the code 2105-2114 to a method, document it, and then create a unit test case?

This createVMFromSpec is too big already.

borisstoyanov · 2018-10-02T07:43:10Z

@blueorangutan package

blueorangutan · 2018-10-02T07:43:57Z

@borisstoyanov a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2018-10-02T08:17:55Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2326

borisstoyanov · 2018-10-02T08:37:38Z

@blueorangutan test

blueorangutan · 2018-10-02T08:37:57Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

GabrielBrascher

@rafaelweingartner point (extract to a new method) seems fair to me. Code LGTM.
Thanks for the PR @kiwiflyer!

blueorangutan · 2018-10-02T16:15:41Z

Trillian test result (tid-3051)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 25545 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2870-t3051-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_multipleips_per_nic.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 66 look OK, 3 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_nic_secondaryip_add_remove	`Error`	32.63	test_multipleips_per_nic.py
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	1143.81	test_privategw_acl.py
test_04_rvpc_network_garbage_collector_nics	`Failure`	497.31	test_vpc_redundant.py

kiwiflyer · 2018-10-02T22:35:46Z

Ok, thanks for the feedback guys. I'll refactor this when I have some time later this week.

borisstoyanov

LGTM

rohityadavcloud

Lgtm

kiwiflyer · 2018-10-19T17:36:24Z

libvirt XML when enabled -

kiwiflyer · 2018-10-19T17:36:48Z

libvirt XML when disabled -

rohityadavcloud · 2018-10-19T18:12:55Z

...ypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java

    }

+    protected void enlightenWindowsVm(VirtualMachineTO vmTO, FeaturesDef features) {
+    // If OS is Windows PV, then enable the features. Features supported on Windows 2008 and later


minor nit - Fix indent of comment to match that of statements inside method.

rohityadavcloud · 2018-10-19T18:14:00Z

@kiwiflyer can you see travis error and fix the checkstyle issue.

rohityadavcloud · 2018-10-19T18:39:24Z

@kiwiflyer still the same error, please check your changes:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:3.0.0:check (cloudstack-checkstyle) on project cloud-plugin-hypervisor-kvm: Failed during checkstyle execution: There is 1 error reported by Checkstyle 8.7 with cloud-style.xml ruleset. -> [Help 1]

andrijapanic-dont-use-this-one · 2018-10-21T22:25:46Z

Simon, a question here (since we have been using this internally in production for 1.5+years, but never pushed upstream...)

Why don't implement this on all Windows* OS types, not just Windows PV - reason being, that, afaik, this is not related only to paravirtualization (Windows PV vs "regular" WIndows OS type in CloudStack), and the main reason for us was NOT the performance (which is there, as you described) - but mainly to avoid the very frequent BSOD with Windows 2008 with heavily loaded MSSQL servers and such.

I believe it makes sense to implement for all OS types matching "Windows*"

And we have been using this on Ubuntu 14.04 hosts all the time (means tested on Ubuntu, as you asked)

hv_relaxed vs. BSOD - slide 15:
https://www.linux-kvm.org/images/0/0a/2012-forum-kvm_hyperv.pdf

kiwiflyer · 2018-10-23T11:55:27Z

@andrijapanic I'm open to changing it to use all Windows OS types. My mine reason for picking one was some backwards compatibility. At least in the KVM world, I'm unsure why we continue to maintain tons of different OS types/names, as running any OS without virtio is extremely painful in terms of performance.

andrijapanic-dont-use-this-one · 2018-10-23T12:04:09Z

I agree on VirtIO stuff absolutely. It's just that, even with all FAQ and so on, still end-user sometimes continue to use specific a Windows OS type instead of Windows PV, so it makes sense to solve all Windows OS types with this nice patch.

As for future support of OS types inside ACS, that is a good point, but doesn't warrant not applying patch for all of Win OS types, in my opinion.

rohityadavcloud · 2018-10-25T01:23:44Z

@kiwiflyer I'll proceed with merging as I agree that for sake of backward compatibility let's avoiding having the feature enabled by default for any windows guest. If we decide to make it available for all windows guests, please submit another PR separately. Thanks.

…nments. apache#2870

…nments. apache#2870 (#62)

KVM HyperV Enlightment Feature Completion

d078130

kiwiflyer added type:enhancement component:kvm labels Oct 1, 2018

kiwiflyer added this to the 4.12.0.0 milestone Oct 1, 2018

kiwiflyer requested review from GabrielBrascher, pdion891 and swill October 1, 2018 22:47

kiwiflyer closed this Oct 2, 2018

kiwiflyer reopened this Oct 2, 2018

rafaelweingartner approved these changes Oct 2, 2018

View reviewed changes

GabrielBrascher approved these changes Oct 2, 2018

View reviewed changes

borisstoyanov approved these changes Oct 3, 2018

View reviewed changes

rohityadavcloud approved these changes Oct 4, 2018

View reviewed changes

rafaelweingartner force-pushed the master branch from 1fa4f10 to fcc87d9 Compare October 7, 2018 21:14

Move enlightenment to method

890db0f

rohityadavcloud reviewed Oct 19, 2018

View reviewed changes

rohityadavcloud approved these changes Oct 19, 2018

View reviewed changes

Check style indent fix

44cebb1

Check style indent fix

b158831

Remove unused import

132e04f

kiwiflyer closed this Oct 19, 2018

kiwiflyer reopened this Oct 19, 2018

rohityadavcloud merged commit c4b621a into apache:master Oct 25, 2018

sureshanaparti mentioned this pull request Apr 15, 2021

protect against stray snapshot-details without snapshot shapeblue/cloudstack#70

Draft

12 tasks

nlgordon added a commit to ippathways/cloudstack that referenced this pull request Nov 22, 2022

Applying PR 2870 from upstream. This adds additional hyper-v enlighte…

d6cd406

…nments. apache#2870

nlgordon mentioned this pull request Nov 22, 2022

AC-124 Applying PR 2870 from upstream ippathways/cloudstack#62

Merged

Slair1 pushed a commit to ippathways/cloudstack that referenced this pull request Nov 27, 2022

Applying PR 2870 from upstream. This adds additional hyper-v enlighte…

6defcc8

…nments. apache#2870 (#62)

KVM HyperV Enlightment for Improved Windows Server 2008+ Performance #2870

KVM HyperV Enlightment for Improved Windows Server 2008+ Performance #2870

Uh oh!

Conversation

kiwiflyer commented Oct 1, 2018

Description

Types of changes

How Has This Been Tested?

Checklist:

Uh oh!

rafaelweingartner Oct 2, 2018

Choose a reason for hiding this comment

Uh oh!

borisstoyanov commented Oct 2, 2018

Uh oh!

blueorangutan commented Oct 2, 2018

Uh oh!

blueorangutan commented Oct 2, 2018

Uh oh!

borisstoyanov commented Oct 2, 2018

Uh oh!

blueorangutan commented Oct 2, 2018

Uh oh!

GabrielBrascher left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Oct 2, 2018

Uh oh!

kiwiflyer commented Oct 2, 2018

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

rohityadavcloud left a comment

Choose a reason for hiding this comment

Uh oh!

kiwiflyer commented Oct 19, 2018

Uh oh!

kiwiflyer commented Oct 19, 2018

Uh oh!

rohityadavcloud Oct 19, 2018

Choose a reason for hiding this comment

Uh oh!

rohityadavcloud commented Oct 19, 2018

Uh oh!

rohityadavcloud commented Oct 19, 2018

Uh oh!

andrijapanic-dont-use-this-one commented Oct 21, 2018

Uh oh!

kiwiflyer commented Oct 23, 2018

Uh oh!

andrijapanic-dont-use-this-one commented Oct 23, 2018

Uh oh!

rohityadavcloud commented Oct 25, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants