WPF Builds fail intermittently due to timeouts

[vatsan-madhavan]

WPF builds fail intermittently due to timeouts when Run Developer Regression Tests on Helix Machine stage fails to start within its timeout period (60 mins).

Example of a failed build: https://dev.azure.com/dnceng/internal/_build/results?buildId=224164

=

_TestHelixAgentPool in eng\pipeline.yml is currently set to Windows.10.Amd64.Open%3bWindows.7.Amd64.Open%3bWindows.10.Amd64.ServerRS5.Open

It should be changed to Windows.10.Amd64.Open%3bWindows.7.Amd64.Open%3bWindows.10.Amd64.Client19H1.Open.

In short, stop running tests on ServerRS5. It's a "server core" machine machine, and seems to have problems running tests right now. Change it to Client19H1.

[vatsan-madhavan]

/cc @MattGal fyi. If you have an issue in core-eng to track fixup of the ServerRS5 pool, please link let us know. We may want to use it again in near-future.

[vatsan-madhavan]

@merriemcgaw, @zsd4yr, are you testing on Windows Server Core SKU's ?

[MattGal]

@vatsan-madhavan I think the machines are actually OK they just take an insane amount of time to provision (likely related to having IIS, SQL server, Git, JDK, and Docker on them?). I filed https://github.com/dotnet/core-eng/issues/6710 but closed it.

My comments before remain; there's literally no reason (though it might work) to run WPF apps on a Server RS5 machine.

[ryalanms]

For reference, here is the MUX configuration file:

https://github.com/microsoft/microsoft-ui-xaml/blob/master/build/AzurePipelinesTemplates/MUX-RunHelixTests-Job.yml

[vatsan-madhavan]

My comments before remain; there's literally no reason (though it might work) to run WPF apps on a Server RS5 machine.

It's a long discussion, but there may be good reasons to run tests on Server Core editions. Perhaps not just yet though.

Our customers have been asking for official support on server core for a while, and most of WPF "works" on Server Core (and even WinPE!) reasonably well - and has done so for long time now. We don't offer official (or any) support on those OS editions because we ourselves don't have a clear picture of all the failure-modes that are possible, and haven't done any work to fail gracefully, harden the product etc. That said, running tests on server core is certainly not a "literally no reason" type of scenario :-) There is even a Server Core App Compatibility Feature On Demand package, BTW.

[vatsan-madhavan]

The PR build to fix this also failed for the same reason :|

https://dev.azure.com/dnceng/public/_build/results?buildId=224573

[vatsan-madhavan]

@MattGal, I tried again and the build finished after around 46 mins.
Our tests should really only need a few minutes to run (<5, IIRC). Is there anything we can do to make this quicker - esp. not run up so close to the timeout?

https://dev.azure.com/dnceng/public/_build/results?buildId=224573

https://mc.dot.net/#/user/dotnet~2Fwpf/dotnet~2Fwpf~2Frefs~2Fpull~2F953~2Fmerge/tests~2Fdrt/20190613.18