This issue is to discuss and track the work required to implement the ability to define and run system tests on windows hosts for integrations.

Prior context:

• Add ability to deploy custom elastic-agents on different OS or runtimes #787
• [system tests] Add support to deploy custom elastic-agents in VMs #829

Problem

Some integrations can run, or run exclusively on windows systems. Currently we lack the ability to run automated tests on windows platforms, neither locally or in CI. This leaves us blind under any possible problems that we may introduce, and we would only find out either performing tedious manual testing or once they are released.

Some of the integrations affected by this are (but maybe not limited to):

• Windows
• System (some data streams)
• network traffic
• IIS
• custom winlog input

Even if the number of affected integrations seems low compared to our total, this effort is still needed for both completeness and the fact that some of these are very widely used integrations.

Requirements

• elastic-package "officially" supports windows
  • Currently some things do not work as expected when running on windows ie Elastic Stack Up not working on Windows or Ubuntu WSL #1282, also some partners pointed out weird behaviour when using it under a native windows host. We might need to add testing to ensure it works properly on windows hosts.
• elastic-package supports running windows images for system tests on windows hosts (related to [system tests] Add support to deploy custom elastic-agents in VMs #829)
  • windows docker images should be able to run on windows hosts if docker is available
• A windows docker image needs to be built for elastic-agent so it can be used by the system tests
• Buildkite migration needs to be completed in integrations
  • Having this done will ease a lot the setup of a windows host and will prevent doubling the work for both Jenkins and Buildkite
• Buildkite windows agents need to be able to run docker
• We need a way to limit the amount of windows hosts required on CI, it might be something as easy as an explicit opt in list of packages or something elastic-package can detect dynamically.

This is a general overview and might be incomplete, would be interesting to gather feedback and ideas to be able to define a specific course of actions towards this.