Description
(discussed here: https://discuss.elastic.co/t/metricbeat-wmi-cpu-usage/69553)
We want to use metricbeat to monitor processes on our Windows servers.
The problem is that we have 200+ processes (all of which we want to monitor) which makes WmiPrvSE.exe consume 90%+ of one CPU core for at least 3-4 minutes. After those 3-4 minutes CPU is back to normal as metricbeat (as explained by Andrew Kroh; AFAIU it's #1043) is using a cached copy of the cmdline after the first fetch for a given PID.
It would be great if Metricbeat somehow allowed user to specify (s)he does not need the system.process.cmdline (e.g. we don't) in order to avoid that dramatic WMI performance overhead.
While CPU usage indeed decreases for just 3-4 minutes after processes are started there still can be - pretty rare I admit it - situations when we don't have those 3-4 minutes. E.g. our processes provide throughput&latency-critical PROD services and that initial delay is only acceptable for us as we start processes ahead of time. So, if it's not a big change and there is a simple and clean way to add a config option to avoid querying WMI, please do it.
As a proof-of-concept we've patched the Metricbeat to not query WMI. Here are the results:
Original version:
Patched version:
Below is a simple batch file that can be used for testing (remove the .txt extension). Note it may be the case that the problem only manifests itself when processes have pretty long command lines (which our PROD processes do). So, the spawn_dummy_processes.bat batch file given below has some quick-and-dirty procedure of spawning processes with long command lines.
spawn_dummy_processes.bat.txt