metricbeat - metricbeat stops publishing events to Redis

[alextxm]

Metricbeat 6.0.0-rc1 stops publishing events to Redis (using the redis output module) after some time, depending on the events volume, but usually in the 4000-5000 events range.

Enabling debugging does not give any futher info; after reaching 4000-5000 events metricbeat continue running but no events are published to redis while the log is filled with INFO level messages like this one:
INFO Non-zero metrics in the last 30s: beat.memstats.memory_alloc=7872 beat.memstats.memory_total=7872

It should be noted that metricbeat successfully works when configured to use Logstash publishing instead (using the logstash output module).
The Redis output module configuration uses default values except for the following customized values: hosts, port, password, key. i've also tried raising the bulk_max_size value but nothing changed.

Tests were performed enablding only the system module, only the vsphere module and finally both of them; all the configurations triggered the problem; please also note the same configurations worked fine in 6.0.0-alpha2.

Tested with:

• metricbeat 6.0.0-rc1 on Windows Server 2008R2
• redis 3.0.2 on Ubuntu 16.04
• redis 4.0.2 on Ubuntu 16.04
• logstash 5.4.0 on Ubuntu 16.04

[alextxm]

I've done some more testing and it happens on both 6.0-beta1/beta2/rc1 but not alpha2 (alpha2 works fine). Publishing stops just after 4096 items as you can see per this screenshots taken from redis-cli.

P.S.: tests were performed with: only events from the vsphere module, only events with the system module, events from both modules; the screenshots are from tests with the vsphere modules.

Could please someone check this issue?
Thanks in advance

[urso]

Thanks for testing. Can you share your redis config? Are you using a list or a channel?

[urso]

I was able to reproduce the problem. It should be resolved with #5441. Thanks for testing.