Description
Please post all questions and issues on https://discuss.elastic.co/c/beats
before opening a Github Issue. Your questions will reach a wider audience there,
and if we confirm that there is a bug, then you can open a new issue.
For confirmed bugs, please report:
- Version: filebeat-5.0.0-beta1
- Operating System: os/x and redhat
- Steps to Reproduce:
A. populate this data(or something else, presumably) into some files :
Oct 19 08:31:02 server4 snort: alert_multiple_requests: ACTIVE
Oct 19 08:31:02 server4 snort: telnet_decode arguments:
Oct 19 08:31:02 server4 snort: snort startup succeeded
Oct 19 08:31:02 server4 snort: Ports to decode telnet on: 21 23 25 119
Oct 19 08:31:03 server4 snort: Snort initialization completed successfully
My files were like this, and the timestamp of course changed to current time for filebeat_test_1.log - filebeat_test_3.log.
B. Process these files with filebeat using a yml similar to this(change the paths):
filebeat:
prospectors:
-
paths:
- /input/filebeat_test*.log
tags: ["filebeat"]
scan_frequency: 10s
close_inactive: 1m
ignore_older: 2m
clean_inactive: 3m
document_type: "feed_filebeat_test"
registry_file: /path/to/my_registry
output:
file:
path: "/path/to/output"
filename: filebeat
rotate_every_kb: 1000
number_of_files: 7
to_files: true
path: /var/log
rotateeverybytes: 104857600
keepfiles: 7
level: debug
C. Wait a bit, until you see we've still got state, but have breached ignore_older. Like these lines:
2016-10-20T09:55:44-07:00 DBG Check file for harvesting: /input/filebeat_test_1.log
2016-10-20T09:55:44-07:00 DBG Ignore file because ignore_older reached: /input/filebeat_test_1.log
2016-10-20T09:55:44-07:00 DBG Check file for harvesting: /input/filebeat_test_2.log
2016-10-20T09:55:44-07:00 DBG Ignore file because ignore_older reached: /input/filebeat_test_2.log
2016-10-20T09:55:44-07:00 DBG Check file for harvesting: /input/filebeat_test_3.log
2016-10-20T09:55:44-07:00 DBG Ignore file because ignore_older reached: /input/filebeat_test_3.log
2016-10-20T09:55:44-07:00 DBG Prospector states cleaned up. Before: 3, After: 3
D. Stop filebeat
E. Add a new file like filebeat_test_4.log, same as 1-3.
F. Start filebeat, and on startup in the log, we should see:
2016-10-20T10:06:51-07:00 ERR State for /input/filebeat_test_1.log should have been dropped, but couldn't as state is not finished.
2016-10-20T10:06:51-07:00 ERR State for /input/filebeat_test_2.log should have been dropped, but couldn't as state is not finished.
2016-10-20T10:06:51-07:00 ERR State for /input/filebeat_test_3.log should have been dropped, but couldn't as state is not finished.
It doesn't seem to stop any data from flowing or do anything other than generating this message, but should probably be corrected.