Reading multiple files ?

[filippog]

Hi,
thanks for grok_exporter, looks really useful! I was wondering if there's support for reading multiple files within the same config/server ?

[fstab]

Currently grok_exporter only supports a single file, so if you want to monitor multiple files you need to start multiple grok_exporter instances. However, I plan to implement support for multiple log files within the next few weeks, as soon as I find the time. I'm leaving this issue open until it's done.

[errordeveloper]

Currently grok_exporter only supports a single file...

Which makes most sense in sidecar use-case 👍

[fstab]

In case anyone wonders why this is not supported yet: Implementing support for multiple log files might require only a few code changes, but there could be unexpected side effects and corner cases, so I would like to implement an extensive automated test first:

• What if events on different files occur at the same time? Are there any races or timing issues?
• What if a file being monitored is renamed, overwriting another file that is also being monitored?
• What are the implications if multiple files are on different filesystems vs on the same filesystem?
• ... there are many more things that should be tested

My plan is to write automated tests first, and then implement support for monitoring multiple files. I will only release it if I am sure it works even with strange corner cases. It might take another few weeks until I find the time.

[filippog]

@fstab thanks, sounds great! I agree watching multiple files isn't trivial to get right. Prometheus at least should contain some sample code that does this for file_sd_configs used with wildcards, for the filesystem-related part that is.

[smartinov]

@fstab is this still WIP? Is there like an "early access branch"?

[fstab]

It's still WIP, but I currently have very limited time. As soon as I find the time it will be the first feature I will work on (except there are bugs, bugs have higher priority). I am hoping I will find some time around Easter, but I can't really promise it. If someone wants to help out and create a pull request: please go ahead!

[fstab]

I created a branch multiple-logfiles and pushed an initial implementation. It works in a quick manual test on my machine, but it's not tested extensively.

The config is pretty simple: Set config_version: 3, and use inputs instead of input. Within inputs you can put a list of inputs, like this:

global:
  config_version: 3
inputs:
- type: file
  path: ./example/exim-rejected-RCPT-examples.log
- type: file
  path: ./example/exim-rejected-RCPT-examples2.log
...

By default, a new label input is added automatically to each metric containing the path of the config file (you can reconfigure that using input_label_name in the global section and input_label_value in the inputs section, see test code for examples).

Apart from tests, there are still some things to figure out.

• Would it be useful to support wildcards for file names in the config?
• Should metrics be restricted to specific files (as of now, all metrics are matched with all files)?

If you have feedback, please comment here.

[AKYD]

@fstab Thank you for your work on this.

Regarding your questions :

• I think wildcards are a nice feature

• Matches should be specified per file, as "error" in a file might not be interesting, or it might be that in a file "error" is counter and in another file is a gauge.

[mohsen0]

Hi @fstab

Would it be useful to support wildcards for file names in the config?
Should metrics be restricted to specific files (as of now, all metrics are matched with all files)?

I think having a folder and setting a wild card like "*.log" would be a better solution since the application log rotation settings sometimes zip the older logs. and they create logs with a timestamp appended to the file name, like appliction-9999_12_30_12:00:00.log

[pznamensky]

Wildcards would be very useful. Besides it would be great to specify wildcards for directories too.
Something like: /var/log/nginx/*/*.log

[roidelapluie]

I do not need wildcards but metrics per file is what I need.

[Perlovka]

Hi =) Any progress with the subject? It would be very useful feature.

[fstab]

I should really start working on it. I hope there will be some time for it in February.

[pete-leese]

Looking forward to wild card support too!