Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metric for discarded lines to file source #14321

Open
bruceg opened this issue Sep 7, 2022 · 1 comment
Open

Add metric for discarded lines to file source #14321

bruceg opened this issue Sep 7, 2022 · 1 comment
Labels
domain: observability Anything related to monitoring/observing Vector source: file Anything `file` source related source: kubernetes_logs Anything `kubernetes_logs` source related

Comments

@bruceg
Copy link
Member

bruceg commented Sep 7, 2022

The file source increments a counter when it discards events (namely component_discarded_events_total). However, it may discard source data before they are translated into events. For example, if a source line is longer than max_line_bytes, it is discarded within the file-server lib and, while an warning log is emitted, no counter is incremented to provide evidence.

This condition should result in a counter being incremented, either component_errors_total with the notion that it represents malformed input data, or some other name if that is not appropriate.

Ref: no metric on File source on max_byte_line exceeded #14144

@bruceg bruceg added source: file Anything `file` source related domain: observability Anything related to monitoring/observing Vector labels Sep 7, 2022
@bruceg
Copy link
Member Author

bruceg commented Sep 8, 2022

Noting here that this should also be reflected in the kubernetes_logs source. Ref: #14144 (comment)

@bruceg bruceg added the source: kubernetes_logs Anything `kubernetes_logs` source related label Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: observability Anything related to monitoring/observing Vector source: file Anything `file` source related source: kubernetes_logs Anything `kubernetes_logs` source related
Projects
None yet
Development

No branches or pull requests

1 participant