Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/stanza] csv_input operator should support auto detection of header #10275

Open
djaglowski opened this issue Mar 22, 2022 · 2 comments
Open
Labels
enhancement New feature or request never stale Issues marked with this label will be never staled and automatically removed priority:p3 Lowest

Comments

@djaglowski
Copy link
Member

CSV format allows headers to be specified as the first line in a file. Ideally, this library should support parsing of such files.

The csv_parser is only responsible for parsing one line of a file at a time, meaning that it cannot handle this basic use case on its own.

A csv_input operator could follow the example of the syslog_input operator, in that is wraps multiple operators together into a more useful form. In this case, it would wrap file_input and csv_parser. It should also provide a config flag such as first_line_headers to indicate whether or not the first line should be interpreted as headers.

@djaglowski djaglowski added the enhancement New feature or request label Mar 22, 2022
@djaglowski djaglowski changed the title csv_input operator [pkg/stanza] csv_input operator May 24, 2022
@djaglowski djaglowski transferred this issue from open-telemetry/opentelemetry-log-collection May 24, 2022
@djaglowski djaglowski added the priority:p3 Lowest label May 24, 2022
@Dylan-M
Copy link

Dylan-M commented Aug 18, 2022

I would point out that it isn't always the first line that is the header. For example, from IIS 10, it is the 4th line, and it has a "#Fields:" prefix:

#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2022-07-21 00:00:02
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

@djaglowski djaglowski changed the title [pkg/stanza] csv_input operator [pkg/stanza] csv_input operator should support auto detection of header Aug 19, 2022
@github-actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

@github-actions github-actions bot added the Stale label Nov 10, 2022
@djaglowski djaglowski added never stale Issues marked with this label will be never staled and automatically removed and removed Stale labels Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request never stale Issues marked with this label will be never staled and automatically removed priority:p3 Lowest
Projects
None yet
Development

No branches or pull requests

2 participants