type assertion in Execd processor when reading managed process's output fails, and large ILP lines fail to be tokenized #15927
Assignees
Labels
area/execd
Issues related to execd or plugins that would be better suited to be used through execd
bug
unexpected problem or unintended behavior
plugin/parser
1. Request for new parser plugins 2. Issues/PRs that are related to parser plugins
Comments
nitti
changed the title
|
Next steps: check if the type assertion fails because the parser is wrapped. |
srebhan
added
area/execd
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Relevant telegraf.conf
Logs from Telegraf
System info
Telegraf 1.32.0, Docker engine 24.0.9 linux/amd64 running on MacOS 14.6.1 arm64 via Colima
Docker
No response
Steps to reproduce
token too long. Data is not emitted from theexecdprocessor.Expected behavior
The data should be processed using a stream parser. Data should be processed and emitted.
Actual behavior
Error
bufio.Scanner: token too longand data is not emitted.Additional info
issue
In the
execdprocessor plugin, if the managed process emits ILP that exceeds the maximum buffer size of thebufio.Scanner, it fails to parse and emit the data due to the token size exceeding the maximum 262,144 bytes (https://github.com/influxdata/telegraf/blob/v1.32.0/plugins/processors/execd/execd.go#L110-L127). There's a type assertion on the parser at the top ofcmdReadOut(https://github.com/influxdata/telegraf/blob/v1.32.0/plugins/processors/execd/execd.go#L105-L108), but in all cases (that I could see), the type of the parser is*models.RunningParser, so the assertion always fails and falls back to using a*bufio.Scannerto tokenize the input. In prior versions of Telegraf, this assertion would succeed and then hand off the input to a*influx.StreamParserwhich is able to handle large inputs.The parser is set here: https://github.com/influxdata/telegraf/blob/v1.32.0/config/config.go#L1118. The call to
addParser()always returns a*models.RunningParser. This type has a public fieldParserwhich has a reference to the underlying parser: https://github.com/influxdata/telegraf/blob/v1.32.0/models/running_parsers.go#L12Potentially a fix may look something like:
other info
This is the script (
processor.shas referred to in the Telegraf config) I threw together to emit ILP, it just cats a sample file when run:file
output.sampleshould contain one line of ILP data, and its length should exceed 262,144 bytes. example: https://gist.githubusercontent.com/nitti/ac86d986d77d1b2ac6d813fad6f13f5d/raw/758cfa849979b80ff27b45bd0b482b87d8628138/output.sampleThank you for your time Telegraf team.
The text was updated successfully, but these errors were encountered: