Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SplunkHecReceiver] Allows users to disable line breaking for Raw endpoint #22788

Closed
splunkericl opened this issue May 25, 2023 · 2 comments · Fixed by #22799
Closed

[SplunkHecReceiver] Allows users to disable line breaking for Raw endpoint #22788

splunkericl opened this issue May 25, 2023 · 2 comments · Fixed by #22799
Assignees
Labels
enhancement New feature or request receiver/splunkhec

Comments

@splunkericl
Copy link
Contributor

Component(s)

receiver/splunkhec

Is your feature request related to a problem? Please describe.

The HEC receiver handleRaw method uses a bufio Scanner that uses the default splitFunc. As an user of splunk hec receiver, I want an option to configure the bufio Scanner to disable line breaking functionality.

Background

Our team has much more extensive line breaking functionality(and will need to support more) so we want to handle line breaking inside our processor. If the events are line broken already at the receiver, our processor can't properly line break.

  • an example is if the event abc\r\n\r\nefg comes in, the user might want to configure to split based on \r\n\r\n but bufio scanner would behave differently.

So instead, we want line breaking to be disabled in receiver all completely.

Describe the solution you'd like

We want to add an option in the config to disable line breaking completely. So the whole raw event will be sent as one single log record.

Add an option in splunkhecreceiver:

type Config struct {
    DisableLineBreaking bool
}

inside handleRaw method:

var neverSplitFunc = func(data []byte, atEOF bool) (int, []byte, error) {
	if atEOF {
		return len(data), data, nil
	}
	return 0, nil, nil
}

sc := bufio.NewScanner(bodyReader)
if c.DisableLineBreaking {
    sc.Split(neverSplitFunc)
}

Describe alternatives you've considered

N/A

Additional context

No response

@splunkericl splunkericl added enhancement New feature or request needs triage New item requiring triage labels May 25, 2023
@github-actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@atoulme
Copy link
Contributor

atoulme commented May 25, 2023

Understood. Is a boolean flag the right config or would you foresee different line breaking strategies?

It looks like you answered the question actually - I'll work on this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request receiver/splunkhec
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants