Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Omit multiline pattern from log #26381

Closed
haimrubinstein opened this issue Sep 4, 2023 · 6 comments
Closed

Omit multiline pattern from log #26381

haimrubinstein opened this issue Sep 4, 2023 · 6 comments
Labels
enhancement New feature or request pkg/stanza receiver/udplog Stale

Comments

@haimrubinstein
Copy link
Contributor

Component(s)

pkg/stanza, receiver/udplog

Is your feature request related to a problem? Please describe.

We have a scenario where we use OTEL in a very high-scale environment. We have solid concerns that we will hit our network limits. TO reduce network traffic we tried sending multiple logs over 1 packet using UDP receiver and multiline configuration. During the tests, we've noticed that the multiline delimiter is included in the log record.
For example, if we send set line_end_pattern to be 'PATTERN' and we send a UDP packet: log1 PATTERN log2, the final result is an array of 2 records [log1 PATTERN , log2], when what we actually need is [log1, log2].

Describe the solution you'd like

I would like to add a flag to the multiline config that if set to true will omit the pattern from the logs.
If the solution is OK with you I can open a pull request.

Describe alternatives you've considered

No response

Additional context

No response
@haimrubinstein haimrubinstein added enhancement New feature or request needs triage New item requiring triage labels Sep 4, 2023
@haimrubinstein
Copy link
Contributor Author

@djaglowski I'd be glad to hear your opinion.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 4, 2023

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@djaglowski
Copy link
Member

@haim6678, makes sense to me. I'd be happy to review a PR.

@haimrubinstein
Copy link
Contributor Author

@haim6678, makes sense to me. I'd be happy to review a PR.

Published my PR :)
https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/26460/files

@github-actions github-actions bot mentioned this issue Sep 6, 2023
Open
@crobert-1 crobert-1 removed the needs triage New item requiring triage label Sep 6, 2023
@haimrubinstein haimrubinstein mentioned this issue Sep 6, 2023
Merged
djaglowski pushed a commit that referenced this issue Sep 19, 2023
@haimrubinstein
Feature/harubins/add multiline pattern config (#26460)
e2e212c 
**Description:** Add a flag to the multiline config that if set to true
will omit the pattern from the logs.

**Link to tracking Issue:**
#26381

**Testing:** manual testing and  unit tests in multile_test file.
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 6, 2023

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Nov 6, 2023
jmsnll pushed a commit to jmsnll/opentelemetry-collector-contrib that referenced this issue Nov 12, 2023
@haimrubinstein @jmsnll
Feature/harubins/add multiline pattern config (open-telemetry#26460)
7a570bf 
**Description:** Add a flag to the multiline config that if set to true
will omit the pattern from the logs.

**Link to tracking Issue:**
open-telemetry#26381

**Testing:** manual testing and  unit tests in multile_test file.
@djaglowski
Copy link
Member

resolved by #26460

@github-actions github-actions bot mentioned this issue Jul 1, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request pkg/stanza receiver/udplog Stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@djaglowski @haimrubinstein @crobert-1