Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.18](backport #42797) [fbreceiver] - Fix batcher's configuration #42828

Open
wants to merge 1 commit into
base: 8.18
Choose a base branch
from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Feb 21, 2025

Proposed commit message

As per batcher's documentation, there are two configurations related to flushing:

  • min_size_items
    • Minimum number of log records/metric points in a request to immediately trigger a batcher flush.
    • For eg. if a plog.Log has more records than min_size_items then it is immediately forwarded to the exporter.
  • max_size_items
    • Maximum log records/metric points per request.
    • For eg. if a plog.Log has more records than max_size_items then it is split into multiple requests.
    • This doesn't necessarily mean that it will be sent to exporter. That decision will be based on flush timeout or min_size_items.

This PR sets min_size_items to 0. This ensures both of the following conditions are satisfied:

  1. No plog.Log contains more than bulk_max_size items.
  2. Batcher flushes the data as soon it gets it
    This is an automatic backport of pull request [fbreceiver] - Fix batcher's configuration #42797 done by Mergify.

* fix: batcher min_size_items

* update tests

* set min_size_items to 0

* fix tests

* fix tests

* fix test

(cherry picked from commit 2f3df16)

# Conflicts:
#	libbeat/otelbeat/beatconverter/beatconverter_test.go
#	libbeat/otelbeat/oteltranslate/outputs/elasticsearch/config_otel_test.go
@mergify mergify bot requested a review from a team as a code owner February 21, 2025 12:58
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Feb 21, 2025
@mergify mergify bot requested review from belimawr and leehinman and removed request for a team February 21, 2025 12:58
Copy link
Contributor Author

mergify bot commented Feb 21, 2025

Cherry-pick of 2f3df16 has failed:

On branch mergify/bp/8.18/pr-42797
Your branch is up to date with 'origin/8.18'.

You are currently cherry-picking commit 2f3df1694.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   libbeat/otelbeat/oteltranslate/outputs/elasticsearch/config_otel.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   libbeat/otelbeat/beatconverter/beatconverter_test.go
	both modified:   libbeat/otelbeat/oteltranslate/outputs/elasticsearch/config_otel_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 21, 2025
@github-actions github-actions bot added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Feb 21, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport conflicts There is a conflict in the backported pull request Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants