-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[receiver/statsd] Add Option to Aggregate on IP/Host #34851
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
povilasv
reviewed
Aug 27, 2024
receiver/statsdreceiver/internal/protocol/statsd_parser_test.go
Outdated
Show resolved
Hide resolved
receiver/statsdreceiver/internal/protocol/statsd_parser_test.go
Outdated
Show resolved
Hide resolved
povilasv
approved these changes
Aug 27, 2024
atoulme
approved these changes
Sep 10, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, needs codeowner review.
dmitryax
approved these changes
Sep 11, 2024
dmitryax
added
the
ready to merge
Code review completed; ready to merge by maintainers
label
Sep 11, 2024
jriguera
pushed a commit
to springernature/opentelemetry-collector-contrib
that referenced
this pull request
Oct 4, 2024
…34851) **Description:** <Describe what has changed.> The `statsdreceiver` only aggregates metrics on `protocol+host+ip`, this leads to issues or inconsistencies when dealing with clients that constantly switch tcp/udp ports. To address the issue, this PR adds a configuration option `enableIPOnlyAggregation` that allows the use to specify if they want to aggregate on the `IP` instead of `IP+Port`. For example: _otel_config.yaml:_ ```yaml receivers: statsd: endpoint: "0.0.0.0:8125" enable_metric_type: true is_monotonic_counter: false aggregation_interval: 10s enable_ip_only_aggregation: true # <-- enable ip only aggregation timer_histogram_mapping: - statsd_type: "timing" observer_type: "histogram" histogram: max_size: 50 exporters: debug: verbosity: detailed service: pipelines: metrics: receivers: - statsd exporters: - debug ``` _run:_ ```sh STATSD_HOST="localhost" STATSD_PORT=8125 for port in {10000..10010}; do echo -n "my.metric:1|c" | nc -w 1 -u -p $port ${STATSD_HOST} ${STATSD_PORT} echo "Sent from port $port" done ``` _result:_ ``` 2024-08-26T23:36:00.224+0200 info ResourceMetrics #0 Resource SchemaURL: ScopeMetrics #0 ScopeMetrics SchemaURL: InstrumentationScope otelcol/statsdreceiver 0.103.0-dev Metric #0 Descriptor: -> Name: -n my.metric -> Description: -> Unit: -> DataType: Sum -> IsMonotonic: false -> AggregationTemporality: Delta NumberDataPoints #0 Data point attributes: -> metric_type: Str(counter) StartTimestamp: 2024-08-26 21:35:50.223101 +0000 UTC Timestamp: 2024-08-26 21:36:00.224252 +0000 UTC Value: 7 {"kind": "exporter", "data_type": "metrics", "name": "debug"} 2024-08-26T23:36:10.224+0200 info MetricsExporter {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1} 2024-08-26T23:36:10.224+0200 info ResourceMetrics #0 Resource SchemaURL: ScopeMetrics #0 ScopeMetrics SchemaURL: InstrumentationScope otelcol/statsdreceiver 0.103.0-dev Metric #0 Descriptor: -> Name: -n my.metric -> Description: -> Unit: -> DataType: Sum -> IsMonotonic: false -> AggregationTemporality: Delta NumberDataPoints #0 Data point attributes: -> metric_type: Str(counter) StartTimestamp: 2024-08-26 21:36:00.224252 +0000 UTC Timestamp: 2024-08-26 21:36:10.224607 +0000 UTC Value: 4 {"kind": "exporter", "data_type": "metrics", "name": "debug"} ``` Instead of generating 11 metrics for each port that was used to send, only 2 metrics are blocks are returned, who's values total 11. ![2024-08-26 23 44 15](https://github.com/user-attachments/assets/6b8a89d1-186e-4257-9c82-90c5f9d14f98) **Link to tracking Issue:** open-telemetry#23809 **Testing:** - [x] Added unit tests **Documentation:** <Describe the documentation added.> - [x] Added information to the statsdreceiver `README.md` describing the option. --------- Co-authored-by: Povilas Versockas <povilas.versockas@coralogix.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
ready to merge
Code review completed; ready to merge by maintainers
receiver/statsd
statsd related issues
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
The
statsdreceiver
only aggregates metrics onprotocol+host+ip
, this leads to issues or inconsistencies when dealing with clients that constantly switch tcp/udp ports. To address the issue, this PR adds a configuration optionenableIPOnlyAggregation
that allows the use to specify if they want to aggregate on theIP
instead ofIP+Port
.For example:
otel_config.yaml:
run:
result:
Instead of generating 11 metrics for each port that was used to send, only 2 metrics are blocks are returned, who's values total 11.
Link to tracking Issue: #23809
Testing:
Documentation:
README.md
describing the option.