Queued Retry is incompatible with Prometheus Remote Write #2949
Labels
bug
Something isn't working
Comments
This was referenced Apr 16, 2021
tigrannajaryan
pushed a commit
that referenced
this issue
Apr 19, 2021
Prometheus remote write (PWR) exporter is supporting queued retry out of the box, but due to the strict requirements of PWR, the queue causes "out of order sample" errors. The queue won't be suitable for Prometheus unless it has a way to shard the data by timeseries. This is not going to break the existing users because if they have this feature enabled, they already can't use the PRW exporter correctly. Fixes #2949.
hughesjj
pushed a commit
to hughesjj/opentelemetry-collector
that referenced
this issue
Apr 27, 2023
* Update exception messages in install.ps1 Edited exception messages for style and clarity. * Update internal/buildscripts/packaging/installer/install.ps1 Co-authored-by: Pablo Collins <pcollins@splunk.com> * Update internal/buildscripts/packaging/installer/install.ps1 Co-authored-by: Pablo Collins <pcollins@splunk.com> * Update internal/buildscripts/packaging/installer/install.ps1 Co-authored-by: Paulo Janotti <pjanotti@splunk.com> --------- Co-authored-by: Pablo Collins <pcollins@splunk.com> Co-authored-by: Paulo Janotti <pjanotti@splunk.com>
codeboten
pushed a commit
to open-telemetry/opentelemetry-collector-contrib
that referenced
this issue
Sep 5, 2024
…te_write_queue.num_consumers being a no-op (#34993) **Description:** This PR documents and adds a warning log if remote_write_queue.num_consumers is set in the prometheusremotewriteexporter's config. Current behavior already doesn't use the configuration for anything, more information can be found in open-telemetry/opentelemetry-collector#2949 **Link to tracking Issue:** Related to #34229 (not a fix) Should we skip changelog here? Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
f7o
pushed a commit
to f7o/opentelemetry-collector-contrib
that referenced
this issue
Sep 12, 2024
…te_write_queue.num_consumers being a no-op (open-telemetry#34993) **Description:** This PR documents and adds a warning log if remote_write_queue.num_consumers is set in the prometheusremotewriteexporter's config. Current behavior already doesn't use the configuration for anything, more information can be found in open-telemetry/opentelemetry-collector#2949 **Link to tracking Issue:** Related to open-telemetry#34229 (not a fix) Should we skip changelog here? Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Prometheus remote write (PWR) exporter is supporting queued retry out of the box, but due to the strict requirements of PWR, the queue causes "out of order sample" errors. For PWR to work correctly, all the samples sent from the collector need to be in chronological order for a time series. If sending queue is enabled:
It causes the samples from the same timeseries to be consumed by different queued and causes the ordering issue I experience.
cc @odeke-em
The text was updated successfully, but these errors were encountered: