Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust and rename ThanosSidecarUnhealthy to ThanosSidecarNoConnectionToStartedPrometheus; Remove ThanosSidecarPrometheusDown alert; Remove unused thanos_sidecar_last_heartbeat_success_time_seconds metrics #4508

Merged
merged 4 commits into from
Sep 24, 2021

Commits on Sep 21, 2021

  1. Refactor sidecar alerts

    Prior to this fix, ThanosSidecarUnhealthy would fire even when
    Prometheus is busy with WAL replay. This would trigger a false positive alert.
    
    This PR considers prometheus_tsdb_data_replay_duration_seconds metric from
    Prometheus for ThanosSidecarUnhealthy alert. In order to correlate
    Thanos and Prometheus metrics we need to specify common label(s) which
    can be confiured through thanosPrometheusCommonDimensions jsonnet
    variable.
    
    This PR also removes ThanosSidecarPrometheusDown as it would fire at the same as ThanosSidecarUnhealthy.
    
    Fixes #3915.
    
    Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
    Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
    arajkumar and bwplotka committed Sep 21, 2021
    Configuration menu
    Copy the full SHA
    4ef68ca View commit details
    Browse the repository at this point in the history
  2. Rename ThanosSidecarUnhealthy to ThanosSidecarNoConnectionToStartedPr…

    …ometheus
    
    Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
    arajkumar committed Sep 21, 2021
    Configuration menu
    Copy the full SHA
    ba8ac83 View commit details
    Browse the repository at this point in the history
  3. Simplify ThanosSidecarNoConnectionToStartedPrometheus using thanos_si…

    …decar_prometheus_up
    
    Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
    arajkumar committed Sep 21, 2021
    Configuration menu
    Copy the full SHA
    9eb576c View commit details
    Browse the repository at this point in the history
  4. Remove unused implementation of thanos_sidecar_last_heartbeat_success…

    …_time_seconds metric
    
    Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
    arajkumar committed Sep 21, 2021
    Configuration menu
    Copy the full SHA
    e9ab760 View commit details
    Browse the repository at this point in the history