Skip to content

Commit

Permalink
chore(storage/remote): collect maxTimestamp when value is 0 as well.
Browse files Browse the repository at this point in the history
This change enables the PrometheusRemoteWriteBehind alert’s expression to be evaluated
even when the remote endpoint has never been reached. As a result, PrometheusRemoteWriteBehind
will fire to easily detect configuration mistakes (such as incorrect endpoint URLs) or
unrecoverable connectivity issues.

See prometheus#14350 for details.

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
  • Loading branch information
machine424 committed Jul 11, 2024
1 parent 58963f7 commit caa71fb
Show file tree
Hide file tree
Showing 4 changed files with 3 additions and 9 deletions.
6 changes: 0 additions & 6 deletions storage/remote/max_timestamp.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,3 @@ func (m *maxTimestamp) Get() float64 {
defer m.mtx.Unlock()
return m.value
}

func (m *maxTimestamp) Collect(c chan<- prometheus.Metric) {
if m.Get() > 0 {
m.Gauge.Collect(c)
}
}
2 changes: 1 addition & 1 deletion storage/remote/queue_manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ func newQueueManagerMetrics(r prometheus.Registerer, rn, e string) *queueManager
Namespace: namespace,
Subsystem: subsystem,
Name: "queue_highest_sent_timestamp_seconds",
Help: "Timestamp from a WAL sample, the highest timestamp successfully sent by this queue, in seconds since epoch.",
Help: "Timestamp from a WAL sample, the highest timestamp successfully sent by this queue, in seconds since epoch. Initialized to 0 when no data has been sent yet.",
ConstLabels: constLabels,
}),
}
Expand Down
2 changes: 1 addition & 1 deletion storage/remote/queue_manager_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ func newHighestTimestampMetric() *maxTimestamp {
Namespace: namespace,
Subsystem: subsystem,
Name: "highest_timestamp_in_seconds",
Help: "Highest timestamp that has come into the remote storage via the Appender interface, in seconds since epoch.",
Help: "Highest timestamp that has come into the remote storage via the Appender interface, in seconds since epoch. Initialized to 0 when no data has been received yet",
}),
}
}
Expand Down
2 changes: 1 addition & 1 deletion storage/remote/write.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ func NewWriteStorage(logger log.Logger, reg prometheus.Registerer, dir string, f
Namespace: namespace,
Subsystem: subsystem,
Name: "highest_timestamp_in_seconds",
Help: "Highest timestamp that has come into the remote storage via the Appender interface, in seconds since epoch.",
Help: "Highest timestamp that has come into the remote storage via the Appender interface, in seconds since epoch. Initialized to 0 when no data has been received yet.",
}),
},
}
Expand Down

0 comments on commit caa71fb

Please sign in to comment.