-
Notifications
You must be signed in to change notification settings - Fork 49
Overhaul stats: Refactor metrics, use label metrics to calculate global metrics #1577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Overhaul stats: Refactor metrics, use label metrics to calculate global metrics #1577
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #1577 +/- ##
===========================================
+ Coverage 84.15% 84.79% +0.64%
===========================================
Files 286 289 +3
Lines 20789 21712 +923
Branches 20789 21712 +923
===========================================
+ Hits 17495 18411 +916
- Misses 2974 2984 +10
+ Partials 320 317 -3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
It allows sum metric samples matching a given criteria. The criteria is
a label set. Sample values are added if they contain all the label
name/value pairs specified in the criteria.
For example, given these metric's samples in Prometheus export text format:
```
udp_tracker_server_requests_accepted_total{request_kind="scrape",server_binding_address_type="plain",server_binding_ip="0.0.0.0",server_binding_port="6969",server_binding_protocol="udp"} 213118
udp_tracker_server_requests_accepted_total{request_kind="announce",server_binding_address_type="plain",server_binding_ip="0.0.0.0",server_binding_port="6969",server_binding_protocol="udp"} 16460553
udp_tracker_server_requests_accepted_total{request_kind="connect",server_binding_address_type="plain",server_binding_ip="0.0.0.0",server_binding_port="6868",server_binding_protocol="udp"} 617
udp_tracker_server_requests_accepted_total{request_kind="connect",server_binding_address_type="plain",server_binding_ip="0.0.0.0",server_binding_port="6969",server_binding_protocol="udp"} 17148137
```
And the criteria: it should contain the label `request_kind` with the value `connect`.
It should return: 617 + 17148137 = 17148754
bd10f58 to
64be847
Compare
|
I've been considering removing the traits:
pub trait Sum {
fn sum(&self, metric_name: &MetricName, label_set_criteria: &LabelSet) -> Option<AggregateValue>;
}
pub trait Sum {
fn sum(&self, label_set_criteria: &LabelSet) -> AggregateValue;
}I wanted to keep MetricCollection and Metric types unaware (decouple) from the aggregate functions. So:
However, I tried to do the refactor, and I ended up with something like: /// Calculate the sum of metric samples for a given metric name and label criteria.
///
/// This function follows the Open/Closed principle by allowing new aggregate functions
/// to be added without modifying the MetricCollection type.
pub fn sum(collection: &MetricCollection, metric_name: &MetricName, label_set_criteria: &LabelSet) -> Option<AggregateValue> {
// Direct implementation without trait dependency
if let Some(counter_collection) = collection.counters.metrics.get(metric_name) {
return Some(counter_collection.sum(label_set_criteria));
}
if let Some(gauge_collection) = collection.gauges.metrics.get(metric_name) {
return Some(gauge_collection.sum(label_set_criteria));
}
None
}The problem with this approach is that the aggregate function will be broken as soon as we add a new type of metric to the metric collection. It doesn't work for the metric collection, but it could work for the metric. Anyway, since we only have one aggregate function, I will keep it as it is for now. We can reconsider other alternatives if we plan to add more aggregate functions in the future. cc @da2ce7 |
…etrics We need to add a new label to make it easier to fileter by the server IP family: IPV4 or IPv6.
Address might be a socket address.
Example:
```
udp_tracker_core_requests_received_total{request_kind="connect",server_binding_address_ip_family="inet",server_binding_address_ip_type="plain",server_binding_ip="0.0.0.0",server_binding_port="6969",server_binding_protocol="udp"} 1
```
It's needed to easily filter metric samples to calculate aggregate
values for a given IP family (IPv4 or IPv6).
08c0c38 to
dcfb5d5
Compare
|
ACK dcfb5d5 |

The idea is to continue exposing the
statsendpoint but removing the internal code to calculate them. Instead, we calculate the metrics from the labeled metrics.That will simplify the event listeners and avoid duplicate code to calculate the same data.
Endpoints for statistics
stats(simple predefined global metrics)metrics(extendable-labelled metrics)Subtasks
MetricCollectionto sum the metric's samples.statsendpoint.After merging this PR
statsendpoint directly in the event listeners.NOTICE: We have two Grafana dashboards, one collecting data from the
statsendpoint and another collecting data frommetrics, and they look identical now. If there is a bug in the way we calculate aggregate values in theMetricCollection, we should see different graphs in the dashboards.Both dashboards (built from stats on the left, built from metrics on the right):
NOTICE: Also, the new methods will be helpful for external users once we publish the metrics crate. Consumers of the Torrust Tracker API (
metricsendpoint) will be able to deserialize and calculate aggregate values from the response.