feat(metrics): add MetricResult ABC and aggregate comparison functions - #2498
Open
shaoming11 wants to merge 1 commit into
Open
feat(metrics): add MetricResult ABC and aggregate comparison functions#2498shaoming11 wants to merge 1 commit into
shaoming11 wants to merge 1 commit into
Conversation
roboflow#1707) - Add MetricResult abstract base class with to_pandas(), plot(), and _get_plot_details() abstract methods - Add PlotDetails dataclass for structured bar-chart data - Make all 5 result dataclasses inherit from MetricResult - Add _get_plot_details() to each result class; refactor plot() to use it - Add aggregate_metric_results() to combine results into a DataFrame - Add plot_aggregate_metric_results() for grouped bar chart comparison - Export new symbols from metrics __init__.py - Add 16 tests covering aggregation, plotting, and validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds shared metric-result abstractions and utilities for comparing model metrics.
Changes:
- Introduces
MetricResultandPlotDetails. - Adds tabular aggregation and grouped plotting.
- Refactors five result classes and adds tests and changelog entries.
Assessment: Code quality 3/5 · Testing 3/5 · Documentation 2/5
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
src/supervision/metrics/core.py |
Adds result and plotting abstractions. |
src/supervision/metrics/utils/aggregate.py |
Implements aggregation and comparison plotting. |
src/supervision/metrics/f1_score.py |
Adopts the shared result interface. |
src/supervision/metrics/precision.py |
Adopts the shared result interface. |
src/supervision/metrics/recall.py |
Adopts the shared result interface. |
src/supervision/metrics/mean_average_precision.py |
Adopts the shared result interface. |
src/supervision/metrics/mean_average_recall.py |
Adopts the shared result interface. |
src/supervision/metrics/__init__.py |
Exports the new public APIs. |
tests/metrics/utils/test_aggregate.py |
Tests aggregation, plotting, and abstractions. |
docs/changelog.md |
Records the new functionality. |
Suppressed comments (1)
src/supervision/metrics/utils/aggregate.py:98
- The documented exception type is incorrect: mixed concrete result types raise
TypeErroron line 108, notValueError. Document the two exception types separately.
Raises:
ValueError: If the list is empty, contains mixed result types, or
*model_names* length does not match *metric_results*.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+38
to
+40
| Raises: | ||
| ValueError: If the list is empty, contains mixed result types, or | ||
| *model_names* length does not match *metric_results*. |
| """Plotting runs without error and calls plt.show().""" | ||
| r1 = _make_f1_result(f1_50=0.8, f1_75=0.6) | ||
| r2 = _make_f1_result(f1_50=0.9, f1_75=0.7) | ||
| plot_aggregate_metric_results([r1, r2], model_names=["YOLO", "DETR"]) |
Comment on lines
+792
to
+796
| title = ( | ||
| f"F1 Score, by Object Size" | ||
| f"\n(target: {self.metric_target.value}," | ||
| f" averaging: {self.averaging_method.value})" | ||
| ) |
Comment on lines
+792
to
+796
| title = ( | ||
| f"Precision, by Object Size" | ||
| f"\n(target: {self.metric_target.value}," | ||
| f" averaging: {self.averaging_method.value})" | ||
| ) |
Comment on lines
+752
to
+756
| title = ( | ||
| f"Recall, by Object Size" | ||
| f"\n(target: {self.metric_target.value}," | ||
| f" averaging: {self.averaging_method.value})" | ||
| ) |
Comment on lines
+247
to
+249
| title = ( | ||
| f"Mean Average Recall, by Object Size\n(target: {self.metric_target.value})" | ||
| ) |
Comment on lines
+18
to
+20
| from supervision.metrics.utils.aggregate import ( | ||
| aggregate_metric_results, | ||
| plot_aggregate_metric_results, |
Comment on lines
+128
to
+129
| labels = all_details[0].labels | ||
| title = all_details[0].title |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #1707 — metrics comparison table and plotting.
MetricResultabstract base class withto_pandas(),plot(), and_get_plot_details()abstract methods, plus aPlotDetailsdataclass for structured bar-chart dataF1ScoreResult,PrecisionResult,RecallResult,MeanAveragePrecisionResult,MeanAverageRecallResult) inherit fromMetricResult_get_plot_details(include_object_sizes)to each result class; refactoredplot()to use itaggregate_metric_results()— combines multipleMetricResultobjects into a singlepd.DataFramewith optionalmodel_namesindex andinclude_object_sizescontrolplot_aggregate_metric_results()— creates a grouped bar chart comparing multiple models side-by-sidesupervision.metricsTest plan
tests/metrics/utils/test_aggregate.pycovering validation, aggregation, plotting, and_get_plot_details()🤖 Generated with Claude Code