[ENH] Implement range-based precision and recall within aeon (replacing prts package)

[SebastianSchmidl]

Describe the feature or idea you want to propose

The package prts is used in the performance evaluation module for anomaly detection as a soft-dependency. However, there have not been any changes to it for 3 years, the maintainers have not reacted to my PR, and the package was not implemented by the original authors of the method/paper.

Describe your proposed solution

We should implement the metrics within aeon directly:

• implement ts_fscore, ts_precision, ts_recall in aeon/benchmarking/metrics/anomaly_detection/range_metrics
  ➡ addressed in PR [ENH] Replace prts metrics #2400
• Update tests for range metrics: The tests should use fixtures instead of the prts implementations.
  ➡ [ENH] Remove prts package and related functions  #2576
  ➡ addressed in [ENH] Hard-Coded Tests for test_metrics.py #2672
• Refactorings, e.g. fix method signatures to ensure compatibility with our other metrics and move tests from tests/test_metrics.py to tests/test_range_metrics.py ...
  ➡ [ENH] Refactor range-based metric API for consistency with other AD metrics #2780
  ➡ addressed in [ENH] Refactor range-based metrics to restore original behavior #2781
• improve Python docs and replace old metric documentation with the new metrics' docs
  ➡ [DOC] Add API documentation for range_metrics module and its functions (ts_recall, ts_precision, ts_fscore) #2767 (obsolete because the old metric API is deprecated in [ENH] Refactor range-based metric API for consistency with other AD metrics #2780)
• The functions in _continuous.py need to be updated to use the new implementations
  ➡ [MNT] Range-based AD metrics: The functions in _continuous.py need to be updated to use the new implementations #2785
• Remove prts dependency and old metric implementations. Requires 1.2.0 release!
  ➡ [ENH] Remove prts package and related functions  #2576

Target module: aeon.benchmarking.metrics.anomaly_detection
Original publication:

Tatbul, Nesime, Tae Jun Lee, Stan Zdonik, Mejbah Alam, and Justin Gottschlich. "Precision and Recall for Time Series." In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), 1920–30. 2018.
http://papers.nips.cc/paper/7462-precision-and-recall-for-time-series.pdf.

Describe alternatives you've considered, if relevant

not relevant

Additional context

No response

[aryanpola]

@aeon-actions-bot assign @aryanpola

[aryanpola]

@CodeLionX So, the true anomaly ranges would use the timepoints to create a range to compare with the algorithm-detected anomaly range. Maybe later find an intersection between them.

[SebastianSchmidl]

Hi @aryanpola,
yes and no. Please read the paper carefully, it is quite detailed in its descriptions on how to compute ts_precision and ts_recall respectively (math formulas and pseudocode).

In their formulation, they convert both the ground truth (point) values and the (point) predictions into ranges, and then iterate over the real/true anomaly ranges for recall, but over the predicted anomaly ranges for precision.

You can use the existing implementation in prts to generate test cases to validate that your implementation is doing the same.

[SebastianSchmidl]

Updated issue description with remaining to-dos.

[aryanpola]

Hi @SebastianSchmidl,
Just need a small clarification on point 3 of the issue. Pytest fixtures are generally useful when we use a singular dataset in multiple tests, but in the current case, the test cases utilize different datasets as input with some initialized parameter values. Would it be a good idea to move further with the suggested implementation?

The code block contains initial values and format if the implementation is applied.

# inital values
def _mk_case(y_pred, y_true, *, gamma="one", bias="flat", beta=1, alpha=0.0,
             expected_p=0.0, expected_r=0.0, expected_f=0.0):
    return {
        "y_pred": np.asarray(y_pred),
        "y_true": np.asarray(y_true),
        "params": {
            "gamma": gamma,
            "bias_type": bias,
            "beta": beta,
            "p_bias": bias,
            "r_bias": bias,
            "p_alpha": alpha,
            "r_alpha": alpha,
        },
        "expected": {"p": expected_p, "r": expected_r, "f": expected_f},
    }