Refactor metrics into its own module

[alejoe91]

This PR includes a major refactor of the metrics concept.

It defines a BaseMetric, with core metadata of individual metrics including dtypes, column names, extension dependance, and a compute function.
Another BaseMetricExtension contains a collection of BaseMetrics and deals with most of the machinery, including:

• setting params
• checking dependencies and removing metrics based on it
• computing metrics
• deleting, merging, splitting metrics
• preparing data that can be shared across metrics (e.g., pca for pca metrics, peaks info and templates for template metrics)

The template_metrics, quality_metrics, and a new spiketrain_metrics extensions are now in the metrics module. The latter only includes num_spikes and firing_rate, which are also imported as quality metrics.

Still finalizing tests, but this should be 90% done

[chrishalcrow]

Hello, this is a new thing. It'd be great to check if we can compute this before we try to do it, for the following situation:

Suppose you originally compute a metric using spikeinterface version 103 (or some fork that you've made yourself... ahem).
Then you open your analyzer in si-gui using version 102. There was a new metric introduced in 103, which 102 doesn't know about. When you try to merge, it errors because it can't compute the new metric. So you do any merging at all due to the inability to merge one metric.
Or you no longer have the recording when you open, so you can't compute sd_ratio or something....

Instead, I'd like to warn if we can't compute and stick in anan. We could could do that here by checking that metric_names are in self.metric_list.

[chrishalcrow]

Oh, I think I meant to write this at line 1207 about the merging step, but also applies to splits!

[chrishalcrow]

Nag nag nag

[chrishalcrow]

These checks were done in case people use e.g. compute_snrs directly, and it still gives them a good error message.

[chrishalcrow]

We used to import all the compute functions, like compute_snrs.

[alejoe91]

I think that users can still access the individual functions with from spikeinterface.metrics.quality.misc_metrics import ...

We can make sure we highlight this in the release notes, but I would avoid clogging the import with all the low-level functions

[chrishalcrow]

I'd be more careful, because our current docs (e.g. https://spikeinterface.readthedocs.io/en/stable/modules/qualitymetrics/firing_range.html) has lots of code which imports the functions directly from spikeinterface.quality_metrics. So I reckon this could break a lot of code.

[chrishalcrow]

This looks great - love the prepare_data idea! And this refactor will make it less awkward to develop new metrics. And the pca file is much neater now - nice.

I think this is a good chance to remove compute_{metric_name} stuff from the docs (modules/qualitymetrics) and replace with analyzer.compute("quality_metrics", metric_names={metric_name}) as our recommended method. More awkward, but much better for provenance etc.

I'd vote to take the chance to make multi channel template metrics included by default: they're very helpful.

[alejoe91]

I'd vote to take the chance to make multi channel template metrics included by default: they're very helpful.

I agree! Maybe we can make it default for number of channel > 64?

[chrishalcrow]

Was having a play - this PR makes it very easy to write your own metrics outside of spikeinterface, and stuff like merging works fine. Not portable, but useful for me!

import spikeinterface.full as si
from spikeinterface.core.analyzer_extension_core import BaseMetric
from spikeinterface.metrics import ComputeQualityMetrics


def compute_something(
    sorting_analyzer,
    unit_ids=None,
):
    if unit_ids is None:
        unit_ids = sorting_analyzer.unit_ids

    result = {unit_id: str(unit_id) + "_hi!!!!" for unit_id in unit_ids}
    return result

class MyCustomMetric(BaseMetric):
    metric_name = "chris_metric"
    metric_function = compute_something
    metric_params = {}
    metric_columns = {"the_unit_id": str}

ComputeQualityMetrics.metric_list.append(MyCustomMetric)

rec, sort = si.generate_ground_truth_recording()
sa = si.create_sorting_analyzer(sort, rec)
sa.compute('quality_metrics')

merged_sa = sa.merge_units(merge_unit_groups = [['1', '2']], new_unit_ids=['12'])
merged_sa.get_extension("quality_metrics").get_data().loc['12']

>>>
num_spikes                    272
firing_rate                  27.2
presence_ratio                NaN
isi_violations_ratio     0.270329
isi_violations_count            6
rp_contamination         0.145792
rp_violations                   2
sliding_rp_violation        0.265
sync_spike_2             0.011029
sync_spike_4                  0.0
sync_spike_8                  0.0
firing_range                 1.44
the_unit_id             12_hi!!!!
Name: 12, dtype: object

[chrishalcrow]

Suggested change

	Metrics
	Metrics module
	--------------

[chrishalcrow]

I don't get the deprecation warning if I do e.g.

from spikeinterface.quality_metrics import compute_quality_metrics

Because of some import * magic. This should fix it in almost all cases:

if __name__ not in ('__main__', 'builtins'):
    warnings.warn(
        "The module 'spikeinterface.qualitymetrics' is deprecated and will be removed in 0.105.0."
        "Please use 'spikeinterface.metrics.quality' instead.",
        DeprecationWarning,
        stacklevel=2,
    )

[chrishalcrow]

Suggested change

	Job keyword arguments to control paralleization
	Job keyword arguments to control parallelization

[chrishalcrow]

Just noting that we've got pandas in core here. You can't load a sorting analyzer with any metrics extensions without pandas. Time to add it to core deps?

[chrishalcrow]

love love love

[chrishalcrow]

We didn't used to save "include_multi_channel_metrics" so self.params["include_multi_channel_metrics"] errors for old analyzers. Replace with get and it's fine

Suggested change

	include_multi_channel_metrics = self.params["include_multi_channel_metrics"] or any(
	include_multi_channel_metrics = self.params.get("include_multi_channel_metrics") or any(

[chrishalcrow]

Doesn't work for recordingless analyzers. Do we store the channel_locations anywhere else?

[chrishalcrow]

Oh yeah, just do sorting_analyzer.get_channel_locations()

[chrishalcrow]

This will error if we don't know how to compute the metrics in metric_names. So if a metric changes name between version, we get an error and can't merge/split. I think we should only give _compute_metrics the intersection of metric_names and self.metric_list?