SpikeInterface Enhancement Proposal: `SortingAnalyzer` object

[samuelgarcia]

Rationale

Since SpikeInterface v0.90, the processing of a recording + a spike sorting output started from the creation of a WaveformExtractor and the subsequent addition of data from the postprocessing and qualitymetrics (QM) modules as WaveformExtension (from v0.93).
While some postprocessing and QM functions need waveforms, this is not the case for all (e.g., computing correlograms, spike amplitude, ISI histograms). In addition, the WaveformExtractor also ended up including a bunch of additional metadata handling for storing and retrieving metadata of the recording and sorting objects, so it doesn’t only handle waveforms.

With this proposal, we therefore suggest to implement a new SpikeSortingResult object which will only combine a recording and sorting object, define sparsity, handle the respective metadata, and implement loading and saving to multiple backends (memory, binary folders, zarr)

Features

• Combines Recording + Sorting
• Caches Sorting and recording info
• Works recordingless
• Supports extensions
• Waveforms is also a ResultExtension living in core
• Other ResultExtension live elsewhere (postprocessing, qualitymetrics, …)
• Ability to compute several extensions one by one ssr.compute(“spike_amplitude”) or spikeinterface.postprocessing.compute_spike_amplitudes(ssr)
• Ability to compute several extensions at once: ssr.compute([“spike_amplitude”, “unit_location”])
• Automate compute logic: registration of an extension automatically generates the function / docstring. etc.
• Native backends: memory + binary (npy) + zarr (TODO)
• Export backends: other backends that are not core-supported
• (run_sorter could output a SpikeSortingResult, to be discussed)

API changes

Old API

recording = read_openephys(...)
sorting = run_sorter(‘tridesclous’, recording, …)
# this create the object and also extractor waveforms
# we=WaveformExtactor
we = extract_waveforms(recording, sorting, ‘/waveorm_folder’)
# then some post processing
amplitudes = compute_spike_amplitude(we)
qm = compute_quality_metrics(we)
unit_locations = compute_unit_locations(we)

plot_waveforms(we)
# this used to be a strange syntax:
plot_quality_metrics(we)

This new API will be

recording = read_openephys(...)
sorting = run_sorter(‘tridesclous’, recording, …)
# this create the object and also extractor waveforms
# ssr=SpikeSortingResult
ssr = create_result(recording, sorting, ‘/result_folder’) # name to be discussed!
# then some post processing
ssr.compute(“spike_amplitude”)
ssr.compute(“quality_metrics”)
ssr.compute(“unit_locations”)
# or call by functions to get some backyard compatibility
# function call return directly the results
amplitudes = compute_spike_amplitude(ssr )
qm = compute_quality_metrics(we)
unit_locations = compute_unit_locations(we)
#or 
ssr.compute([“spike_amplitude”, “quality_metrics”, “unit_locations”])

plot_waveforms(ssr)
# this is now a better syntax
plot_quality_metrics(ssr)

Refactor needed

Internal:

• Quality metrics
• Postprocessing
• Widgets
• Exporters
• Curation

External:

• SI-GUI
• Lussac
• NeuroConv

Transition plan

For a transition phase, we will need to keep a DummyWaveformExtractor that mimics the behavior of the WaveformExtractor but internally use a SpikeSortingReult

• All docs and examples will need to be updated.
• Important: keep a load_sorting_result_from_waveforms() for old results
• New tutorial on SpikeSortingResult

Agenda

December 2023 : first public discussion draft
January 2024 : start dev in a separate branch
February 2024 : last release 0.100.0 with WaveformExtractor
Day after the release : main go in 100_bug_fix branch and dev become main
June 2024 : release 0.101.0 with SpikeSortingResult

Advanced users : we need your feeback!

@magland @colehurwitz @mhhennig @khl02007 @h-mayorquin @zm711 @DradeAW @yger @ferchaure @bendichter @CodyCBakerPhD @JoeZiminski @TomBugnon @grahamfindlay @cwindolf @julienboussard

Let's discuss here and then we will open a project for sub tasks.

[DradeAW]

I love the idea!
I don't really like the name SpikeSortingResult, but as of now I can't think of a better name. I'll try to think of one :)

Maybe the waveforms could be an extension? (like ssr.compute("waveforms / templates", **params)).

Also another subject: Would it be possible to think of a way to handle multiple filterings?
Let me explain: I like to apply a different filter when computing the PCA, or when computing the spikes amplitude, and when I look at the templates I like them unfiltered. The current implementation forces me a create a new WaveformExtractor for every operation, which is a bit annoying.
So my question is: am I the only one that likes to do this? Would it be possible to implement this easily and not make everything a mess?

[samuelgarcia]

Maybe the waveforms could be an extension? (like ssr.compute("waveforms / templates", **params)).

Yes it is the plan.

Also another subject: Would it be possible to think of a way to handle multiple filterings? Let me explain: I like to apply a different filter when computing the PCA, or when computing the spikes amplitude, and when I look at the templates I like them unfiltered. The current implementation forces me a create a new WaveformExtractor for every operation, which is a bit annoying. So my question is: am I the only one that likes to do this? Would it be possible to implement this easily and not make everything a mess?

This is un interresting feature.
A bit challenging.
We could think about having a main recording and also a list of other recording and every extenssion could choose which recording.

This is particularlly relevant to have raw vs processed metrics.

[DradeAW]

I don't really like the name SpikeSortingResult, but as of now I can't think of a better name. I'll try to think of one :)

To expend on that, to me the Sorting class is the spike-sorting result.
The combination of recording and sorting is more like an analysis, but that's not a very good name.

Here are the suggestions from ChatGPT, maybe it can give us some ideas?

1. ElectroSort
2. NeuroRecorder
3. SpikeTraceAnalyzer
4. ElectroPhysSorter
5. DataSpikeIntegrator
6. NeuralWaveSorter
7. VoltageSpikeMapper
8. NeuroPhysioSort
9. SpikeDataFusion
10. ElectroSpikeSynthesizer

This is un interresting feature.
A bit challenging.

I hope you mean "an interesting feature" and not "un-interesting feature" 😅
Yes I know it's challenging and I don't want to complicate the new class too much. I'm emitting the idea because I think it's a cool feature, but I would understand if it's too niche and/or too complicated.

[h-mayorquin]

@DradeAW says:

Sorting class is the spike-sorting result.
The combination of recording and sorting is more like an analysis, but that's not a very good name.

If by Sorting you mean BaseSorting then Yes, +1 on both points.

This seems like a class that can be convenient for analysis and that is lighter (in expectation) than the waveform extractor. I don't think this should or need to be tied to the output of the sorters.

From the point of view of neurconv, I would opose having the proposed class as an output of run_sorter. This would complicate things for us and I think the heuristic of not mixing representations that are good for IO with representations that are good for analysis is a good one.

From the point of view of spikeinterface it would be good for me personally to understand what are the undesirable features of the WaveformExtractor as that seems to be the driving force of this: to have a class that is useful for analysis yet simpler in some sense than the current WaveformExtractor. That will allow me to be helpful in this discussion as I am not a power user of the processing features of the library.

[DradeAW]

From the point of view of spikeinterface it would be good for me personally to understand what are the undesirable features of the WaveformExtractor as that seems to be the driving force of this: to have a class that is useful for analysis yet simpler in some sense than the current WaveformExtractor. That will allow me to be helpful in this discussion as I am not a power user of the processing features of the library.

I believe that the main driving force is that to create a WaveformExtractor object, you need to extract the waveforms (which is a slow process). And in some cases, you need a waveform extractor object, but actually don't need the waveforms.
Yes you can technically do wvf_extractor = si.WaveformExtractor(recording, sorting) and the waveforms aren't extracted, but this is not very pretty, and the name doesn't correspond to the object.

[h-mayorquin]

@DradeAW Thanks.

[samuelgarcia]

@h-mayorquin:
the main motivations:

• mainly: avoid extract waveforms when not necessary
• avoid spaghetti code of WaveformExtactor that handle both : save/load + gather rec/sorting + waveforms handling
• a logical semantic plot_spike_amplitudes(WaveformExtactor) is somehow very strange because you do not
  need waveforms for that. It is hard for a begenner that WaveformExtactor is the first step for post processing.
• make several postprocessing at once
• with zarr backend would be easy to make a high quality and interactive viewer (webbased or qt based) that stream
  on fly everything from this obect without any data duplication.

The last is long term but would be a game changer:

• recording on cloud
• sorting on computing machine
• push SpikeSortingResult on cloud
• visualize directly with a rich app given an url

[grahamfindlay]

As someone who often uses a WaveformExtractor for things that don't touch waveforms, this makes a lot of sense. Thinking of a good name is hard. SortingAnalyzer, SpikeAnalyzer, PostsortingExtractor, PostprocesssingExtractor?

[magland]

This sounds great! Here's my name suggestion

RecordingSorting or SortingRecording

def some_function(X: si.BaseRecordingSorting):
    ...

[khl02007]

• no strong opinion, can see how this might be useful
• why is "work recordingless" a feature to aim for? isn't that what a sorting is? or does it mean a recording is optional (but the metadata about it is still required)?
• suggested name: SortedRecording

[yger]

I think it would be a great feature, indeed. I think I would go for a name such as RecordingSorting, as proposed by @magland . A couple of ideas also:

• most of the time, we don't need waveforms to compute metrics, so +1 to be able to bypass this computation. It would be great, however, if such an object could handle the Templates (added by @h-mayorquin ) (often available at the end of a spike sorting pipelines). Because with the Templates, some metrics like SNR could be also available directly, without the need to extract waveforms.
• Should the RecordingSorting object keep track of the preprocessing chain that has been performed to obtain the sorting? By doing so, when asked to extract waveforms, the object would immedialy know what algorithmic steps should be re-applied to get the waveforms corresponding to the sorting. Then, maybe one option would be to play with such preprocessing pipelines (as requested by @DradeAW ), and be able to extract from preprocessed, from raw, or from custom chains (and thus exposing a dict of metrics, depending on the preprocessing steps?)

[DradeAW]

I like the names RecordingSorting and SortedRecording (I slightly prefer the latter).

[yger]

Yes, SortedRecording works also for me, I think this is even better

[JoeZiminski]

This looks really nice, I like SortedRecording also