Error using si.run_sorter [Mountainsort5] with TDT data on Windows 10 #2489

jesusdpa1 · 2024-02-14T22:53:38Z

Hi, I am encountering an error when running mountainsort5. I follow the instructions as describe in #2471

n_jobs = 1
create a tmp dir in a not nested folder

OS: Windows 10
Ram: 128
SSD available: 1.2Tb
Data Size after filter and zarr compress: 340mb
Si version: '0.100.0'
MS5 version: '0.5.3'

the code that I am using is:

# imports 
import numcodecs
from pathlib import Path
import spikeinterface.full as si
import spikeinterface.extractors as se
import probeinterface as pi
from probeinterface import read_probeinterface
from probeinterface.plotting import plot_probe_group
print(f"SpikeInterface version: {si.__version__}") # 0.100.0

#%%
import matplotlib.pyplot as plt
import numpy as np
from pathlib import Path
import os

import warnings
warnings.simplefilter("ignore")

# %matplotlib widget
%matplotlib widget

save_folder = drv_dir.joinpath("preprocessed_compressed.zarr")
if (save_folder).is_dir():
    recording_saved = si.read_zarr(save_folder)
else:
    import numcodecs
    compressor = numcodecs.Blosc(cname="zstd", clevel=9, shuffle=numcodecs.Blosc.BITSHUFFLE)
    recording_saved = recording_sub.save(format="zarr", folder=save_folder,
                                         compressor=compressor,
                                         **job_kwargs)
#%%
si.get_default_sorter_params('mountainsort5')

#%%
tmp_folder = Path.home().joinpath('.tmp')
#%%
sorter_params = {'filter': False,
                 'temporary_base_dir': tmp_folder.__str__(),
                 'n_jobs_for_preprocessing': 1}
#%%
sorting_ = si.run_sorter(sorter_name='mountainsort5', recording=recording_saved, remove_existing_folder=True,
                         output_folder=tmp_folder.joinpath('results_').__str__(), 
                         **sorter_params,
                         verbose=True)

# params

{
    "sorter_name": "mountainsort5",
    "sorter_params": {
        "scheme": "2",
        "detect_threshold": 5.5,
        "detect_sign": -1,
        "detect_time_radius_msec": 0.5,
        "snippet_T1": 20,
        "snippet_T2": 20,
        "npca_per_channel": 3,
        "npca_per_subdivision": 10,
        "snippet_mask_radius": 250,
        "scheme1_detect_channel_radius": 150,
        "scheme2_phase1_detect_channel_radius": 200,
        "scheme2_detect_channel_radius": 50,
        "scheme2_max_num_snippets_per_training_batch": 200,
        "scheme2_training_duration_sec": 300,
        "scheme2_training_recording_sampling_mode": "uniform",
        "scheme3_block_duration_sec": 1800,
        "freq_min": 300,
        "freq_max": 6000,
        "filter": false,
        "whiten": true,
        "temporary_base_dir": "C:\\Users\\jesus.penalozaa\\.tmp",
        "n_jobs_for_preprocessing": 1
    }
}

# Error

{
    "sorter_name": "mountainsort5",
    "sorter_version": "0.5.3",
    "datetime": "2024-02-14T17:37:37.833633",
    "runtime_trace": [],
    "error": true,
    "error_trace": "Traceback (most recent call last):\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\shutil.py\", line 634, in _rmtree_unsafe\n    os.unlink(fullname)\nPermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\jesus.penalozaa\\\\.tmp\\\\tmp5b8tjy78\\\\recording.dat'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\site-packages\\spikeinterface\\sorters\\basesorter.py\", line 258, in run_from_folder\n    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\site-packages\\spikeinterface\\sorters\\external\\mountainsort5.py\", line 189, in _run_from_folder\n    with TemporaryDirectory(dir=p[\"temporary_base_dir\"]) as tmpdir:\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\tempfile.py\", line 946, in __exit__\n    self.cleanup()\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\tempfile.py\", line 950, in cleanup\n    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\tempfile.py\", line 930, in _rmtree\n    _shutil.rmtree(name, onexc=onexc)\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\shutil.py\", line 808, in rmtree\n    return _rmtree_unsafe(path, onexc)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\shutil.py\", line 636, in _rmtree_unsafe\n    onexc(os.unlink, fullname, err)\n  File \"c:\\Users\\jesus.penalozaa\\AppData\\Local\\miniconda3\\envs\\roboz\\Lib\\tempfile.py\", line 905, in onexc\n    _os.unlink(path)\nPermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\jesus.penalozaa\\\\.tmp\\\\tmp5b8tjy78\\\\recording.dat'\n",
    "run_time": null
}

zm711 · 2024-02-15T13:46:14Z

@jesusdpa1. Could you do us one other favor and post the stack trace. We have the traceback but it is super hard to read going from left to right. If you have your terminal open and copy those lines we can try to read everything that is going on. Since Mountainsort5 provides updates as it is working it would also help us to see how far it gets while running Mountainsort5 and that information is not provided in the error_trace.

cc @magland

jesusdpa1 · 2024-02-15T14:58:08Z

Of course 👍

Using training recording of duration 300 sec with the sampling mode uniform
*** MS5 Elapsed time for SCHEME2 get_sampled_recording_for_training: 0.000 seconds ***
Running phase 1 sorting
Number of channels: 14
Number of timepoints: 7324218
Sampling frequency: 24414.0625 Hz
Channel 0: [150.   0.]
Channel 1: [150. 500.]
Channel 2: [150. 750.]
Channel 3: [ 150. 1000.]
Channel 4: [ 150. 1250.]
Channel 5: [ 150. 1500.]
Channel 6: [ 150. 1750.]
Channel 7: [550.   0.]
Channel 8: [550. 500.]
Channel 9: [550. 750.]
Channel 10: [ 550. 1000.]
Channel 11: [ 550. 1250.]
Channel 12: [ 550. 1500.]
Channel 13: [ 550. 1750.]
Loading traces
*** MS5 Elapsed time for load_traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 200
[[0], [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13]]

m = 0 (nbhd size: 1)
m = 1 (nbhd size: 1)
m = 2 (nbhd size: 1)
m = 3 (nbhd size: 1)
m = 4 (nbhd size: 1)
m = 5 (nbhd size: 1)
m = 6 (nbhd size: 1)
m = 7 (nbhd size: 1)
m = 8 (nbhd size: 1)
m = 9 (nbhd size: 1)
m = 10 (nbhd size: 1)
m = 11 (nbhd size: 1)
m = 12 (nbhd size: 1)
m = 13 (nbhd size: 1)
Detected 34149 spikes
*** MS5 Elapsed time for detect_spikes: 0.751 seconds ***
Removing duplicate times
*** MS5 Elapsed time for remove_duplicate_times: 0.000 seconds ***
Extracting 33728 snippets
*** MS5 Elapsed time for extract_snippets: 0.224 seconds ***
Computing PCA features with npca=42
*** MS5 Elapsed time for compute_pca_features: 0.522 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
Found 23 clusters
*** MS5 Elapsed time for isosplit6_subdivision_method: 3.194 seconds ***
Computing templates
*** MS5 Elapsed time for compute_templates: 0.153 seconds ***
Determining optimal alignment of templates
Template alignment converged.
Align templates offsets:  [ 0 -2  0  1  1  6  0  0  0 16  0  0  0  1  4  4  0  0  1  0 14 16  0]
*** MS5 Elapsed time for align_templates: 0.342 seconds ***
Aligning snippets
*** MS5 Elapsed time for align_snippets: 0.091 seconds ***
Clustering aligned snippets
Computing PCA features with npca=42
*** MS5 Elapsed time for compute_pca_features: 0.480 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
*** MS5 Elapsed time for isosplit6_subdivision_method: 2.954 seconds ***
Found 23 clusters after alignment
Computing templates
*** MS5 Elapsed time for compute_templates: 0.155 seconds ***
Offsetting times to peak
Offsets to peak: [ 0 -2  4  4  1  6  0  0 16  0  0  0  1  1  0  0 -7  0  0 14 16 14  0]
*** MS5 Elapsed time for determine_offsets_to_peak: 0.001 seconds ***
Sorting times
*** MS5 Elapsed time for sorting times: 0.000 seconds ***
Removing out of bounds times
*** MS5 Elapsed time for removing out of bounds times: 0.000 seconds ***
Reordering units
*** MS5 Elapsed time for reordering units: 0.001 seconds ***
Creating sorting object
*** MS5 Elapsed time for creating sorting object: 0.003 seconds ***
*** MS5 Elapsed time for SCHEME2 sorting_scheme1: 8.908 seconds ***
*** MS5 Elapsed time for SCHEME2 get_times_labels_from_sorting: 0.003 seconds ***
Loading training traces
*** MS5 Elapsed time for SCHEME2 training_recording.get_traces: 0.088 seconds ***
Training classifier
*** MS5 Elapsed time for SCHEME2 training classifier step 1: 0.012 seconds ***
Adding snippets from phase 1 sorting
Fitting models
*** MS5 Elapsed time for SCHEME2 fitting models: 0.110 seconds ***
Chunk size: 292.57146368 sec
Time chunk 1 of 1
Loading traces
*** MS5 Elapsed time for SCHEME2 loading traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 50
[[0], [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13]]

Scheme 2 detected 34149 spikes in chunk 1 of 1
*** MS5 Elapsed time for SCHEME2 detecting spikes: 0.727 seconds ***
Extracting and classifying snippets
*** MS5 Elapsed time for SCHEME2 extracting and classifying snippets: 0.520 seconds ***
Updating events
Removing duplicates
*** MS5 Elapsed time for SCHEME2 updating events: 0.014 seconds ***
Concatenating results
*** MS5 Elapsed time for SCHEME2 concatenating results: 0.000 seconds ***
Perorming label mapping
*** MS5 Elapsed time for SCHEME2 label mapping: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for SCHEME2 creating sorting object: 0.002 seconds ***
Error running mountainsort5

SpikeSortingError                         Traceback (most recent call last)
File [c:\Users\jesus.penalozaa\Documents\GitHub\ephys-robot\tests\spike_test.py:2](~Documents/GitHub/ephys-robot/tests/spike_test.py:2)
      [1](~Documents/GitHub/ephys-robot/tests/spike_test.py:1) #%%
----> [2](~Documents/GitHub/ephys-robot/tests/spike_test.py:2) sorting_ = si.run_sorter(sorter_name='mountainsort5', recording=recording_saved, remove_existing_folder=True,
      [3](~Documents/GitHub/ephys-robot/tests/spike_test.py:3)                          output_folder=tmp_folder.joinpath('results_').__str__(), 
      [4](~Documents/GitHub/ephys-robot/tests/spike_test.py:4)                          **sorter_params,
      [5](~Documents/GitHub/ephys-robot/tests/spike_test.py:5)                          verbose=True)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\runsorter.py:174](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:174), in run_sorter(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, docker_image, singularity_image, delete_container_files, with_output, **sorter_params)
    [167](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:167)             container_image = singularity_image
    [168](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:168)     return run_sorter_container(
    [169](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:169)         container_image=container_image,
    [170](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:170)         mode=mode,
    [171](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:171)         **common_kwargs,
    [172](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:172)     )
--> [174](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:174) return run_sorter_local(**common_kwargs)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\runsorter.py:224](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:224), in run_sorter_local(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, **sorter_params)
    [222](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:222) SorterClass.set_params_to_folder(recording, output_folder, sorter_params, verbose)
    [223](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:223) SorterClass.setup_recording(recording, output_folder, verbose=verbose)
--> [224](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:224) SorterClass.run_from_folder(output_folder, raise_error, verbose)
    [225](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:225) if with_output:
    [226](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:226)     sorting = SorterClass.get_result_from_folder(output_folder, register_recording=True, sorting_info=True)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py:293](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:293), in BaseSorter.run_from_folder(cls, output_folder, raise_error, verbose)
    [290](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:290)         print(f"{sorter_name} run time {run_time:0.2f}s")
    [292](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:292) if has_error and raise_error:
--> [293](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:293)     raise SpikeSortingError(
    [294](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:294)         f"Spike sorting error trace:\n{log['error_trace']}\n"
    [295](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:295)         f"Spike sorting failed. You can inspect the runtime trace in {output_folder}/spikeinterface_log.json."
    [296](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:296)     )
    [298](file:~miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:298) return run_time

SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 634, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmpyshsfy45\\recording.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\external\mountainsort5.py", line 189, in _run_from_folder
    with TemporaryDirectory(dir=p["temporary_base_dir"]) as tmpdir:
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 946, in __exit__
    self.cleanup()
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 950, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 808, in rmtree
    return _rmtree_unsafe(path, onexc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 636, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmpyshsfy45\\recording.dat'

Spike sorting failed. You can inspect the runtime trace in [C:\Users\jesus.penalozaa\.tmp\results_/spikeinterface_log.json.](~.tmp/results_/spikeinterface_log.json.)

zm711 · 2024-02-16T12:43:25Z

@jesusdpa1 could you try just changing the temporary_base_dir? I want to see if not filtering could be causing this?

zm711 · 2024-02-16T14:08:30Z

Also I posted onto the main repo for Mountainsort5. I'll actually be sorting some data later on Windows, so if it triggers on my own data I'll let you know.

jesusdpa1 · 2024-02-16T15:10:03Z

Yes! I also realized by testing spykingcircus2 that my electrode design required a different function to be called. Spikingcircus2 is extracting the data now.

I did the following tests

Changing the function

# Changing the function 

# %%
si.get_default_sorter_params("mountainsort5")

# %%
tmp_folder = Path.home().joinpath(".tmp")
# %%
sorter_params = {
    "filter": False,
    "temporary_base_dir": tmp_folder.__str__(),
    "n_jobs_for_preprocessing": 1,
}
# %%

aggregate_sorting = si.run_sorter_by_property(
    sorter_name="mountainsort5",
    recording=recording_saved,
    grouping_property="group",
    working_folder=tmp_folder.__str__(),
    remove_existing_folder=True,
    **sorter_params,
)

Verbose

# output
Using training recording of duration 300 sec with the sampling mode uniform
*** MS5 Elapsed time for SCHEME2 get_sampled_recording_for_training: 0.000 seconds ***
Running phase 1 sorting
Number of channels: 7
Number of timepoints: 7324218
Sampling frequency: 24414.0625 Hz
Channel 0: [150.   0.]
Channel 1: [150. 500.]
Channel 2: [150. 750.]
Channel 3: [ 150. 1000.]
Channel 4: [ 150. 1250.]
Channel 5: [ 150. 1500.]
Channel 6: [ 150. 1750.]
Loading traces
*** MS5 Elapsed time for load_traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 200
[[0], [1], [2], [3], [4], [5], [6]]

m = 0 (nbhd size: 1)
m = 1 (nbhd size: 1)
m = 2 (nbhd size: 1)
m = 3 (nbhd size: 1)
m = 4 (nbhd size: 1)
m = 5 (nbhd size: 1)
m = 6 (nbhd size: 1)
Detected 12504 spikes
*** MS5 Elapsed time for detect_spikes: 0.301 seconds ***
Removing duplicate times
*** MS5 Elapsed time for remove_duplicate_times: 0.000 seconds ***
Extracting 12258 snippets
*** MS5 Elapsed time for extract_snippets: 0.073 seconds ***
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.073 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
Found 9 clusters
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.651 seconds ***
Computing templates
*** MS5 Elapsed time for compute_templates: 0.030 seconds ***
Determining optimal alignment of templates
Template alignment converged.
Align templates offsets:  [ 1  0 11 18  0  0  0  1  0]
*** MS5 Elapsed time for align_templates: 0.045 seconds ***
Aligning snippets
*** MS5 Elapsed time for align_snippets: 0.013 seconds ***
Clustering aligned snippets
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.075 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.639 seconds ***
Found 10 clusters after alignment
Computing templates
*** MS5 Elapsed time for compute_templates: 0.029 seconds ***
Offsetting times to peak
Offsets to peak: [ 1  0  1  0 18 11  0  0  1  0]
*** MS5 Elapsed time for determine_offsets_to_peak: 0.001 seconds ***
Sorting times
*** MS5 Elapsed time for sorting times: 0.000 seconds ***
Removing out of bounds times
*** MS5 Elapsed time for removing out of bounds times: 0.000 seconds ***
Reordering units
*** MS5 Elapsed time for reordering units: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for creating sorting object: 0.001 seconds ***
*** MS5 Elapsed time for SCHEME2 sorting_scheme1: 1.948 seconds ***
*** MS5 Elapsed time for SCHEME2 get_times_labels_from_sorting: 0.001 seconds ***
Loading training traces
*** MS5 Elapsed time for SCHEME2 training_recording.get_traces: 0.030 seconds ***
Training classifier
*** MS5 Elapsed time for SCHEME2 training classifier step 1: 0.006 seconds ***
Adding snippets from phase 1 sorting
Fitting models
*** MS5 Elapsed time for SCHEME2 fitting models: 0.052 seconds ***
Chunk size: 585.1428864 sec
Time chunk 1 of 1
Loading traces
*** MS5 Elapsed time for SCHEME2 loading traces: 0.000 seconds ***
Detecting spikes
Adjacency for detect spikes with channel radius 50
[[0], [1], [2], [3], [4], [5], [6]]

Scheme 2 detected 12504 spikes in chunk 1 of 1
*** MS5 Elapsed time for SCHEME2 detecting spikes: 0.299 seconds ***
Extracting and classifying snippets
*** MS5 Elapsed time for SCHEME2 extracting and classifying snippets: 0.184 seconds ***
Updating events
Removing duplicates
*** MS5 Elapsed time for SCHEME2 updating events: 0.005 seconds ***
Concatenating results
*** MS5 Elapsed time for SCHEME2 concatenating results: 0.000 seconds ***
Perorming label mapping
*** MS5 Elapsed time for SCHEME2 label mapping: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for SCHEME2 creating sorting object: 0.001 seconds ***

Error

# error
File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\runsorter.py:224](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:224), in run_sorter_local(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, **sorter_params)
    [222](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:222) SorterClass.set_params_to_folder(recording, output_folder, sorter_params, verbose)
    [223](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:223) SorterClass.setup_recording(recording, output_folder, verbose=verbose)
--> [224](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:224) SorterClass.run_from_folder(output_folder, raise_error, verbose)
    [225](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:225) if with_output:
    [226](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/runsorter.py:226)     sorting = SorterClass.get_result_from_folder(output_folder, register_recording=True, sorting_info=True)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py:293](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:293), in BaseSorter.run_from_folder(cls, output_folder, raise_error, verbose)
    [290](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:290)         print(f"{sorter_name} run time {run_time:0.2f}s")
    [292](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:292) if has_error and raise_error:
--> [293](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:293)     raise SpikeSortingError(
    [294](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:294)         f"Spike sorting error trace:\n{log['error_trace']}\n"
    [295](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:295)         f"Spike sorting failed. You can inspect the runtime trace in {output_folder}/spikeinterface_log.json."
    [296](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:296)     )
    [298](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz/Lib/site-packages/spikeinterface/sorters/basesorter.py:298) return run_time

SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 634, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp47_157il\\recording.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\external\mountainsort5.py", line 189, in _run_from_folder
    with TemporaryDirectory(dir=p["temporary_base_dir"]) as tmpdir:
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 946, in __exit__
    self.cleanup()
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 950, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 808, in rmtree
    return _rmtree_unsafe(path, onexc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 636, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp47_157il\\recording.dat'

Spike sorting failed. You can inspect the runtime trace in [C:\Users\jesus.penalozaa\.tmp\0/spikeinterface_log.json.](file:///C:/Users/jesus.penalozaa/.tmp/0/spikeinterface_log.json.)

changing the filtering from False to True

# changing the filtering from False to True
tmp_folder = Path.home().joinpath(".tmp")
# %%
sorter_params = {
    "filter": True, # changed 
    "temporary_base_dir": tmp_folder.__str__(),
    "n_jobs_for_preprocessing": 1,
}
# %%

aggregate_sorting = si.run_sorter_by_property(
    sorter_name="mountainsort5",
    recording=recording_saved,
    grouping_property="group",
    working_folder=tmp_folder.__str__(),
    remove_existing_folder=True,
    **sorter_params,
)

Verbose

Using training recording of duration 300 sec with the sampling mode uniform
*** MS5 Elapsed time for SCHEME2 get_sampled_recording_for_training: 0.000 seconds ***
Running phase 1 sorting
Number of channels: 7
Number of timepoints: 7324218
Sampling frequency: 24414.0625 Hz
Channel 0: [150.   0.]
Channel 1: [150. 500.]
Channel 2: [150. 750.]
Channel 3: [ 150. 1000.]
Channel 4: [ 150. 1250.]
Channel 5: [ 150. 1500.]
Channel 6: [ 150. 1750.]
Loading traces
*** MS5 Elapsed time for load_traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 200
[[0], [1], [2], [3], [4], [5], [6]]

m = 0 (nbhd size: 1)
m = 1 (nbhd size: 1)
m = 2 (nbhd size: 1)
m = 3 (nbhd size: 1)
m = 4 (nbhd size: 1)
m = 5 (nbhd size: 1)
m = 6 (nbhd size: 1)
Detected 11558 spikes
*** MS5 Elapsed time for detect_spikes: 0.288 seconds ***
Removing duplicate times
*** MS5 Elapsed time for remove_duplicate_times: 0.001 seconds ***
Extracting 11358 snippets
*** MS5 Elapsed time for extract_snippets: 0.070 seconds ***
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.067 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
Found 11 clusters
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.673 seconds ***
Computing templates
*** MS5 Elapsed time for compute_templates: 0.027 seconds ***
Determining optimal alignment of templates
Template alignment converged.
Align templates offsets:  [ 0  0 -1  8  5  0 -1  0  0  1  0]
*** MS5 Elapsed time for align_templates: 0.066 seconds ***
Aligning snippets
*** MS5 Elapsed time for align_snippets: 0.013 seconds ***
Clustering aligned snippets
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.065 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.509 seconds ***
Found 9 clusters after alignment
Computing templates
*** MS5 Elapsed time for compute_templates: 0.026 seconds ***
Offsetting times to peak
Offsets to peak: [ 0 -1  0  5  8 -1  0  1  0]
*** MS5 Elapsed time for determine_offsets_to_peak: 0.000 seconds ***
Sorting times
*** MS5 Elapsed time for sorting times: 0.000 seconds ***
Removing out of bounds times
*** MS5 Elapsed time for removing out of bounds times: 0.000 seconds ***
Reordering units
*** MS5 Elapsed time for reordering units: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for creating sorting object: 0.001 seconds ***
*** MS5 Elapsed time for SCHEME2 sorting_scheme1: 1.820 seconds ***
*** MS5 Elapsed time for SCHEME2 get_times_labels_from_sorting: 0.001 seconds ***
Loading training traces
*** MS5 Elapsed time for SCHEME2 training_recording.get_traces: 0.028 seconds ***
Training classifier
*** MS5 Elapsed time for SCHEME2 training classifier step 1: 0.006 seconds ***
Adding snippets from phase 1 sorting
Fitting models
*** MS5 Elapsed time for SCHEME2 fitting models: 0.048 seconds ***
Chunk size: 585.1428864 sec
Time chunk 1 of 1
Loading traces
*** MS5 Elapsed time for SCHEME2 loading traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 50
[[0], [1], [2], [3], [4], [5], [6]]

Scheme 2 detected 11558 spikes in chunk 1 of 1
*** MS5 Elapsed time for SCHEME2 detecting spikes: 0.283 seconds ***
Extracting and classifying snippets
*** MS5 Elapsed time for SCHEME2 extracting and classifying snippets: 0.161 seconds ***
Updating events
Removing duplicates
*** MS5 Elapsed time for SCHEME2 updating events: 0.005 seconds ***
Concatenating results
*** MS5 Elapsed time for SCHEME2 concatenating results: 0.000 seconds ***
Perorming label mapping
*** MS5 Elapsed time for SCHEME2 label mapping: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for SCHEME2 creating sorting object: 0.001 seconds ***

Error

SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 634, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmpbk9w8djn\\recording.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\external\mountainsort5.py", line 189, in _run_from_folder
    with TemporaryDirectory(dir=p["temporary_base_dir"]) as tmpdir:
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 946, in __exit__
    self.cleanup()
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 950, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 808, in rmtree
    return _rmtree_unsafe(path, onexc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 636, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmpbk9w8djn\\recording.dat'

Spike sorting failed. You can inspect the runtime trace in [C:\Users\jesus.penalozaa\.tmp\0/spikeinterface_log.json.](file:///C:/Users/jesus.penalozaa/.tmp/0/spikeinterface_log.json.)

zm711 · 2024-02-16T15:13:21Z

The only other thing I was thinking about is it looks like you are in Juypter of some sort. I wonder if the TemporaryDirectory + Jupyter + Windows is contributing. I have no reason to think that in particular, but I'm just spit balling ideas right now.

jesusdpa1 · 2024-02-16T16:40:20Z

Thanks! I just tried the new idea and included another one changing the python version. moved the script to a different .py and got the following errors. from python=3.11.7 it seems to be a bad path name although the file exists and I can get to the path through the shell

Python=3.11.7
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\Users\jesus.penalozaa\.tmp\tmp7pydq656\recording.dat'

Spike sorting failed. You can inspect the runtime trace in C:\Users\jesus.penalozaa.tmp\0/spikeinterface_log.json.

.py

from pathlib import Path
import spikeinterface.full as si

print(f"SpikeInterface version: {si.__version__}")
import warnings

warnings.simplefilter("ignore")

home = Path.home()
dir_name = r"Documents\Data\SpikeInterface\02-09-24_8898-1_testSubject-HD EMG electrodes 5th try"
dir_path = home.joinpath(dir_name)
print(dir_path)

drv_dir = dir_path.joinpath("drv")

if not drv_dir.exists():
    drv_dir.mkdir()

save_folder = drv_dir.joinpath("preprocessed_compressed.zarr")
if (save_folder).is_dir():
    recording_saved = si.read_zarr(save_folder)
else:
    print("No preprocessed data found")

si.get_default_sorter_params("mountainsort5")

tmp_folder = Path.home().joinpath(".tmp")

sorter_params = {
    "filter": False,
    "temporary_base_dir": tmp_folder.__str__(),
    "n_jobs_for_preprocessing": 1,
}

aggregate_sorting = si.run_sorter_by_property(
    sorter_name="mountainsort5",
    recording=recording_saved,
    grouping_property="group",
    working_folder=tmp_folder.__str__(),
    remove_existing_folder=True,
    **sorter_params,
    verbose=True,
)

ERROR Shell/ Python=3.12

SpikeInterface version: 0.100.0
C:\Users\jesus.penalozaa\Documents\Data\SpikeInterface\02-09-24_8898-1_testSubject-HD EMG electrodes 5th try
whitening
write_binary_recording: 100%|##################################################################################| 61/61 [00:01<00:00, 39.81it/s]
Using training recording of duration 300 sec with the sampling mode uniform
*** MS5 Elapsed time for SCHEME2 get_sampled_recording_for_training: 0.000 seconds ***
Running phase 1 sorting
Number of channels: 7
Number of timepoints: 7324218
Sampling frequency: 24414.0625 Hz
Channel 0: [150.   0.]
Channel 1: [150. 500.]
Channel 2: [150. 750.]
Channel 3: [ 150. 1000.]
Channel 4: [ 150. 1250.]
Channel 5: [ 150. 1500.]
Channel 6: [ 150. 1750.]
Loading traces
*** MS5 Elapsed time for load_traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 200
[[0], [1], [2], [3], [4], [5], [6]]

m = 0 (nbhd size: 1)
m = 1 (nbhd size: 1)
m = 2 (nbhd size: 1)
m = 3 (nbhd size: 1)
m = 4 (nbhd size: 1)
m = 5 (nbhd size: 1)
m = 6 (nbhd size: 1)
Detected 12504 spikes
*** MS5 Elapsed time for detect_spikes: 0.298 seconds ***
Removing duplicate times
*** MS5 Elapsed time for remove_duplicate_times: 0.000 seconds ***
Extracting 12258 snippets
*** MS5 Elapsed time for extract_snippets: 0.074 seconds ***
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.073 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
Found 9 clusters
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.645 seconds ***
Computing templates
*** MS5 Elapsed time for compute_templates: 0.029 seconds ***
Determining optimal alignment of templates
Template alignment converged.
Align templates offsets:  [ 1  0 11 18  0  0  0  1  0]
*** MS5 Elapsed time for align_templates: 0.046 seconds ***
Aligning snippets
*** MS5 Elapsed time for align_snippets: 0.015 seconds ***
Clustering aligned snippets
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.073 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.689 seconds ***
Found 11 clusters after alignment
Computing templates
*** MS5 Elapsed time for compute_templates: 0.029 seconds ***
Offsetting times to peak
Offsets to peak: [ 1  0  1  1  0 18 11  0  0  1  0]
*** MS5 Elapsed time for determine_offsets_to_peak: 0.001 seconds ***
Sorting times
*** MS5 Elapsed time for sorting times: 0.000 seconds ***
Removing out of bounds times
*** MS5 Elapsed time for removing out of bounds times: 0.000 seconds ***
Reordering units
*** MS5 Elapsed time for reordering units: 0.001 seconds ***
Creating sorting object
*** MS5 Elapsed time for creating sorting object: 0.001 seconds ***
*** MS5 Elapsed time for SCHEME2 sorting_scheme1: 1.988 seconds ***
*** MS5 Elapsed time for SCHEME2 get_times_labels_from_sorting: 0.001 seconds ***
Loading training traces
*** MS5 Elapsed time for SCHEME2 training_recording.get_traces: 0.030 seconds ***
Training classifier
*** MS5 Elapsed time for SCHEME2 training classifier step 1: 0.006 seconds ***
Adding snippets from phase 1 sorting
Fitting models
*** MS5 Elapsed time for SCHEME2 fitting models: 0.049 seconds ***
Chunk size: 585.1428864 sec
Time chunk 1 of 1
Loading traces
*** MS5 Elapsed time for SCHEME2 loading traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 50
[[0], [1], [2], [3], [4], [5], [6]]

Scheme 2 detected 12504 spikes in chunk 1 of 1
*** MS5 Elapsed time for SCHEME2 detecting spikes: 0.299 seconds ***
Extracting and classifying snippets
*** MS5 Elapsed time for SCHEME2 extracting and classifying snippets: 0.184 seconds ***
Updating events
Removing duplicates
*** MS5 Elapsed time for SCHEME2 updating events: 0.006 seconds ***
Concatenating results
*** MS5 Elapsed time for SCHEME2 concatenating results: 0.000 seconds ***
Perorming label mapping
*** MS5 Elapsed time for SCHEME2 label mapping: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for SCHEME2 creating sorting object: 0.001 seconds ***
Error running mountainsort5
Traceback (most recent call last):
  File "C:\Users\jesus.penalozaa\Documents\GitHub\ephys-robot\tests\ms5_shell.py", line 35, in <module>
    aggregate_sorting = si.run_sorter_by_property(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\launcher.py", line 297, in run_sorter_by_property
    sorting_list = run_sorter_jobs(job_list, engine=engine, engine_kwargs=engine_kwargs, return_output=True)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\launcher.py", line 106, in run_sorter_jobs
    sorting = run_sorter(**kwargs)
              ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\runsorter.py", line 174, in run_sorter
    return run_sorter_local(**common_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\runsorter.py", line 224, in run_sorter_local
    SorterClass.run_from_folder(output_folder, raise_error, verbose)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 293, in run_from_folder
    raise SpikeSortingError(
spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 634, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp7oklpeui\\recording.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\site-packages\spikeinterface\sorters\external\mountainsort5.py", line 189, in _run_from_folder
    with TemporaryDirectory(dir=p["temporary_base_dir"]) as tmpdir:
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 946, in __exit__
    self.cleanup()
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 950, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 930, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 808, in rmtree
    return _rmtree_unsafe(path, onexc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\shutil.py", line 636, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz\Lib\tempfile.py", line 905, in onexc
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp7oklpeui\\recording.dat'

Spike sorting failed. You can inspect the runtime trace in C:\Users\jesus.penalozaa\.tmp\0/spikeinterface_log.json.

Shell/ Python=3.11

SpikeInterface version: 0.100.0
C:\Users\jesus.penalozaa\Documents\Data\SpikeInterface\02-09-24_8898-1_testSubject-HD EMG electrodes 5th try
whitening
write_binary_recording: 100%|##################################################################################| 61/61 [00:02<00:00, 26.92it/s]
Using training recording of duration 300 sec with the sampling mode uniform
*** MS5 Elapsed time for SCHEME2 get_sampled_recording_for_training: 0.000 seconds ***
Running phase 1 sorting
Number of channels: 7
Number of timepoints: 7324218
Sampling frequency: 24414.0625 Hz
Channel 0: [150.   0.]
Channel 1: [150. 500.]
Channel 2: [150. 750.]
Channel 3: [ 150. 1000.]
Channel 4: [ 150. 1250.]
Channel 5: [ 150. 1500.]
Channel 6: [ 150. 1750.]
Loading traces
*** MS5 Elapsed time for load_traces: 0.001 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 200
[[0], [1], [2], [3], [4], [5], [6]]

m = 0 (nbhd size: 1)
m = 1 (nbhd size: 1)
m = 2 (nbhd size: 1)
m = 3 (nbhd size: 1)
m = 4 (nbhd size: 1)
m = 5 (nbhd size: 1)
m = 6 (nbhd size: 1)
Detected 12504 spikes
*** MS5 Elapsed time for detect_spikes: 0.310 seconds ***
Removing duplicate times
*** MS5 Elapsed time for remove_duplicate_times: 0.000 seconds ***
Extracting 12258 snippets
*** MS5 Elapsed time for extract_snippets: 0.068 seconds ***
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.078 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
Found 9 clusters
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.636 seconds ***
Computing templates
*** MS5 Elapsed time for compute_templates: 0.029 seconds ***
Determining optimal alignment of templates
Template alignment converged.
Align templates offsets:  [ 1  0 11 18  0  0  0  1  0]
*** MS5 Elapsed time for align_templates: 0.046 seconds ***
Aligning snippets
*** MS5 Elapsed time for align_snippets: 0.017 seconds ***
Clustering aligned snippets
Computing PCA features with npca=21
*** MS5 Elapsed time for compute_pca_features: 0.077 seconds ***
Isosplit6 clustering with npca_per_subdivision=10
*** MS5 Elapsed time for isosplit6_subdivision_method: 0.557 seconds ***
Found 10 clusters after alignment
Computing templates
*** MS5 Elapsed time for compute_templates: 0.033 seconds ***
Offsetting times to peak
Offsets to peak: [ 1  0  1  0 18 11  0  0  1  0]
*** MS5 Elapsed time for determine_offsets_to_peak: 0.000 seconds ***
Sorting times
*** MS5 Elapsed time for sorting times: 0.000 seconds ***
Removing out of bounds times
*** MS5 Elapsed time for removing out of bounds times: 0.000 seconds ***
Reordering units
*** MS5 Elapsed time for reordering units: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for creating sorting object: 0.002 seconds ***
*** MS5 Elapsed time for SCHEME2 sorting_scheme1: 1.870 seconds ***
*** MS5 Elapsed time for SCHEME2 get_times_labels_from_sorting: 0.001 seconds ***
Loading training traces
*** MS5 Elapsed time for SCHEME2 training_recording.get_traces: 0.037 seconds ***
Training classifier
*** MS5 Elapsed time for SCHEME2 training classifier step 1: 0.006 seconds ***
Adding snippets from phase 1 sorting
Fitting models
*** MS5 Elapsed time for SCHEME2 fitting models: 0.052 seconds ***
Chunk size: 585.1428864 sec
Time chunk 1 of 1
Loading traces
*** MS5 Elapsed time for SCHEME2 loading traces: 0.000 seconds ***
Detecting spikes

Adjacency for detect spikes with channel radius 50
[[0], [1], [2], [3], [4], [5], [6]]

Scheme 2 detected 12504 spikes in chunk 1 of 1
*** MS5 Elapsed time for SCHEME2 detecting spikes: 0.312 seconds ***
Extracting and classifying snippets
*** MS5 Elapsed time for SCHEME2 extracting and classifying snippets: 0.181 seconds ***
Updating events
Removing duplicates
*** MS5 Elapsed time for SCHEME2 updating events: 0.005 seconds ***
Concatenating results
*** MS5 Elapsed time for SCHEME2 concatenating results: 0.000 seconds ***
Perorming label mapping
*** MS5 Elapsed time for SCHEME2 label mapping: 0.000 seconds ***
Creating sorting object
*** MS5 Elapsed time for SCHEME2 creating sorting object: 0.001 seconds ***
Error running mountainsort5
---------------------------------------------------------------------------
SpikeSortingError                         Traceback (most recent call last)
File ~\Documents\GitHub\ephys-robot\tests\ms5_shell.py:35
     27 tmp_folder = Path.home().joinpath(".tmp")
     29 sorter_params = {
     30     "filter": False,
     31     "temporary_base_dir": tmp_folder.__str__(),
     32     "n_jobs_for_preprocessing": 1,
     33 }
---> 35 aggregate_sorting = si.run_sorter_by_property(
     36     sorter_name="mountainsort5",
     37     recording=recording_saved,
     38     grouping_property="group",
     39     working_folder=tmp_folder.__str__(),
     40     remove_existing_folder=True,
     41     **sorter_params,
     42     verbose=True,
     43 )

File ~\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\launcher.py:297, in run_sorter_by_property(sorter_name, recording, grouping_property, working_folder, mode_if_folder_exists, engine, engine_kwargs, verbose, docker_image, singularity_image, **sorter_params)
    286     job = dict(
    287         sorter_name=sorter_name,
    288         recording=rec,
   (...)
    293         **sorter_params,
    294     )
    295     job_list.append(job)
--> 297 sorting_list = run_sorter_jobs(job_list, engine=engine, engine_kwargs=engine_kwargs, return_output=True)
    299 unit_groups = []
    300 for sorting, group in zip(sorting_list, recording_dict.keys()):

File ~\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\launcher.py:106, in run_sorter_jobs(job_list, engine, engine_kwargs, return_output)
    103 if engine == "loop":
    104     # simple loop in main process
    105     for kwargs in job_list:
--> 106         sorting = run_sorter(**kwargs)
    107         if return_output:
    108             out.append(sorting)

File ~\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\runsorter.py:174, in run_sorter(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, docker_image, singularity_image, delete_container_files, with_output, **sorter_params)
    167             container_image = singularity_image
    168     return run_sorter_container(
    169         container_image=container_image,
    170         mode=mode,
    171         **common_kwargs,
    172     )
--> 174 return run_sorter_local(**common_kwargs)

File ~\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\runsorter.py:224, in run_sorter_local(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, **sorter_params)
    222 SorterClass.set_params_to_folder(recording, output_folder, sorter_params, verbose)
    223 SorterClass.setup_recording(recording, output_folder, verbose=verbose)
--> 224 SorterClass.run_from_folder(output_folder, raise_error, verbose)
    225 if with_output:
    226     sorting = SorterClass.get_result_from_folder(output_folder, register_recording=True, sorting_info=True)

File ~\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\basesorter.py:293, in BaseSorter.run_from_folder(cls, output_folder, raise_error, verbose)
    290         print(f"{sorter_name} run time {run_time:0.2f}s")
    292 if has_error and raise_error:
--> 293     raise SpikeSortingError(
    294         f"Spike sorting error trace:\n{log['error_trace']}\n"
    295         f"Spike sorting failed. You can inspect the runtime trace in {output_folder}/spikeinterface_log.json."
    296     )
    298 return run_time

SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp7pydq656\\recording.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\tempfile.py", line 878, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp7pydq656\\recording.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\site-packages\spikeinterface\sorters\external\mountainsort5.py", line 189, in _run_from_folder
    with TemporaryDirectory(dir=p["temporary_base_dir"]) as tmpdir:
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\tempfile.py", line 904, in __exit__
    self.cleanup()
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\tempfile.py", line 908, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\tempfile.py", line 881, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\shutil.py", line 603, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "C:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-11\Lib\shutil.py", line 600, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
         ^^^^^^^^^^^^^^^^
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\jesus.penalozaa\\.tmp\\tmp7pydq656\\recording.dat'

Spike sorting failed. You can inspect the runtime trace in C:\Users\jesus.penalozaa\.tmp\0/spikeinterface_log.json.

~\.tmp> ls                                                                                                                                                                                    02/16/24 11:38:35 AM
╭───┬─────────────┬──────┬─────────┬────────────────╮
│ # │    name     │ type │  size   │    modified    │
├───┼─────────────┼──────┼─────────┼────────────────┤
│ 0 │ 0           │ dir  │ 4.0 KiB │ 27 seconds ago │
│ 1 │ tmp7oklpeui │ dir  │     0 B │ 20 minutes ago │
│ 2 │ tmp7pydq656 │ dir  │     0 B │ 12 minutes ago │
│ 3 │ tmpbk9w8djn │ dir  │     0 B │ 2 hours ago    │
│ 4 │ tmpddh34k3j │ dir  │     0 B │ 21 minutes ago │
│ 5 │ tmph519o4gh │ dir  │     0 B │ 30 minutes ago │
│ 6 │ tmpkhnuf7tx │ dir  │     0 B │ 32 seconds ago │
╰───┴─────────────┴──────┴─────────┴────────────────╯

zm711 · 2024-02-16T18:14:12Z

Two more things:

which terminal are you using. It seems command prompt works best, but since you staying within python and not moving to a different program (like Matlab) I don't think that is the problem.
Could you try not doing the .. I'm wondering if the .tmp is causing some problem on windows which is super particular about its paths? So maybe just call it tmp and not .tmp

zm711 · 2024-02-16T18:57:53Z

So I just got this Mountainsort5 error to repeat. This has come up before with trying to use TemporaryDirectory on Windows. In 3.12 they finally decided to add a delete argument just to prevent this error on windows. Then the user needs to cleanup themselves.
@magland, is there any reason why you can't do this the way kilosort.py does this (ie, write the binary file to use and then give an argument to delete the binary itself later). Otherwise I wonder if you'll lose Windows users. This seems to be intermittent failures in the cleanup. I can't predict how or why this happens to be honest. The context manager has extra arguments added in 3.10 and 3.12 to make it useable by Windows users, but "useable" just means that it won't try to delete and then the user would need to go delete anyway.

jesusdpa1 · 2024-02-16T20:02:45Z

maybe not useful but I think on windows tmpfile is open automatically and never closed unless indicated, is there place where tmpdir.close() can be implemented? I couldn't find it on the github repo

zm711 · 2024-02-16T20:18:29Z

https://docs.python.org/3/library/tempfile.html

Here are the docs @jesusdpa1. Basically when using tempdir the user has to clean up. The goal of TemporaryDirectory as a context manager is that it handles all the cleanup. Unfortunately it is a well known problem with Windows that the file is permanently locked once opened (until the full process completes) and it is actually non-trivial to get the file to close. In other OSes like macOS and Linux files to be deleted are flagged for deletion and deleted when the OS is able to, but on Windows it tries to do the process right away which leads to the failures. So the options for a Windows user are to try and try again until it works or use WSL to use Linux file system management rather than Windows. (Unless the context manager is removed from mountainsort5).

zm711 · 2024-02-16T22:50:09Z

@jesusdpa1,

If you are comfortable installing from source (which it doesn't look like you're currently so happy to help you do that if you want) and using at least python 3.10 (which it seems like you are) then the work around is to open the file mounainsort5.py and edit this line:

spikeinterface/src/spikeinterface/sorters/external/mountainsort5.py

Line 189 in d459b4e

with TemporaryDirectory(dir=p["temporary_base_dir"]) as tmpdir:

with TemporaryDirectory(dir=p["temporary_base_dir"], ignore_cleanup_error=True) as tmpdir:

Then you just have to go and delete the temp folder yourself that you put in temporary_base_dir. In this case for some files it might succeed and you don't need to cleanup for others the cleanup will fail, but it won't crash the program. It will just pass and return the sorting object into your terminal.

Otherwise we wait for mountainsort5 to be edited.

jesusdpa1 · 2024-02-20T18:51:15Z

Thanks!

I follow these instructions to install spikeinterface, and made the modifications

# Python 3.10 
git clone https://github.com/SpikeInterface/spikeinterface.git
cd spikeinterface
pip install -e .
cd ..

but I got the following error

SpikeSortingError                         Traceback (most recent call last)
File [c:\Users\jesus.penalozaa\Documents\GitHub\ephys-robot\tests\ms5_shell.py:33](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:33)
     [26](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:26) print(tmp_folder.absolute())
     [27](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:27) sorter_params = {
     [28](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:28)     "filter": False,
     [29](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:29)     # "temporary_base_dir": tmp_folder.absolute().__str__(),
     [30](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:30)     "n_jobs_for_preprocessing": 1,
     [31](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:31) }
---> [33](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:33) aggregate_sorting = si.run_sorter_by_property(
     [34](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:34)     sorter_name="mountainsort5",
     [35](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:35)     recording=recording_saved,
     [36](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:36)     grouping_property="group",
     [37](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:37)     working_folder=tmp_folder.absolute().__str__(),
     [38](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:38)     remove_existing_folder=True,
     [39](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:39)     **sorter_params,
     [40](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:40)     verbose=True,
     [41](file:///C:/Users/jesus.penalozaa/Documents/GitHub/ephys-robot/tests/ms5_shell.py:41) )

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\launcher.py:297](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:297), in run_sorter_by_property(sorter_name, recording, grouping_property, working_folder, mode_if_folder_exists, engine, engine_kwargs, verbose, docker_image, singularity_image, **sorter_params)
    [286](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:286)     job = dict(
    [287](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:287)         sorter_name=sorter_name,
    [288](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:288)         recording=rec,
   (...)
    [293](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:293)         **sorter_params,
    [294](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:294)     )
    [295](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:295)     job_list.append(job)
--> [297](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:297) sorting_list = run_sorter_jobs(job_list, engine=engine, engine_kwargs=engine_kwargs, return_output=True)
    [299](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:299) unit_groups = []
    [300](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:300) for sorting, group in zip(sorting_list, recording_dict.keys()):

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\launcher.py:106](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:106), in run_sorter_jobs(job_list, engine, engine_kwargs, return_output)
    [103](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:103) if engine == "loop":
    [104](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:104)     # simple loop in main process
    [105](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:105)     for kwargs in job_list:
--> [106](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:106)         sorting = run_sorter(**kwargs)
    [107](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:107)         if return_output:
    [108](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/launcher.py:108)             out.append(sorting)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\runsorter.py:174](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:174), in run_sorter(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, docker_image, singularity_image, delete_container_files, with_output, **sorter_params)
    [167](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:167)             container_image = singularity_image
    [168](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:168)     return run_sorter_container(
    [169](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:169)         container_image=container_image,
    [170](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:170)         mode=mode,
    [171](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:171)         **common_kwargs,
    [172](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:172)     )
--> [174](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:174) return run_sorter_local(**common_kwargs)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\runsorter.py:224](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:224), in run_sorter_local(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, **sorter_params)
    [222](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:222) SorterClass.set_params_to_folder(recording, output_folder, sorter_params, verbose)
    [223](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:223) SorterClass.setup_recording(recording, output_folder, verbose=verbose)
--> [224](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:224) SorterClass.run_from_folder(output_folder, raise_error, verbose)
    [225](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:225) if with_output:
    [226](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/runsorter.py:226)     sorting = SorterClass.get_result_from_folder(output_folder, register_recording=True, sorting_info=True)

File [c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\basesorter.py:293](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:293), in BaseSorter.run_from_folder(cls, output_folder, raise_error, verbose)
    [290](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:290)         print(f"{sorter_name} run time {run_time:0.2f}s")
    [292](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:292) if has_error and raise_error:
--> [293](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:293)     raise SpikeSortingError(
    [294](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:294)         f"Spike sorting error trace:\n{log['error_trace']}\n"
    [295](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:295)         f"Spike sorting failed. You can inspect the runtime trace in {output_folder}/spikeinterface_log.json."
    [296](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:296)     )
    [298](file:///C:/Users/jesus.penalozaa/AppData/Local/miniconda3/envs/roboz-10/lib/site-packages/spikeinterface/sorters/basesorter.py:298) return run_time

SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\basesorter.py", line 258, in run_from_folder
    SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
  File "c:\Users\jesus.penalozaa\AppData\Local\miniconda3\envs\roboz-10\lib\site-packages\spikeinterface\sorters\external\mountainsort5.py", line 189, in _run_from_folder
    with TemporaryDirectory(dir=p["temporary_base_dir"], ignore_cleanup_error=True) as tmpdir:
TypeError: TemporaryDirectory.__init__() got an unexpected keyword argument 'ignore_cleanup_error'

Spike sorting failed. You can inspect the runtime trace in [C:\Users\jesus.penalozaa\tmp\0/spikeinterface_log.json.](file:///C:/Users/jesus.penalozaa/tmp/0/spikeinterface_log.json.)

zm711 · 2024-02-20T18:53:14Z

Completely my fault. I mistyped.
It's ignore_cleanup_errors. Notice the s that I forgot to include.

jesusdpa1 · 2024-02-20T19:09:16Z

Thanks! it worked 👍

If I have other question regarding on how to optimize ms5 for spike sorting, is there a discord channel available? I am starting to collect data using myomatrix electrodes on muscles [https://camber.emory.edu/technology/index.html] and would like to make sure the probe design input to spikeinterface and the waveform extraction is correct.

zm711 · 2024-02-20T19:19:32Z

There's not a discord, but there is a slack. I'm not sure who admins that so I'll tag @alejoe91 and I'm sure he can help get you set up. In general probe based questions can go to probeinterface waveform extraction questions would go here. And if you have electrophysiology reading (ie from one of the extractors in the spikeinterface.extractors module) would go to neo. This issue tracker is pretty responsive so most questions can go here just so they can help the overall community (I know other libraries really try to divide between a chat and the issue tracker, but again @alejoe91 can give the definitive answer for that).

alejoe91 · 2024-04-08T09:35:50Z

@jesusdpa1 can you send us an email? I'll send you a slack invite there!

zm711 · 2024-04-10T15:38:46Z

Closed by #2690.

We no longer need the temp dir fix for use on Windows.

zm711 added the bug in sorter The bug is in sorter code itself, not in SI label Feb 15, 2024

zm711 mentioned this issue Feb 16, 2024

TemporaryDirectory context manager failing on Windows flatironinstitute/mountainsort5#31

Closed

zm711 mentioned this issue Mar 20, 2024

Mountainsort5 Wrapper Todo list #2607

Closed

zm711 mentioned this issue Apr 4, 2024

Mountiansort5 wrapper tempdir error #2670

Closed

zm711 closed this as completed Apr 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error using si.run_sorter [Mountainsort5] with TDT data on Windows 10 #2489

Error using si.run_sorter [Mountainsort5] with TDT data on Windows 10 #2489

jesusdpa1 commented Feb 14, 2024

zm711 commented Feb 15, 2024

jesusdpa1 commented Feb 15, 2024

zm711 commented Feb 16, 2024

zm711 commented Feb 16, 2024

jesusdpa1 commented Feb 16, 2024

zm711 commented Feb 16, 2024

jesusdpa1 commented Feb 16, 2024

zm711 commented Feb 16, 2024

zm711 commented Feb 16, 2024

jesusdpa1 commented Feb 16, 2024

zm711 commented Feb 16, 2024 •

edited

Loading

zm711 commented Feb 16, 2024 •

edited

Loading

jesusdpa1 commented Feb 20, 2024 •

edited

Loading

zm711 commented Feb 20, 2024

jesusdpa1 commented Feb 20, 2024

zm711 commented Feb 20, 2024

alejoe91 commented Apr 8, 2024

zm711 commented Apr 10, 2024

Error using si.run_sorter [Mountainsort5] with TDT data on Windows 10 #2489

Error using si.run_sorter [Mountainsort5] with TDT data on Windows 10 #2489

Comments

jesusdpa1 commented Feb 14, 2024

zm711 commented Feb 15, 2024

jesusdpa1 commented Feb 15, 2024

zm711 commented Feb 16, 2024

zm711 commented Feb 16, 2024

jesusdpa1 commented Feb 16, 2024

Changing the function

Verbose

Error

changing the filtering from False to True

Verbose

Error

zm711 commented Feb 16, 2024

jesusdpa1 commented Feb 16, 2024

.py

ERROR Shell/ Python=3.12

Shell/ Python=3.11

zm711 commented Feb 16, 2024

zm711 commented Feb 16, 2024

jesusdpa1 commented Feb 16, 2024

zm711 commented Feb 16, 2024 • edited Loading

zm711 commented Feb 16, 2024 • edited Loading

jesusdpa1 commented Feb 20, 2024 • edited Loading

zm711 commented Feb 20, 2024

jesusdpa1 commented Feb 20, 2024

zm711 commented Feb 20, 2024

alejoe91 commented Apr 8, 2024

zm711 commented Apr 10, 2024

zm711 commented Feb 16, 2024 •

edited

Loading

zm711 commented Feb 16, 2024 •

edited

Loading

jesusdpa1 commented Feb 20, 2024 •

edited

Loading