Loading in Blackrock Data

[mtjacobsResearcher]

My lab is very new to using Spike Interface. We are trying to load Blackrock data in, but we are getting an error.

AssertionError: Inconsistent ns2 and nev file. 559 segments present in .nev file, but 280 in ns2 file.

It seems to be always twice as many seggments in the .nev than in the .ns2 or .ns5 file. Can you provide any guidance on why we might be facing this issue?

[samuelgarcia]

Hi @mtjacobsResearcher
the reader are based on neo.
Sometime it is good to try directly with the neo interface and send a bug there.

But in your case this is strange because in spikeinterface the nev file is not supposed to be read!
Is it a big dataset ?

Could you share the minimal code to reproduce this and copy/paste the full tarces ?

[mtjacobsResearcher]

Thank you so much for your feedback. Thank you for being patient with a trainee.

We removed the .nev and it worked....

Thank you so much!!!!!

[samuelgarcia]

I re open this because even withg the nev inside the folder this should works!!!!!

[mtjacobsResearcher]

Sounds good. I will show you everything I have going on. Basically we took the tutorial Jupiter notebook, and tried to make our own cells doing what the tutorial did but with our own data. We got stuck on step one. And no, the data sets are extremely tiny. I am currently testing on single cell recordings, but we plan on making a pipeline for utah arrays afterwards.

#Matt & Adarsh's test
base_folder2=Path('\Users\User\Documents\GitHub\Spike interface tutorial\')
oe_folder2 = base_folder2 / 'BR_test_2/Tony_2023_12_04_001.ns5'
print(oe_folder2)
full_raw_rec2 = si.read_blackrock(oe_folder2)

#this code works when we only had a .ns5 in the folder. I have tried it for several different days of recordings and the .nev always have 2*n+1 segments to the n segments of the .nsx (2 or 5).

Here is the output when I add a .nev
{
"name": "AssertionError",
"message": "Inconsistent ns5 and nev file. 773 segments present in .nev file, but 387 in ns5 file.",
"stack": "---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
c:\Users\User\Documents\GitHub\Spike interface tutorial\SpikeInterface_Tutorial.ipynb Cell 11 line 5
3 oe_folder2 = base_folder2 / 'BR_test_2/Tony_2023_12_04_001.ns5'
4 print(oe_folder2)
----> 5 full_raw_rec2 = si.read_blackrock(oe_folder2)

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages\spikeinterface\extractors
eoextractors\blackrock.py:51, in BlackrockRecordingExtractor.init(self, file_path, stream_id, stream_name, block_index, all_annotations, use_names_as_ids)
49 if ".ns" in suffix:
50 neo_kwargs["nsx_to_load"] = int(suffix[-1])
---> 51 NeoBaseRecordingExtractor.init(
52 self,
53 stream_id=stream_id,
54 stream_name=stream_name,
55 all_annotations=all_annotations,
56 use_names_as_ids=use_names_as_ids,
57 **neo_kwargs,
58 )
59 self._kwargs.update({"file_path": str(Path(file_path).absolute())})

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages\spikeinterface\extractors
eoextractors
eobaseextractor.py:185, in NeoBaseRecordingExtractor.init(self, stream_id, stream_name, block_index, all_annotations, use_names_as_ids, **neo_kwargs)
156 def init(
157 self,
158 stream_id: Optional[str] = None,
(...)
163 **neo_kwargs: Dict[str, Any],
164 ) -> None:
165 """
166 Initialize a NeoBaseRecordingExtractor instance.
167
(...)
182
183 """
--> 185 _NeoBaseExtractor.init(self, block_index, **neo_kwargs)
187 kwargs = dict(all_annotations=all_annotations)
188 if block_index is not None:

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages\spikeinterface\extractors
eoextractors
eobaseextractor.py:25, in _NeoBaseExtractor.init(self, block_index, **neo_kwargs)
23 def init(self, block_index, **neo_kwargs):
24 if not hasattr(self, "neo_reader"): # Avoid double initialization
---> 25 self.neo_reader = self.get_neo_io_reader(self.NeoRawIOClass, **neo_kwargs)
27 if self.neo_reader.block_count() > 1 and block_index is None:
28 raise Exception(
29 "This dataset is multi-block. Spikeinterface can load one block at a time. "
30 "Use 'block_index' to select the block to be loaded."
31 )

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages\spikeinterface\extractors
eoextractors
eobaseextractor.py:64, in _NeoBaseExtractor.get_neo_io_reader(cls, raw_class, **neo_kwargs)
62 neoIOclass = getattr(rawio_module, raw_class)
63 neo_reader = neoIOclass(**neo_kwargs)
---> 64 neo_reader.parse_header()
66 return neo_reader

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages
eo\rawio\baserawio.py:179, in BaseRawIO.parse_header(self)
166 def parse_header(self):
167 """
168 This must parse the file header to get all stuff for fast use later on.
169
(...)
177
178 """
--> 179 self._parse_header()
180 self._check_stream_signal_channel_characteristics()

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages
eo\rawio\blackrockrawio.py:359, in BlackrockRawIO._parse_header(self)
357 if self._avail_files['nev']:
358 for nsx_nb in self.nsx_to_load:
--> 359 self.__match_nsx_and_nev_segment_ids(nsx_nb)
361 self.nsx_datas = {}
362 self.sig_sampling_rates = {}

File c:\Users\User\miniconda3\envs\si_env\lib\site-packages
eo\rawio\blackrockrawio.py:1269, in BlackrockRawIO.__match_nsx_and_nev_segment_ids(self, nsx_nb)
1266 ev_ids[mask_after_seg] += 1
1268 # consistency check: same number of segments for nsx and nev data
-> 1269 assert nb_possible_nev_segments == len(nonempty_nsx_segments), \
1270 ('Inconsistent ns{0} and nev file. {1} segments present in .nev file, but {2} in '
1271 'ns{0} file.'.format(nsx_nb, nb_possible_nev_segments,
1272 len(nonempty_nsx_segments)))
1274 new_nev_segment_id_mapping = dict(zip(range(nb_possible_nev_segments),
1275 sorted(list(nonempty_nsx_segments))))
1277 # replacing event ids by matched event ids in place

AssertionError: Inconsistent ns5 and nev file. 773 segments present in .nev file, but 387 in ns5 file."
}