Skip to content

mcorr fails if input video has less than 1000 frames #180

Open
@kushalkolar

Description

@kushalkolar

traceback, this input video had 300 frames:

---------------------------------------------------------------------------
BatchItemUnsuccessfulError                Traceback (most recent call last)
Cell In [24], line 2
      1 df = df.caiman.reload_from_disk()
----> 2 df.iloc[-1].mcorr.get_output()

File ~/repos/mesmerize-core/mesmerize_core/caiman_extensions/_utils.py:24, in validate.<locals>.dec.<locals>.wrapper(self, *args, **kwargs)
     22 if not self._series["outputs"]["success"]:
     23     tb = self._series["outputs"]["traceback"]
---> 24     raise BatchItemUnsuccessfulError(f"Batch item was unsuccessful, traceback from subprocess:\n{tb}")
     25 return func(self, *args, **kwargs)

BatchItemUnsuccessfulError: Batch item was unsuccessful, traceback from subprocess:
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/kushalk/repos/CaImAn/caiman/summary_images.py", line 784, in local_correlations_movie_parallel
    mv = cm.load(mv_name, subindices=idx, in_memory=True)
  File "/home/kushalk/repos/CaImAn/caiman/base/movies.py", line 1447, in load
    return load_movie_chain(file_name,
  File "/home/kushalk/repos/CaImAn/caiman/base/movies.py", line 1774, in load_movie_chain
    m = load(f,
  File "/home/kushalk/repos/CaImAn/caiman/base/movies.py", line 1706, in load
    images = images[subindices]
  File "/home/kushalk/python-venvs/mescore/lib/python3.10/site-packages/numpy/core/memmap.py", line 334, in __getitem__
    res = super().__getitem__(index)
IndexError: index -700 is out of bounds for axis 0 with size 300
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/kushalk/repos/mesmerize-core/mesmerize_core/algorithms/mcorr.py", line 95, in run_algo
    Cns = local_correlations_movie_offline(
  File "/home/kushalk/repos/CaImAn/caiman/summary_images.py", line 773, in local_correlations_movie_offline
    parallel_result = dview.map_async(local_correlations_movie_parallel, params).get(4294967)
  File "/usr/local/lib/python3.10/multiprocessing/pool.py", line 771, in get
    raise self._value
IndexError: index -700 is out of bounds for axis 0 with size 300

The issue actually comes from computing the local correlations, we set the default window_size to 1000: https://github.com/nel-lab/mesmerize-core/blob/master/mesmerize_core/algorithms/mcorr.py#L98

We should probably allow a parameter outside of "main" where these things can be set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions