LinAlgError: SVD did not converge in tests on main branch

[Udayscode]

Describe the bug
Tests in movement are failing on the main branch with the error numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares. This occurs during filtering tests that use the Savitzky-Golay filter (savgol_filter) on datasets with NaN values.

To Reproduce

1. Clone the movement repository: git clone https://github.com/neuroinformatics-unit/movement.git

2. Checkout the main branch: git checkout main

3. Install dependencies: pip install .[dev] (uses pyproject.toml)

4. Run the tests: pytest tests/

5. The following tests fail with numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares:

   • tests/test_integration/test_filtering.py::test_nan_propagation_through_filters (4 tests)

   • tests/test_unit/test_filtering.py::TestFilteringValidDataset::test_filter_with_nans_on_position (2 tests)

   • tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window (8 tests)

Expected behaviour
These tests should pass on the main branch, as they are designed to verify that the Savitzky-Golay filter correctly handles NaN values in datasets (e.g., valid_poses_dataset_with_nan, valid_bboxes_dataset_with_nan). The filter should either handle NaNs gracefully or the tests should account for such failures.

Log file
=============================================== short test summary info ===============================================
FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[3] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[5] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[6] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[13] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDataset::test_filter_with_nans_on_position[savgol_filter-filter_kwargs1-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDataset::test_filter_with_nans_on_position[savgol_filter-filter_kwargs1-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-3-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-3-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-5-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-5-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-6-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-6-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-10-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares
FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-10-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

Computer used (please complete the following information):

• OS: Windows 11

Additional context

• I’m using Python 3.13.2 and NumPy version 2.1.3. This is a relatively new Python version, which might be causing compatibility issues if movement is tested with older versions (e.g., Python 3.9).
• This was identified while working on PR Rename source format #525, where @niksirbi noted that these failures might be a broader issue since they are reproducible on main. I’m happy to assist with debugging if there are any specific steps to try (e.g., testing with a different NumPy version).

[niksirbi]

Thanks for reporting this @Udayscode!

pip install .[test]

Do you perhaps mean pip install .[dev]?

I’m using Python 3.13.2 and NumPy version 2.1.3. This is a relatively new Python version, which might be causing compatibility issues if movement is tested with older versions (e.g., Python 3.9)

That's not the case, we test on recent versions, including with Python 3.13 on Windows.

To help diagnose this could you run movement info on a terminal and paste the output here?
Similarly, please also run pip show bottleneck and report the output.

[Udayscode]

Hi @niksirbi,

Apologies for the confusion —I had mistakenly written the wrong command again. I’ve now corrected it and rerun the installation with the correct command, pip install .[dev], to ensure all development dependencies are properly installed.

As requested, here are the outputs:

Output of movement info:

 movement: 0.1.dev229+g7d6ad36
 Python: 3.12.9
 NumPy: 2.1.3
 xarray: 2025.3.0
 pandas: 2.2.3
 napari: 0.5.6
 Platform: Windows-11-10.0.22631-SP0

Output of pip show bottleneck:

Name: Bottleneck
Version: 1.4.2
Summary: Fast NumPy array functions written in C
Home-page: https://github.com/pydata/bottleneck
Author:
Author-email:
License: Simplified BSD
Location: C:\Users\Dell\.conda\envs\movement_env\Lib\site-packages
Requires: numpy
Required-by:

[niksirbi]

Alright, thanks.
As a first step we'll see if we can replicate the bug on one of our Windows machines.

[lochhh]

Hey @Udayscode I ran the tests on Windows 11 with both:
Python: 3.12.9; NumPy: 2.1.3 and Python: 3.13.2; NumPy: 2.1.3
and can't replicate your issue.
Could you please try to recreate your conda env?

[Udayscode]

Thanks @lochhh , @niksirbi , for your continued guidance! I wanted to provide an update on the issue. As suggested, I recreated my conda environment using:

conda create -n movement_env -c conda-forge python=3.13.2 movement napari pyqt vispy pytest pytest-cov
conda activate movement_env
pip install .[dev]

I also resolved the vispy build error by installing it via conda-forge.

However, even after recreating the environment and confirming my OS, I’m still seeing the LinAlgError: SVD did not converge in Linear Least Squares when running pytest tests/. The test failures are the same as before:

=============================================== short test summary info =============================================== FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[3] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[5] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[6] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_integration/test_filtering.py::test_nan_propagation_through_filters[13] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDataset::test_filter_with_nans_on_position[savgol_filter-filter_kwargs1-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDataset::test_filter_with_nans_on_position[savgol_filter-filter_kwargs1-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-3-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-3-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-5-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-5-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-6-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-6-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-10-valid_poses_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares FAILED tests/test_unit/test_filtering.py::TestFilteringValidDatasetWithNaNs::test_filter_with_nans_on_position_varying_window[savgol_filter-10-valid_bboxes_dataset_with_nan] - numpy.linalg.LinAlgError: SVD did not converge in Linear Least Squares

Since you couldn’t replicate the issue on Windows 11 with a similar setup (Python 3.13.2, NumPy 2.1.3), could this be related to a hardware-specific issue or a subtle environment difference? I can share more hardware details if needed (e.g., System Manufacturer : Dell Inc.
System Model : Vostro 14 3435
System Type : x64-based PC
Processor(s) : 1 Processor(s) Installed.[01]: AMD64 Family 25 Model 80 Stepping 0 AuthenticAMD ~2000 Mhz
BIOS Version : Dell Inc. 1.16.0, 8/16/2024). I’m happy to try any additional debugging steps you suggest.

[niksirbi]

Hmm, it could be hardware-related, but I cannot think of an easy way to pin that down.

Just to be extra sure, try creating a new environment with these exact commands (I'm assuming you have the repo cloned, the main branch checked out, and you are in the repository's root):

conda create -n movement-dev -c conda-forge python=3.13 pytables
conda activate movement-dev
pip install -e '.[dev]'
pytest

[lochhh]

Hi again @Udayscode , I managed to replicate this using an older version of conda (23.x) and updating to the latest conda 25.x resolved it for me. If you're using an older version of conda, could you please try to update it?

[Udayscode]

Ohh Thanks @lochhh for checking and confirming .
Also, just a thought — do you (@lochhh , @niksirbi ) think it might be helpful to explore using Docker for this repo in the future? It could help avoid environment-related issues across different setups. Totally up to you, of course!

[adamltyson]

Docker may be useful for some tasks, but as a toolbox, movement needs to work well in lots of different environments, particularly as a dependency.

[niksirbi]

Thanks for the suggestion @Udayscode, but I agree with Adam here. We want our users to be able to install and use movement across platforms, without resorting to Docker.

[Udayscode]

Got it, Thanks again @adamltyson @niksirbi for your help — as the issue is resolved, so I’ll go ahead and close this issue.

I really appreciate the guidance and look forward to contributing more to the project, either through GSoC if selected, or as a regular contributor.

[niksirbi]

Yeap, closing this issue as it has been resolved by updating the conda version (good to know for future reference).