Matplotlib Error when using the DREAM fit algorithm

[jamescrake-merani]

Describe the bug
When trying to run a fit with the DREAM algorithm, I'm getting an error coming from Matplotlib. The fit still works but I'm guessing something is going to be wrong with one of the graphs.

11:56:43 - sas.qtgui.Perspectives.Fitting.FittingWidget - ERROR - Traceback (most recent call last):
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/src/sas/qtgui/MainWindow/GuiManager.py", line 1172, in showFitResults
    self.results_panel.onPlotResults(output_data, optimizer=self.perspective().optimizer)
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/src/sas/qtgui/Utilities/ResultPanel.py", line 76, in onPlotResults
    self.correlationView.update(result.uncertainty_state)
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/bumps/gui/uncertainty_view.py", line 60, in update
    self.plot()
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/bumps/gui/uncertainty_view.py", line 55, in plot
    dream_views.plot_corrmatrix(history.draw())
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/bumps/dream/views.py", line 89, in plot_corrmatrix
    c.plot()
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/bumps/dream/corrplot.py", line 65, in plot
    self.ax = _plot(fig, self.hists, self.labels, self.N)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/bumps/dream/corrplot.py", line 99, in _plot
    mapper.set_array(np.zeros(0))
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/matplotlib/image.py", line 689, in set_array
    self.set_data(A)
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/matplotlib/image.py", line 1359, in set_data
    super().set_data(A)
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/matplotlib/image.py", line 675, in set_data
    self._A = self._normalize_image_array(A)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jamescrake-merani/repos/sasview-repos/sasview/.venv/lib/python3.12/site-packages/matplotlib/image.py", line 643, in _normalize_image_array
    raise TypeError(f"Invalid shape {A.shape} for image data")

To Reproduce
Steps to reproduce the behavior:

1. Make sure you have the DREAM fitting algorithm selected in File -> Preferences -> Fit Optimizers
2. Load in some data, and run a fit.
3. You'll see the error once the fit has completed.

SasView version:

• Version: 6.0.1 (on the release branch)

Operating system (please complete the following information):

• OS: Linux (I can reproduce both within, and outside of the Flatpak)

[krzywon]

I just tested this on Windows (bumps v0.9.3, sasview release branch) and I'm seeing the same error. The correlation plots are now broken and none of the DREAM results (correlation, parameter trace, and uncertainty) appear in the Results panel.

[krzywon]

Technical meeting says this needs to be fixed for 6.0.1

[krzywon]

Side note - this is only affecting the correlation plots. If you fit a single parameter (no correlations), the uncertainty and parameter trace succeed.

[krzywon]

This issue was fixed in bumps 2 months ago, on the master (v1) branch with the commit bumps/bumps@d2b16b9.

[jamescrake-merani]

Looking at the commit message, this appears to be an issue introduced to Matplotlib 3.10, and I remember you saying that we can't upgrade yet to Bumps 1.* due to compatibility issues. Should we perhaps make a pin for 3.9 for now, and then we can upgrade later in 6.1.0 when we also fix the compatibility issues with Bumps?

[jamescrake-merani]

I built the Flatpak with an older version of Matplotlib (3.9.4), and it worked on there so I think we do indeed need to pin the version of Matplotlib.

[krzywon]

I will add a matplotlib version pin for the 6.0.1 release as well. I was going to make a new bumps release (v0.9.4), cherry-picking that single commit I linked to here, but that breaks bumps because of the scipy-pytest issue. All of these competing issues are fixed in bumps v1+ and sasview v6.1 (main), but applying these individual changes for the imminent release is a bridge too far.

[krzywon]

I tested locally using mpl v3.9.4 on Windows and the DREAM plots are working again.

[krzywon]

The Windows installer built in #3225 with mpl v3.9.4 is able to generate and display the correlation plots.

[krzywon]

This is fixed in the release_6.0.1 branch, but I am keeping this open until #3165 is merged.