Conversation
|
This turns out to be tricky to test, because the array that is created with the padded values doesn't survive the exporting process. But here's a demonstration: the reason it doesn't show up in the output is that there's later a loop over the channels in the original raw so the excess spurious channels (which are always at the end) are never added to the list of |
cbrnr
left a comment
There was a problem hiding this comment.
Good catch! I didn't know that padding was applied to all axes either, but it makes sense.
|
CI fails look like they might all be due to random number generator issues? Linux pip-pre
Windows pip-pre
Locally I have NumPy 2.1.3 (currently numba forbids anything higher), but regardless I can't find anything in the NumPy changelogs from 2.1.3 onward that seems likely to affect |
|
Often these things crop up due to some change in linalg, like an update to OpenBLAS for example. It changes slightly the results of pseudo-inversion etc. and so some tolerances are too strict. We usually just bump them as long as they aren't way out of range. These all look bumpable to me |
OK, I bumped whatever I could figure out how to bump. I suspect that these two won't actually be fixed by my changes though:
|
|
Okay I think the differences are due to changes in |
|
... I'll merge this one as-is and then open a separate PR for the cobyla stuff. |
|
Thanks @drammock ! |
* upstream/main: (149 commits) FIX make_watershed_bem to handle missing talairach_with_skull.lta courtesy Freesurfer 8 (mne-tools#13172) ENH: Add upsampling for MEG helmet surface (mne-tools#13179) MAINT: Update code credit (mne-tools#13180) BUG: Fix bug with least-squares sphere fit (mne-tools#13178) fix EDF export (mne-tools#13174) fix typo (mne-tools#13171) [pre-commit.ci] pre-commit autoupdate (mne-tools#13164) Fix dev installation guide (mne-tools#13163) expose 'mode' for plotting dipole on brain (mne-tools#13162) turn dipole attrs into properties (mne-tools#13153) remove misformatted (and unused) crossref anchor (mne-tools#13155) doc: point to read_dipole (mne-tools#13149) [pre-commit.ci] pre-commit autoupdate (mne-tools#13152) BUG: Fix bug with not short-circuiting n_jobs=1 (mne-tools#13147) FIX: Missing coordinates.xml in MFF file (mne-tools#13148) FIX: Gracefully handle bad XML files in EGI reader (mne-tools#13145) Fixes for Latest IPython (9.0.1) (mne-tools#13146) Fix intersphinx (mne-tools#13143) BUG: Fix bug with parallel doc build (mne-tools#13140) [pre-commit.ci] pre-commit autoupdate (mne-tools#13141) ...
* upstream/main: (40 commits) fix typo (missing space) that messed up rst rendering (mne-tools#13217) MAINT: Restore VTK dev (mne-tools#13214) [pre-commit.ci] pre-commit autoupdate (mne-tools#13212) BUG: Fix bug with example (mne-tools#13210) MAINT: Fix pip-pre with PyVista (mne-tools#13207) Move FCBG to former partners (mne-tools#13205) ENH: Update related software list (mne-tools#13202) fix sfreq estimation for snirf files (mne-tools#13184) ENH: Use data-based padding instead of "odd" padding when filtering in raw.plot (mne-tools#13183) FIX: Bumps (mne-tools#13198) DOC: fix typo in examples/io/read_impedances.py (mne-tools#13197) [pre-commit.ci] pre-commit autoupdate (mne-tools#13173) FIX make_watershed_bem to handle missing talairach_with_skull.lta courtesy Freesurfer 8 (mne-tools#13172) ENH: Add upsampling for MEG helmet surface (mne-tools#13179) MAINT: Update code credit (mne-tools#13180) BUG: Fix bug with least-squares sphere fit (mne-tools#13178) fix EDF export (mne-tools#13174) fix typo (mne-tools#13171) [pre-commit.ci] pre-commit autoupdate (mne-tools#13164) Fix dev installation guide (mne-tools#13163) ...
The export code for EDF files was calling
np.padincorrectly, causing the channels axis to also get padded with however many extra samples were needed to make equal-sized data blocks along the time axis. Marking as draft because it's the end of the day and I haven't had time to write a test yet.This should shrink almost all exported EDF files, possibly quite significantly.EDIT: the error is self-correcting in the output, because the extra channels are never added to the exported file (there's a for-loop over the channels in the original raw). But it is a bug in that it triggers allocating an unnecessarily large intermediate array, and for me was leading toMemoryErrors when the pad value was big.https://numpy.org/doc/stable/reference/generated/numpy.pad.html