Skip to content

Conversation

@mgxd
Copy link
Collaborator

@mgxd mgxd commented Apr 22, 2021

This updates the base images for our circle tests, and bumps the numpy and scipy versions as the one we're currently using is raising an error.

Given that this is the LTS branch, are we comfortable moving to a different minor of the above packages?

@mgxd
Copy link
Collaborator Author

mgxd commented Apr 22, 2021

Okay, I looked into this a little more.

A freeze of 20.2.1 (LTS) python versions
alabaster==0.7.12
annexremote==1.4.3
appdirs==1.4.4
asn1crypto==0.24.0
async-generator==1.10
attrs==20.3.0
Babel==2.8.0
bids-validator==1.5.7
bleach==3.2.1
boto==2.49.0
cached-property==1.5.2
certifi==2020.6.20
cffi==1.11.5
chardet==3.0.4
ci-info==0.2.0
citeproc-py==0.5.1
click==7.1.2
codecov==2.1.10
conda==4.5.11
coverage==5.3
cryptography==2.3.1
cycler==0.10.0
Cython==0.29.21
datalad==0.13.5
decorator==4.4.2
defusedxml==0.6.0
Deprecated==1.2.10
docopt==0.6.2
docutils==0.16
duecredit==0.8.0
entrypoints==0.3
etelemetry==0.2.2
fasteners==0.15
filelock==3.0.12
fmriprep==20.2.1
future==0.18.2
h5py==3.1.0
humanize==3.1.0
idna==2.7
imageio==2.9.0
imagesize==1.2.0
importlib-metadata==2.0.0
indexed-gzip==1.3.2
iniconfig==1.1.1
ipython-genutils==0.2.0
iso8601==0.1.13
isodate==0.6.0
jeepney==0.4.3
Jinja2==2.11.2
joblib==0.17.0
jsmin==2.2.2
jsonschema==3.2.0
jupyter-client==6.1.7
jupyter-core==4.6.3
jupyterlab-pygments==0.1.2
keyring==21.4.0
keyrings.alt==4.0.0
kiwisolver==1.2.0
lockfile==0.12.2
lxml==4.6.1
MarkupSafe==1.1.1
matplotlib==2.2.2
mistune==0.8.4
mkl-fft==1.0.6
mkl-random==1.0.1
monotonic==1.5
msgpack==1.0.0
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nbsphinx==0.8.0
nest-asyncio==1.4.2
networkx==2.5
nibabel==3.2.0
niflow-nipype1-workflows==0.0.4
nilearn==0.6.2
nipype==1.5.1
nitime==0.8.1
nitransforms==20.0.0rc3
niworkflows==1.3.2
num2words==0.5.10
numpy==1.15.4
packaging==20.4
pandas==0.23.4
pandocfilters==1.4.3
patool==1.12
patsy==0.5.1
Pillow==8.0.1
pluggy==0.13.1
pockets==0.9.1
prov==2.0.0
psutil==5.7.3
py==1.9.0
pybids==0.12.3
pycosat==0.6.3
pycparser==2.18
pydot==1.4.1
pydotplus==2.0.2
PyGithub==1.53
Pygments==2.7.2
PyJWT==1.7.1
pyOpenSSL==18.0.0
pyparsing==2.4.7
pyrsistent==0.17.3
PySocks==1.6.8
pytest==6.1.2
pytest-env==0.6.2
python-dateutil==2.8.1
pytz==2020.1
PyWavelets==1.1.1
PyYAML==5.3.1
pyzmq==19.0.2
rdflib==5.0.0
requests==2.19.1
ruamel-yaml==0.15.46
scikit-image==0.17.2
scikit-learn==0.19.1
scipy==1.1.0
sdcflows==1.3.3
seaborn==0.11.0
SecretStorage==3.1.2
sentry-sdk==0.19.2
simplejson==3.17.2
six==1.15.0
sklearn==0.0
smriprep==0.7.0
snowballstemmer==2.0.0
Sphinx==3.3.0
sphinx-argparse==0.2.5
sphinx-rtd-theme==0.5.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-napoleon==0.7
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
SQLAlchemy==1.3.20
svgutils==0.3.1
TBB==0.1
tedana==0.0.9a1
templateflow==0.6.3
testpath==0.4.4
threadpoolctl==2.1.0
tifffile==2020.10.1
toml==0.10.2
tornado==6.0.4
tqdm==4.51.0
traitlets==5.0.5
traits==4.6.0
transforms3d==0.3.1
urllib3==1.23
webencodings==0.5.1
Whoosh==2.7.4
wrapt==1.12.1
zipp==3.4.0

Looking at the docker build step more closely, we do have some incompatibilities...

ERROR: scikit-image 0.18.1 has requirement numpy>=1.16.5, but you'll have numpy 1.15.4 which is incompatible.
ERROR: tedana 0.0.9 has requirement scikit-learn>=0.22, but you'll have scikit-learn 0.19.1 which is incompatible.
ERROR: tedana 0.0.9 has requirement scipy>=1.3.3, but you'll have scipy 1.1.0 which is incompatible.

My inclination is to remove these by pinning to the following major/minors:

  • numpy == 0.16.*
  • scikit-image == 0.18.*
  • scikit-learn == 0.22.*
  • scipy == 1.3.*

WDYT?

@effigies
Copy link
Member

I'm mostly okay relaxing the patch versions, but I'd really rather not keep pushing things. It seems that tedana 0.0.9 is doing most of the pushing, so what about the following?

  • numpy == 1.15.*
  • scikit-image == 0.17.*
  • scikit-learn == 0.19.*
  • scipy == 1.1.*
  • tedana == 0.0.9a1

@mgxd
Copy link
Collaborator Author

mgxd commented Apr 22, 2021

I don't see how that will fix the problem we are currently running into:
https://app.circleci.com/pipelines/github/nipreps/fmriprep/80/workflows/a13b7179-22e9-4b2f-833d-223225d69bf1/jobs/493

Skimming through the sklearn issue, it seems upgrading numpy to 1.16.* is the way.

I'd prefer to get all the incompatibilities out of the way at once, but I'd settle for just numpy to get this (and #2400) in quicker.

@effigies
Copy link
Member

I'm proposing using an older skimage that is known to be compatible with the numpy series we have...

@mgxd mgxd requested a review from effigies April 22, 2021 20:54
@effigies
Copy link
Member

I think we should still set tedana==0.0.9a1 because of ERROR: tedana 0.0.9 has requirement scipy>=1.3.3, but you'll have scipy 1.1.0 which is incompatible..

@tsalo Would be interested in your thoughts here.

Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonetheless, okay with merging as is.

@mgxd
Copy link
Collaborator Author

mgxd commented Apr 22, 2021

Opened #2402 - let's merge to get green again

@mgxd mgxd merged commit 1a8dabb into nipreps:maint/20.2.x Apr 22, 2021
@mgxd mgxd deleted the ci/circle-fix branch April 22, 2021 21:11
HippocampusGirl added a commit to HippocampusGirl/fmriprep that referenced this pull request Feb 11, 2022
Seems to have been pinned to avoid a CI issue, not due to an actual
incompatibility with newer version (nipreps#2401)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants