Drop Python3.7

[Zeitsperre]

Starting from September 2021, Python3.7 will only be receiving security fixes (PEP 537). This being the case, Python3.7 is gradually being phased out by many scientific software dependencies we rely upon.

To borrow a few links from the rioxarray project (corteva/rioxarray#451):
Pandas dropping Python3.7: pandas-dev/pandas#41678
NumPy dropping Python3.7: https://numpy.org/neps/nep-0029-deprecation_policy.html

Once Xarray drops Python3.7, we should follow suit.

In the meantime, here are the new features that will be allowed for Python3.8.

EDIT: Python3.8 will give us the "Walrus Operator"!

[aulemahal]

All in for following xarray on this. However, I see that PAVICS's jupyterlab currently runs 3.7.12. We'll have to sort that out before! @tlvu, any issues we might need to fix before upgrading?

[tlvu]

All in for following xarray on this. However, I see that PAVICS's jupyterlab currently runs 3.7.12. We'll have to sort that out before! @tlvu, any issues we might need to fix before upgrading?

We are stuck on 3.7 due to one library vcs, see https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/eaa2b01954a748b999b71c3ceeef115e6f714cd2/docker/Dockerfile#L41-L46

That vcs library was added to run https://github.com/ESGF/esgf-compute-api/tree/devel/examples notebooks. @huard I don't remember if I've asked you if we can remove this constraint. If yes then we can move to python 3.9.

[Zeitsperre]

As of today, xarray@main has officially dropped Python3.7 (pydata/xarray#5892). The next release of xclim should be the last one officially supporting Python3.7 as well.

We can likely unofficially support current xarray and Python3.7 for now, but once breaking changes start coming in, we will really be pressured to solve the PAVICS-related issues above.

[Zeitsperre]

@tlvu There's a fix for GDAL in RavenPy master that might help this environment issue. I'll publish a new RavenPy and Raven-WPS to see if that helps.

[tlvu]

@tlvu There's a fix for GDAL in RavenPy master that might help this environment issue. I'll publish a new RavenPy and Raven-WPS to see if that helps.

@Zeitsperre David replied me via Slack we can drop the need to run ESGF notebooks as in my comment #966 (comment) so I think our Jupyter env can go up to 3.9 now, without any fixes needed for Gdal.

[tlvu]

FYI next Jupyter env in PR Ouranosinc/PAVICS-e2e-workflow-tests#95 will have python 3.8 and not 3.9. Apparently xESMF has not been tested on 3.9 yet.

[Zeitsperre]

@tlvu Testing for Python3.9 on xESMF is now being performed regularly: pangeo-data/xESMF#144

[tlvu]

@tlvu Testing for Python3.9 on xESMF is now being performed regularly: pangeo-data/xESMF#144

@Zeitsperre Oh maybe @huard was not aware of this? Or there are more testing that needs to be performed in his mind? He's the one telling me to not use python 3.9 on our Jupyter yet.

[huard]

I am, it's just very, very new!

[tlvu]

Jupyter env with python 3.8 in PR Ouranosinc/PAVICS-e2e-workflow-tests#95 is live now. We can go ahead with deprecating 3.7 in xclim.

When will xarray scheduled to drop 3.8 next time?

[Zeitsperre]

Next release, whenever it's ready to go out. The xclim release today will be the last one supporting Python3.7. Thanks @tlvu !

[tlvu]

Next release, whenever it's ready to go out.

@Zeitsperre next release of xarray will drop python 3.8?! Shouldn't they give some head start for the community to prepare with some clear date, well in advance?

[aulemahal]

https://numpy.org/neps/nep-0029-deprecation_policy.html

I think there was a misunderstanding here. Python 3.7 was droppped by Xarray on thursday. Usually, 3.8 will be dropped in 2023, not too long after numpy (so May I guess).