Consider revising our minimum dependency version policy

[shoyer]

Our current policy is that xarray supports "the minor version (X.Y) initially published no more than N months ago" where N is:

• Python: 42 months (NEP 29)
• numpy: 24 months (NEP 29)
• pandas: 12 months
• scipy: 12 months
• sparse, pint and other libraries that rely on NEP-18 for integration: very latest available versions only,
• all other libraries: 6 months

I think this policy is too aggressive, particularly for pandas, SciPy and other libraries. Some of these projects can go 6+ months between minor releases. For example, version 2.3 of zarr is currently more than 6 months old. So if zarr released 2.4 today and xarray issued a new release tomorrow, and then our policy would dictate that we could ask users to upgrade to the new version.

In #4178, I misinterpreted our policy as supporting "the most recent minor version (X.Y) initially published more than N months ago". This version makes a bit more sense to me: users only need to upgrade dependencies at least every N months to use the latest xarray release.

I understand that NEP-29 chose its language intentionally, so that distributors know ahead of time when they can drop support for a Python or NumPy version. But this seems like a (very) poor fit for projects without regular releases. At the very least we should adjust the specific time windows.

I'll see if I can gain some understanding of the motivation for this particular language over on the NumPy tracker...

[max-sixty]

Agree, I was holding the same (incorrect) interpretation. Thanks for the excellent outline on the numpy tracker.

[dcherian]

"the most recent minor version (X.Y) initially published more than N months ago". This version makes a bit more sense to me: users only need to upgrade dependencies at least every N months to use the latest xarray release.

I agree.

[keewis]

I think we talked about this in a meeting a few weeks ago, but I'm not sure I remember correctly: was the only issue that nobody picked this up? From the comments above it seems we all agree on changing the policy to "support the latest minor version (X.Y) available N months ago" – officially

the most recent minor version (X.Y) initially published more than N months ago

[keewis]

actually, this version of the policy won't work: if we require the most recent python version from 42 months ago (Aug 2017) we get python 3.6, which we can't maintain because most of our dependencies won't support it anymore by the time we would be allowed to drop it. So I think we will have to find a way to work around that, either by changing the support window of python or follow NEP29 and require at least M supported versions, or something completely different.

One edge case of the rules above would be if a package has no release for more than N months and then multiple releases over the course of a few weeks.

[keewis]

I wonder if finding a policy for all libraries is possible at all, and if maybe we should have a separate policy for python and numpy, which have a regular release cycle (every N months)?

That would mean: exactly 42 months support for python (together with packaging tools like pip and setuptools, should we depend on that) and exactly 18 months for numpy (as in "N months after the initial release we will stop supporting it"), but at least 12 or 6 months support for libraries like pandas, dask, scipy, and zarr which don't have frequent or regular releases ("the most recent minor version N months ago")

[shoyer]

If you look at the linked NumPy issue, I was also suggesting adjusting the number of months to compensate:

• 24 months old Python
• 18 months old NumPy

Those timeframes basically match up with those in the NumPy NEP.

[keewis]

ah, I seem to have missed that. Now that python has a yearly release cycle, I would extend the support window for a few months, though: with 24 months we could drop 3.8 as soon as 3.10 comes out (not sure if that's an issue in practice: projects like numba seem to take 4-5 months until they release a compatible version).