Why does the filter method of SpecifierSet with a single specifier behave differently to Specifier?

[notatallshaw]

Here are two cases where I don't understand why SpecifierSet and Specifier behave differently.

One where only a pre-release is available out of all versions, according to the spec I would expect the pre-release to be available:

>>> list(SpecifierSet('<=2.0.0').filter(["1.0a1"]))
[]
>>> list(Specifier('<=2.0.0').filter(["1.0a1"]))
['1.0a1']

Two, where there is a pre-release and non-prerelease but only the pre-release matches the specifier requirements, again according to the spec I would expect the pre-release to be available:

>>> list(SpecifierSet('>1.1').filter(["1.1", "2.0a1"]))
[]
>>> list(Specifier('>1.1').filter(["1.1", "2.0a1"]))
['2.0a1']

Because pip heavily uses SpecifierSet (I think because the Specifier object is newer) it means pip is often not following the spec as expected.

[Carreau]

I think maybe it's related so I won't open a new issue but, I have the following issue:

pyodide/micropip#162

In [5]: from packaging.requirements import Requirement
   ...: from packaging.version import Version
   ...: req_numpy_with_ver = Requirement('numpy<3,>=1.23')
   ...: req_numpy_no_ver = Requirement('numpy')
   ...:
   ...: print(list(req_numpy_with_ver.specifier.filter([Version('2.2.0.dev')])))
   ...: print(list(req_numpy_with_ver.specifier.filter([Version('2.2.0.dev')], prereleases=True)))
   ...:
   ...: print(list(req_numpy_no_ver.specifier.filter([Version('2.2.0.dev')])))
   ...: print(list(req_numpy_no_ver.specifier.filter([Version('2.2.0.dev')], prereleases=True)))
   ...:
   ...:
[]                       
[<Version('2.2.0.dev0')>]
[<Version('2.2.0.dev0')>]
[<Version('2.2.0.dev0')>]

In [7]: packaging.__version__
Out[7]: '24.1'

I don't understand why "just numpy" assumes 2.2.0.dev0 is ok, but if I pin to numpy<3, then 2.2.0.dev0 is not Ok.

Sorry I'm just starting to dig into packaging; so there might be a smaller minimum example.

[Carreau]

I guess this can be reduced to

In [29]: list(SpecifierSet('').filter(['1.0','2.2.0.dev']))
Out[29]: ['1.0']

In [30]: list(SpecifierSet('').filter(['2.2.0.dev']))
Out[30]: ['2.2.0.dev']

Which seem to be the spec, but this means filer([A, B]) != filter([A]) + filter([B])

Which is quite problematic as it is generally understood that filter is distributif, and SpecifierSet(...).filter is not.

[notatallshaw]

Packaging maintainers, I'm trying to understand if this is a bug or a design choice, if it's a bug I'm happy to work on this, if it's a design choice then I think I need to do some work on pip side to avoid SpecifierSet.filter so that resolution can comply with the spec.

Which seem to be the spec, but this means filer([A, B]) != filter([A]) + filter([B])

Which is quite problematic as it is generally understood that filter is distributif, and SpecifierSet(...).filter is not.'

Yes, due to pre-releases Python packaging version specifiers generally do not conform to "normal" mathematics, some recent discussion here: https://discuss.python.org/t/proposal-intersect-and-disjoint-operations-for-python-version-specifiers/71888. I'm hoping that discussion will lead to a strong spec on this, as currently it's at best confusing