Make choice on case sensitivity for arbitrary equality (`===`)

[notatallshaw]

As part of implementing #943 in #954 it is unclear to me if arbitrary equality should be case sensitive or not.

There are two relevant parts of the spec:

Firstly, under normalization on case sensitivity:

All ascii letters should be interpreted case insensitively within a version and the normal form is lowercase. This allows versions such as 1.1RC1 which would be normalized to 1.1rc1.

Secondly, the first line of arbitrary equality:

Arbitrary equality comparisons are simple string equality operations which do not take into account any of the semantic information such as zero padding or local versions.

This line leaves me with two questions:

• Is case sensitivity (and for that matter other normalization rules) "semantic information"?
• And does "simple string equality operations" mean just ==, or can it mean .lower() == .lower()?

Currently arbitrary equality is case insensitive, but there are no tests to verify this, I will make two PRs, one that simply adds tests, and the other that changes the behavior.

[brettcannon]

Do you or anyone else have any feeling as to whether people are relying on the case-insensitivity?

To me, the specific rule for the === operator trumps the normalization clause. My thinking behind this is this part:

The primary use case for arbitrary equality is to allow for specifying a version which cannot otherwise be represented by this specification.

That suggests that the operator is specifically opting out of any other semantics in the spec regarding how to handle versions.

[notatallshaw]

Apologies, before raising this issue I had planned to dive into the history of where this lower case comparison comes from, as I have no particular feel for if anyone is relying on this.

That suggests that the operator is specifically opting out of any other semantics in the spec regarding how to handle versions.

That was my takeaway from reading the spec, which is why I was finding this a bit of a sticking point when implementing #954. But now I've read through the commit history I've come to the opposite conclusion:

• In 2014, the original implementation in packaging comes from commit 854565a and is case insensitive, which took the version scheme from pkg_resources.parse_version in setuptools
• Case insensitivity was explicitly tested for shortly after: 8fc910b#diff-9ab45dbf39b5b413329c431076d5c7962fc6397a606213310fa96a22b1216109R977
• These tests were lost when LegacySpecifier and LegacyVersion were dropped

Looking in the setuptools commit history, in 2005 parse_version was added in commit in pypa/setuptools@1a7cff6 and was case insensitive:

def parse_version(s):
    ...
    parts = []
    for part in _parse_version_parts(s.lower()):  <-- Causes versions to be parsed case insensitive
        ...

Checking the commit comment:

distutils' StrictVersion, with the flexibility of LooseVersion. It also deals heuristically with common concepts like alpha/beta/candidate/rc and pre/preview, as well as '-' and 'pl' branching schemes.

Checking the CPython commit history I find in 1998 that StrictVersion and LooseVersion were added in commit python/cpython@ee789b9, but these versions only ever accepted lower case versions.

So I would say, if anyone is using === to match versions prior to the modern PEP 440 style version, they will expect this to be case insensitive, but the current specification is not clear about this.

Given the spec on arbitrary equality says:

This operator is special and acts as an escape hatch to allow someone using a tool which implements this specification to still install a legacy version which is otherwise incompatible with this specification.

I think it makes sense to change the language to explicitly call out case insensitivity, and I have raised a DPO post to confirm a change to the spec: https://discuss.python.org/t/clarify-that-arbitrary-equality-is-supposed-to-be-case-insensitive/104885

[notatallshaw]

Case insensitive was the answer:

• https://discuss.python.org/t/clarify-that-arbitrary-equality-is-supposed-to-be-case-insensitive/104885
• Specify arbitrary equality case insensitivity. packaging.python.org#1959