dev-tools replace deprecated typing aliases with those from collections

[micpap25]

Partial on #1653
As suggested by @mhucka isolate the changes from #1677 to smaller subsets of the codebase so that the review process is easier.
This PR removes compatibility with pre-3.10 python versions, but Qualtran isn't intended to target those anymore.

[gemini-code-assist]

Code Review

This pull request updates type annotations across the codebase to use built-in collection types and collections.abc equivalents, replacing the older typing module generics. It also simplifies imports in protocol buffer interface files by removing conditional typing_extensions logic in favor of the standard typing module. I have no feedback to provide as there were no review comments.

[mhucka]

Thanks for this work! It's quite thorough.

I found one place where there's a subtle bug, and a few suggestions for minor tweaks elsewhere, but otherwise this is almost done!

[mhucka]

Just double-checking: @micpap25 how did you (re)generate the .pyi files?

[micpap25]

I'm not sure why it doesn't show the diffs because they are quite straightforward, but I do just manually change them to remove typing and python 3.8 support like any other file.

[mhucka]

Ah. _pb2.pyi files are usually generated by a program, not written manually, and I guess GitHub doesn't make diffs for them by default.

I think this project uses the script in dev_tools/build-protos.sh to generate them, but I've never done it myself, so have to defer to @mpharrigan or @tanujkhattar about the exact procedure that is supposed to be used.

(The hand-edited changes look reasonable to my eyes, but in principle, there's a risk that the other type changes elsewhere will affect what dev_tools/build-protos.sh will produce. I would rather err on the side of caution than introduce some subtle change that might not be picked up by existing unit test cases yet might affect things like the Qualtran web service.)

[mhucka]

This one is incorrect. The typing.Set and the built-in set are not considered equal when used in type annotations, even if they contain the same arguments. Here is a small test program to demonstrate:

from typing import ForwardRef, Set

# Original annotation using typing.Set
original = Set[ForwardRef('BloqCountT')]

# PEP 585 annotation using built-in set
pep585 = set[ForwardRef('BloqCountT')]

print(f"Is {original} == {pep585}?")
print(f"Result: {original == pep585}")

In the typing module, parameterizing a type with a string automatically wraps that string in a ForwardRef object. That meant that typing.Set['BloqCountT'] was identical to typing.Set[typing.ForwardRef('BloqCountT')] in the previous version of the code. However, PEP 585's built-in collections do not auto-convert strings to ForwardRef objects. When it evaluates set['BloqCountT'], Python directly creates a types.GenericAlias containing the literal string 'BloqCountT'.

The fix in this particular case turns out to be easy: get rid of the ForwardRef completely and do:

    if annot['return'] != set['BloqCountT']:  # type: ignore[misc]

[micpap25]

Good to know, because I replace typing.Set with set throughout the other parts of the code; I could go and fix that as well / remove the ForwardRef usage there too? I will make the suggested fix, though.

[micpap25]

Regarding this bug, it seems that making the change you recommended causes an issue in the dev-tools CI about BloqCountT being undefined. It seems fixing this for every type typing.Set is changed to the built-in set will require a lot of TYPE_CHECKING imports?

[mhucka]

Huh. Let me look at it over the weekend.

[mhucka]

Regarding this bug, it seems that making the change you recommended causes an issue in the dev-tools CI about BloqCountT being undefined. It seems fixing this for every type typing.Set is changed to the built-in set will require a lot of TYPE_CHECKING imports?

I don't have an answer to this yet, but I see all the other comments have been resolved (thank you), so what's left is this question and the .pyi files question above.