🧱 static integration testing

[jorenham]

Currently limited and numpy and only uses mypy. I plan to add (based)pyright in a follow-up.

Closes ##25 I guess

[jorenham]

So numpy isn't compatible with our HasArrayNamespace protocol, because numpy uses string literals, instead of str in the input position (which is contravariant).

[nstarman]

Overall looks great. Couple Q's.

[nstarman]

Don't we want to add other hooks like a toml linter?

[jorenham]

yea, I had dprint in mind for that

[jorenham]

lefthook fixes/reformats the files, but i couldn't find a flag to run it so that it'll error if it needs fixing/reformatting

[nstarman]

Can we detect a git diff?

lefthook run --no-tty pre-commit
git diff --exit-code

[jorenham]

Sounds doable. But the advantage of using "bare" ruff here, is that it plays well with github, and annotates lines with errors and stuff

[nstarman]

The GH compat is nice, but running the full lefthook in CI is also important. As we add more hooks this will become necessary.

[jorenham]

Yea good point. But it should then either fail if there's a diff, or commit the changes if there are any. Both options aren't trivial, and I can't immediately picture the best approach for it. If you have a proposal I'd be happy to put it in.

[jorenham]

commiting it here is also fine by me

[lucascolley]

it should then either fail if there's a diff

Yes, I want to figure out how to make this happen

[nstarman]

What string literals does it expect? Limited to the list of compatible versions?

[jorenham]

What string literals does it expect? Limited to the list of compatible versions?

In numpy 2.3.1 it [looks like this]:(https://github.com/numpy/numpy/blob/4d833e5df760c382f24ee3eb643dc20c3da4a5a1/numpy/__init__.pyi#L1684)

_ArrayAPIVersion: TypeAlias = L["2021.12", "2022.12", "2023.12", "2024.12"]

# --snip--

def __array_namespace__(self, /, *, api_version: _ArrayAPIVersion | None = None) -> ModuleType: ...

in 2.0.2 it looks like this:

def __array_namespace__(self, *, api_version: str | None = ...) -> Any: ...

and <2.0.0 there was no __array_namespace__

[jorenham]

Huh, why is mypy using the stubs from numpy 2.3.1 when numpy 1.25.0 is installed?
https://github.com/data-apis/array-api-typing/actions/runs/16028175155/job/45221469410?pr=35

[jorenham]

So mypy apparently completely ignores build isolation...

The workaround is to use uv pip install, instead of uv run --with, which layers the dependencies. It's disgustingly ugly, but at least the tests are failing for the right reasons now...

[jorenham]

Ok, now the question remains: What to do with numpy<2? Ignore it (for now), script our way around it by using array-api-compat if numpy<2, or split it up into two separate integration tests?

[lucascolley]

My feeling is that we are going to end up using array-api-compat anyway, no? I guess not for older versions of the standard, but as the standard is developed.

[jorenham]

My feeling is that we are going to end up using array-api-compat anyway, no? I guess not for older versions of the standard, but as the standard is developed.

for the purposes of integration testing it might be a good idea to rely on it as little as we can

[jorenham]

My feeling is that we are going to end up using array-api-compat anyway, no? I guess not for older versions of the standard, but as the standard is developed.

Testing both is also an option 🤔

[lucascolley]

may as well ignore it for now if we can pass without it, and see what happens down the line

[jorenham]

Nevermind, I managed to get this working on numpy<2 by using np.array_api there. It required a little bit of black voodoo magic in the gha workflow, and it's not ideal in terms of IDE support, but it's the cleanest way I could think of.