Mypy errors with the last version of _typed_ops.pyi

[benbovy]

What happened:

Since #5569 I get a lot of mypy errors from _typed_ops.pyi (see below). What's weird is that it is not happening in all cases:

$ mypy                          # ok
$ mypy .                        # errors
$ pre-commit run --all-files    # ok
$ pre-commit run                # errors
$ git commit                    # (via pre-commit hooks) errors

I also tried pre-commit clean with no luck. EDIT: I also tried on a freshly cloned xarray repository.

@max-sixty @Illviljan Any idea on what's happening?

What you expected to happen:

No mypy error in all cases.

Anything else we need to know?:

xarray/core/_typed_ops.pyi:32: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:33: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:34: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:35: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:36: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:37: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:38: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:39: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:40: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:41: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:42: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:43: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:44: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:45: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:46: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:47: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:48: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:49: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:50: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:51: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:52: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:53: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:54: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:55: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:56: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:57: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:60: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:61: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:62: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:63: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:64: error: The erased type of self "xarray.core.dataset.Dataset" is not a supertype of its class "xarray.core._typed_ops.DatasetOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:65: error: The erased type of self "xarray.core.dataset.Dataset" is not a supertype of its class "xarray.core._typed_ops.DatasetOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:66: error: The erased type of self "xarray.core.dataset.Dataset" is not a supertype of its class "xarray.core._typed_ops.DatasetOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:67: error: The erased type of self "xarray.core.dataset.Dataset" is not a supertype of its class "xarray.core._typed_ops.DatasetOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:77: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:83: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:89: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:95: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:101: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:107: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:113: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:119: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:125: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:131: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:137: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:143: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:149: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:155: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:161: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:167: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:173: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:179: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:185: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:191: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:197: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:203: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:209: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:215: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:221: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:227: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:230: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:231: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:232: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:233: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:234: error: The erased type of self "xarray.core.dataarray.DataArray" is not a supertype of its class "xarray.core._typed_ops.DataArrayOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:235: error: The erased type of self "xarray.core.dataarray.DataArray" is not a supertype of its class "xarray.core._typed_ops.DataArrayOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:236: error: The erased type of self "xarray.core.dataarray.DataArray" is not a supertype of its class "xarray.core._typed_ops.DataArrayOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:237: error: The erased type of self "xarray.core.dataarray.DataArray" is not a supertype of its class "xarray.core._typed_ops.DataArrayOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:247: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:253: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:259: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:265: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:271: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:277: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:283: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:289: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:295: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:301: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:307: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:313: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:319: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:325: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:331: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:337: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:343: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:349: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:355: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:361: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:367: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:373: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:379: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:385: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:391: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:397: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:400: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:401: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:402: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:403: error: Self argument missing for a non-static method (or an invalid type for self)  [misc]
xarray/core/_typed_ops.pyi:404: error: The erased type of self "xarray.core.variable.Variable" is not a supertype of its class "xarray.core._typed_ops.VariableOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:405: error: The erased type of self "xarray.core.variable.Variable" is not a supertype of its class "xarray.core._typed_ops.VariableOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:406: error: The erased type of self "xarray.core.variable.Variable" is not a supertype of its class "xarray.core._typed_ops.VariableOpsMixin"  [misc]
xarray/core/_typed_ops.pyi:407: error: The erased type of self "xarray.core.variable.Variable" is not a supertype of its class "xarray.core._typed_ops.VariableOpsMixin"  [misc]

Environment:

mypy 0.910
python 3.9.6 (also tested with 3.8)

[Illviljan]

Before T_Dataset etc. was:

xarray/xarray/core/_typed_ops.pyi

Lines 20 to 22 in d58a511

	T_Dataset = TypeVar("T_Dataset", bound="DatasetOpsMixin")
	T_DataArray = TypeVar("T_DataArray", bound="DataArrayOpsMixin")
	T_Variable = TypeVar("T_Variable", bound="VariableOpsMixin")

Now it is:

xarray/xarray/core/types.py

Lines 20 to 22 in 6b59d9a

	T_Dataset = TypeVar("T_Dataset", bound="Dataset")
	T_DataArray = TypeVar("T_DataArray", bound="DataArray")
	T_Variable = TypeVar("T_Variable", bound="Variable")

This seems fixable by adding back

xarray/xarray/core/_typed_ops.pyi

Lines 19 to 22 in d58a511

	# DatasetOpsMixin etc. are parent classes of Dataset etc.
	T_Dataset = TypeVar("T_Dataset", bound="DatasetOpsMixin")
	T_DataArray = TypeVar("T_DataArray", bound="DataArrayOpsMixin")
	T_Variable = TypeVar("T_Variable", bound="VariableOpsMixin")

What I wonder about is why we called it T_Dataset instead of T_DatasetOpsMixin?
Is it a trick to make it look good in docs or?

[max-sixty]

Edit: @Illviljan and I commented at the same time, he seems to have found the error!

---

Great spot @benbovy . I think what's happening is that the standard run only checks .py files, rather than .pyi files. So mypy . collects it, but pre-commit and mypy doesn't.

ref https://github.com/pre-commit/mirrors-mypy/issues/33 for the pre-commit issue.

On the errors themselves — @rhkleijn I think I have upset your magnum opus despite the best of intentions. If you're up for having a look, that would be great. Otherwise we could try and revert my original PR, though I worry this sort of refactor is difficult to revert.

[max-sixty]

What I wonder about is why we called it T_Dataset instead of T_DatasetOpsMixin?
Is it a trick to make it look good in docs or?

The goal was for the type that's used throughout Xarray to be called T_Dataset, but not sure why that means we have to change the definition — I don't think there's a good reason.

Should we change that back, and change the mypy file files definition to .py[i]?

[Illviljan]

What I wonder about is why we called it T_Dataset instead of T_DatasetOpsMixin?
Is it a trick to make it look good in docs or?

The goal was for the type that's used throughout Xarray to be called T_Dataset, but not sure why that means we have to change the definition — I don't think there's a good reason.

Should we change that back, and change the mypy file files definition to .py[i]?

Agreed and maybe add a comment/TODO why it's called T_Dataset instead of T_DatasetOpsMixin if there is any reason.

[benbovy]

Thank you very much @max-sixty and @Illviljan for looking into this and for the PR!