upgrading mypy to 0.790 fails

[keewis]

Continuing from #4567, trying to upgrade mypy to 0.790 fails with:

xarray/core/utils.py:466: error: Value of type variable "_LT" of "sorted" cannot be "K"

See also #4567 (comment) and #4567 (comment)

[mathause]

For context, mypy 0.790 now requires __lt__ if something should be sorted. Thus the following fails (used to work):

from typing import TypeVar
T = TypeVar("T")

def sort(lst: List[T]) -> List[T]):
    return sorted(lst)

which returns the error message above (_LT stands for _LowerThan). The workaround is to define a TypeVar that is Sortable:

from typing import TypeVar, List, Protocol

class Sortable(Protocol):
    def __lt__(self, __other: Any) -> bool: ...

SortableT = TypeVar("SortableT", bound=Sortable)

def sort(lst: List[SortableT]) -> List[SortableT]):
    return sorted(lst)

What we want is a sortable dict key. However, we define dict keys as Hashable (e.g. dims) which is not sortable (does not have __lt__). I see two possibilities:

1. type all keys of a SortedKeysDict as SortableHashable instead of Hashable (that would be a lot)
2. ignore this and add # type: ignore to core/utils.py#L466

But maybe there is a better way?

Here is how 1. could look like (simplified):

import collections
from typing import Any, Dict, Iterator, Mapping, Protocol, TypeVar


class HashableSortable(Protocol):
    def __lt__(self, __other: Any) -> bool:
        ...

    def __hash__(self) -> int:
        ...


HashableSortableK = TypeVar("HashableSortableK", bound=HashableSortable)
V = TypeVar("V")

# use UserDict for brevity
class SortedKeysDict(collections.UserDict[HashableSortableK, V]): # used to be [K, V]
    def __iter__(self) -> Iterator[HashableSortableK]:
        return iter(sorted(self.data))

# now we need to update the typing of all uses of `SortedKeysDict`, e.g. dims

class Dataset:

    _dims: Dict[HashableSortable, int] # used to be [Hashable, int]

    def __init__(self, dims) -> None:

        self._dims = dims

    @property
    def dims(self) -> Mapping[HashableSortable, int]:
        return SortedKeysDict(self._dims)

[max-sixty]

Thanks for the excellent research @mathause

I would vote for #2. I think in time we could move away from SortedKeysDict given that dicts are now ordered. (There is a recent issue on this but I couldn't immediately find it)

[mathause]

Jup, given the effort required for 1. I also vote for 2. I learned a lot about typing/ mypy, though...