Skip to content

Conversation

@flaeppe
Copy link

@flaeppe flaeppe commented May 13, 2024

@github-actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/dtypes/dtypes.py:1173: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:519: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:534: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:575: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:590: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:1145: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/nanops.py:1191: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/missing.py:258: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/io/formats/format.py:1573: error: Item "Sequence[float]" of "ndarray[Any, Any] | Sequence[float]" has no attribute "round"  [union-attr]
+ pandas/io/formats/format.py:1584: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1584: note: Possible overload variants:
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, slice, /) -> Sequence[float]
+ pandas/io/formats/format.py:1586: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1586: note: Possible overload variants:
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, slice, /) -> Sequence[float]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/test/iostream_test.py:820: error: Unused "type: ignore" comment  [unused-ignore]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/test_timefuncs.py:1213: error: Expression is of type "Any", not "TimestampSeries"  [assert-type]
+ tests/test_series.py:1458: error: Expression is of type "Any", not "DataFrame"  [assert-type]
+ tests/test_series.py:1464: error: Expression is of type "Any", not "DataFrame"  [assert-type]
+ tests/test_series.py:3187: error: Expression is of type "Any", not "Series[type[object]]"  [assert-type]
+ tests/test_plotting.py:598: error: Expression is of type "Any", not "Series[Any]"  [assert-type]
+ tests/test_plotting.py:602: error: Expression is of type "Any", not "Axes"  [assert-type]
+ tests/test_indexes.py:360: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:395: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:403: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[float]]"  [assert-type]
+ tests/test_indexes.py:447: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:490: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:506: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:517: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[float]]"  [assert-type]
+ tests/test_indexes.py:555: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]
+ tests/test_indexes.py:610: error: Expression is of type "IntervalIndex[Any]", not "IntervalIndex[Interval[int]]"  [assert-type]

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/tests/config/config_options_tests.py:1679: error: Expression is of type "Any", not "dict[str, dict[Any, Any]]"  [assert-type]

@sobolevn
Copy link
Member

@flaeppe can you please analyze the output?

Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my old PR, wow :)
Thank you!

Someone else has to review it, obviously.

@flaeppe
Copy link
Author

flaeppe commented May 14, 2024

Yes, it's just a rebase of your old PR.

I've only tried to check the pandas changes, but I'm suspecting that now getting the following changes is incorrect:

+ pandas/io/formats/format.py:1573: error: Item "Sequence[float]" of "ndarray[Any, Any] | Sequence[float]" has no attribute "round"  [union-attr]
+ pandas/io/formats/format.py:1584: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1584: note: Possible overload variants:
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1584: note:     def __getitem__(self, slice, /) -> Sequence[float]
+ pandas/io/formats/format.py:1586: error: No overload variant of "__getitem__" of "Sequence" matches argument type "ndarray[Any, dtype[bool_]]"  [call-overload]
+ pandas/io/formats/format.py:1586: note: Possible overload variants:
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, int, /) -> float
+ pandas/io/formats/format.py:1586: note:     def __getitem__(self, slice, /) -> Sequence[float]

It's within the following function and stems from this line: https://github.com/pandas-dev/pandas/blob/283a2dcb2f91db3452a9d2ee299632a109b224f4/pandas/io/formats/format.py#L1561

I haven't been able to find a reproducible case but essentially what is happening here is that there's a narrowing of a union, via a helper similar to something like this:

def helper(val: Any) -> np.ndarray: ...

percentiles: np.ndarray | Sequence[float]
percentiles = helper(percentiles)
reveal_type(percentiles)  # Revealed type is "np.ndarray | Sequence[float]"

There's a couple of @overloads and stuff in there, but that percentiles value is coerced to a np.ndarray type but it seems that previously it was picked up on but now the union is kept instead. Not sure if it's an issue caused by these changes or if the changes revealed a new subsequent issue.

Here's a related issue and comments: #2008 (comment)

@flaeppe
Copy link
Author

flaeppe commented May 14, 2024

Correct me if I'm wrong, but if I'm reading the comments in #2008 and the Always narrow types section in #16472 (comment) correctly I don't think issue mentioned in the above comment #17239 (comment) is caused by the changes here.

@Viicos
Copy link
Contributor

Viicos commented May 17, 2024

Haven't looked into the PR yet, but I'm wondering if this now complies with the recently added chapter in the spec (see python/typing#1667)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Overload ambiguity is ignored in self-annotated methods

3 participants