Use "|" union syntax in type aliases and base classes too #6612
Conversation
…ript
import sys
for line in sys.stdin:
filename, lineno, message = line.split(":")
lineno = int(lineno) - 1
assert "Use PEP 604 syntax" in message
new = message.split("`")[1]
parts = new.split(" | ")
assert all(part.count("[") == part.count("]") for part in parts)
olds = []
olds.append("Union[" + ", ".join(parts) + "]")
if len(parts) == 2 and parts.count("None") == 1:
[x] = [p for p in parts if p != "None"]
olds.append("Optional[" + x + "]")
lines = open(filename).readlines()
for old in olds:
if lines[lineno].count(old) == 1:
lines[lineno] = lines[lineno].replace(old, new)
open(filename, "w").writelines(lines)
import sys
for line in sys.stdin:
filename, lineno, _, message = line.split(":")
filename = filename.strip()
lineno = int(lineno) - 1
to_remove = message.split('"')[1]
lines = open(filename).readlines()
if lines[lineno].count(to_remove) == 1:
lines[lineno] = lines[lineno].replace(to_remove + ",", "")
lines[lineno] = lines[lineno].replace(to_remove, "")
open(filename, "w").writelines(lines)
Akuli
changed the title
This comment has been minimized.
This comment has been minimized.
|
Not sure it's going to be possible to get the stubtest tests to pass right now, since stubtest is still pinned to MyPy 0.910? No idea about why the other mypy tests are still failing, though. |
|
I just realized that this will have to wait until #6589 is merged. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉 |
|
FWIW, I'd prefer to figure out why the new syntax doesn't work in some files before merging this PR. Maybe the issue affects additional stubs but some errors are just not caught by mypy_primer. Also it seems like this will make some new versions of third-party stubs at least somewhat unusable on mypy releases earlier than 0.920? I think it would be a good idea to give users some time to update before starting to use new features in stubs. One option would be to first update stdlib stubs, since they will be shipped with mypy and thus won't cause issues. |
|
I'm confused about the remaining CI failures, and I can reproduce them locally. If I just run mypy, everything works, but if I run mypy.stubtest it fails with "mypy build errors": |
I really don't think we should wait longer than a mypy release to update our third-party stubs. Frankly, mypy is what is holding us back most. Other type checkers can benefit from improved types and I don't want to wait longer than necessary to improve our stubs. I also don't want to remember what features we can use and what features we can't use, this is what our tests are for. I find it completely acceptable for our users to continue to use older stubs when they are unable to upgrade mypy. |
I'm not asking to wait indefinitely, but I don't think it's unreasonable to wait, say, a few weeks after a release before using new features. This would also give mypy developers and the mypy plugin community time to fix any potential regressions the latest mypy release might have while being able to use the latest stubs. (We are also actively working on speeding up the mypy release process.)
I agree with the sentiment, but it turns out that a lot of mypy users like to use the latest and the greatest, and sometimes there are regressions blocking the use of the latest mypy by some projects, even if we try to carefully avoid them. A little extra time could make the experience better for users that don't pin mypy or stub package versions -- they could continue using the latest stubs, even if they have to pin to a slightly older mypy release temporarily. |
|
I've been trying to narrow some of these errors down, and it's been driving me nuts. If I change a single line in --- a/stdlib/_typeshed/__init__.pyi
+++ b/stdlib/_typeshed/__init__.pyi
@@ -189,7 +189,7 @@ ReadOnlyBuffer = bytes # stable
# for it. Instead we have to list the most common stdlib buffer classes in a Union.
WriteableBuffer = Union[bytearray, memoryview, array.array[Any], mmap.mmap, ctypes._CData] # stable
# Same as _WriteableBuffer, but also includes read-only buffer types (like bytes).
-ReadableBuffer = Union[ReadOnlyBuffer, WriteableBuffer] # stable
+ReadableBuffer = ReadOnlyBuffer | WriteableBuffer # stablethen I get the following output from mypy 0.930 when I run it on the typeshed repo (using Python 3.10):But I'm really struggling to reproduce this error outside the typeshed repo! |
|
Once #4913 is done, maybe |
Hopefully. But I still don't understand why this only appear to error inside the typeshed repo. I tried creating the following directory structure:
`test_union/_typeshed/__init__.pyi` contains the following code:import array
import ctypes
import mmap
from typing import Any, Union
ReadOnlyBuffer = bytes # stable
WriteableBuffer = Union[bytearray, memoryview, array.array[Any], mmap.mmap, ctypes._CData] # stable
ReadableBuffer = ReadOnlyBuffer | WriteableBuffer # stable`test_union/hashlib.pyi` contains the following code:from ._typeshed import ReadableBuffer
class _Hash(object):
def __init__(self, data: ReadableBuffer = ...) -> None: ...It appears as though this exact same code/directory structure causes mypy to error when it's inside the typeshed repo. But when I run Am I doing something obviously wrong here? |
|
If nothing else works, I'll try to just start deleting files from a local copy of typeshed until the errors go away. I have done that with big projects in the past, and even though this technique takes some time, it seems to work every time, no matter how weird the problem is. |
|
Using @Akuli's technique, I have managed to narrow down the bug significantly. The bug is still reproducible in a branch of typeshed I've created over here, in which the vast majority of the standard library has been deleted. |
|
Well, this is my best stab at a bug report: python/mypy#11887 |
|
There's another blocker for this too: |
No description provided.