Adjust SCC setup to enable earlier collections.abc import in typeshed (…

…#14088)

Fixes #11860 (?)

Typeshed is currently unable to import Sequence, MutableSequence, or
ByteString from collections.abc within builtins.pyi. It seems this is
because:

1. In order to analyze `collections.abc`, we first need to analyze
`collections`.

2. Since `collections` is a package containing an `__init__.pyi` file,
the `add_implicit_module_attrs` function will try adding the `__path__`
variable to the symboltable.

3. The `__path__` variable has type `builtins.str`. But str is a
subclass of Sequence, which we have not analyzed yet since we're still
in the middle of analyzing `collections` and `collections.abc`.

This diff tries repairing this by:

1. Adding `_collections_abc` and `collections.abc` to the set of
special-cased core modules we deliberately process early.

2. Modifying `add_implicit_module_attrs` so it does the same trick we do
for the `__doc__` symbol and fall back to using an UnboundType if
`builtins.str` is not defined yet.

To be 100% honest, I'm not really sold on this PR for a few reasons:

- I was able to test these changes manually, but wasn't sure how to
write tests for them.
- We have 3-4 subtly different lists of "core modules" scattered
throughout mypy. For example, see `CORE_BUILTIN_MODULES` in
mypy/build.py or try grepping for the string `"typing"` in the mypy dir.
Arguably, we should defer landing this PR until we've had a chance to
consolidate these lists and confirm there are no additional places where
we need to special-case `_collections_abc`, `collections`, and
`collections.abc`.
- PEP 585 attempted to declare that we should one day remove entries
like Sequence from `typing` module, but this realistically doesn't seem
ever achievable given that (a) it would break backwards compat and (b)
there doesn't seem to be any incentives for users to proactively switch.
In that case, is there any pressing reason to change typeshed?

Regardless, this is a crash and my goal atm is to de-crash mypy, so I'm
throwing this over the wall.

mypy/semanal.py

@@ -615,9 +615,12 @@ def refresh_top_level(self, file_node: MypyFile) -> None:
 
     def add_implicit_module_attrs(self, file_node: MypyFile) -> None:
         """Manually add implicit definitions of module '__name__' etc."""
+        str_type: Type | None = self.named_type_or_none("builtins.str")
+        if str_type is None:
+            str_type = UnboundType("builtins.str")
         for name, t in implicit_module_attrs.items():
             if name == "__doc__":
-                typ: Type = UnboundType("__builtins__.str")
+                typ: Type = str_type
             elif name == "__path__":
                 if not file_node.is_package_init_file():
                     continue
@@ -630,7 +633,7 @@ def add_implicit_module_attrs(self, file_node: MypyFile) -> None:
                 if not isinstance(node, TypeInfo):
                     self.defer(node)
                     return
-                typ = Instance(node, [self.str_type()])
+                typ = Instance(node, [str_type])
             elif name == "__annotations__":
                 sym = self.lookup_qualified("__builtins__.dict", Context(), suppress_errors=True)
                 if not sym:
@@ -639,7 +642,7 @@ def add_implicit_module_attrs(self, file_node: MypyFile) -> None:
                 if not isinstance(node, TypeInfo):
                     self.defer(node)
                     return
-                typ = Instance(node, [self.str_type(), AnyType(TypeOfAny.special_form)])
+                typ = Instance(node, [str_type, AnyType(TypeOfAny.special_form)])
             else:
                 assert t is not None, f"type should be specified for {name}"
                 typ = UnboundType(t)

mypy/semanal_main.py

@@ -66,7 +66,14 @@
 
 # Number of passes over core modules before going on to the rest of the builtin SCC.
 CORE_WARMUP: Final = 2
-core_modules: Final = ["typing", "builtins", "abc", "collections"]
+core_modules: Final = [
+    "typing",
+    "_collections_abc",
+    "builtins",
+    "abc",
+    "collections",
+    "collections.abc",
+]
 
 
 def semantic_analysis_for_scc(graph: Graph, scc: list[str], errors: Errors) -> None: