Now you can run separate mypy test modules 🎉

[sobolevn]

One test is failing right now on purpose, so you can take a look on how new namespaces work.
I will fix it in the next commit, I hope the logs will be preserved for some time to see them.

Here's a quick recap of everything in this PR:

• Here's how collect tree looks like: Meta issue: improving test runner #11662 (comment)
• You can now run a single .test file with mypy/test/testcheck.py::TypeCheckSuite::check-basic.test
• Single test will now be mypy/test/testcheck.py::TypeCheckSuite::check-basic.test::testAssignmentAndVarDef

Screenshots:

Refs #11662

[sobolevn]

Live example of a failing test: https://github.com/python/mypy/runs/4424328047?check_suite_focus=true#step:8:179

[sobolevn]

Test numbers:

• This PR: ========== 9641 passed, 370 skipped, 7 xfailed in 1035.09s (0:17:15) ===========
• Previous commit to master (Follow up to #11150, refactors ifs into util function #11663): =========== 9641 passed, 370 skipped, 7 xfailed in 676.28s (0:11:16) ===========

I hope that the time difference is caused by GitHub Action runner fluctuations and not my changes 🙂

Update: previous failing commit took ====== 1 failed, 9640 passed, 370 skipped, 7 xfailed in 832.98s (0:13:52) ======, which hopefully prooves my point about GitHub runners.

I am also going to friendly ping @bluetech as pytest's core dev and mypy contributor. Maybe you would have some time to give this PR a quick review? 🙂

[bluetech]

Nice! Whenever I worked on mypy I did the "comment out the other files" bit which is annoying.

My only suggestion is to not make DataFileCollector a pytest.Class since it's not a class, really:

diff --git a/mypy/test/data.py b/mypy/test/data.py
index 559b5be2a..65fb26735 100644
--- a/mypy/test/data.py
+++ b/mypy/test/data.py
@@ -247,7 +247,7 @@ class DataDrivenTestCase(pytest.Item):
         # TODO: add a better error message for when someone uses skip and xfail at the same time
         elif self.xfail:
             self.add_marker(pytest.mark.xfail)
-        suite = self.parent.obj()
+        suite = self.getparent(DataSuiteCollector).obj()
         suite.setup()
         try:
             suite.run_case(self)
@@ -609,11 +609,12 @@ class DataSuiteCollector(pytest.Class):
             yield DataFileCollector.from_parent(parent=self, name=data_file)
 
 
-class DataFileCollector(pytest.Class):
+class DataFileCollector(pytest.Collector):
     """Represents a single `.test` data driven test file.
 
     More context: https://github.com/python/mypy/issues/11662
     """
+    parent: DataSuiteCollector
 
     @classmethod  # We have to fight with pytest here:
     def from_parent(  # type: ignore[override]
@@ -623,14 +624,13 @@ class DataFileCollector(pytest.Class):
         name: str,
     ) -> 'DataFileCollector':
         instance = super().from_parent(parent, name=name)  # type: ignore[no-untyped-call]
-        instance.obj = parent.obj  # We need to copy `DataSuite` object deeper.
         return instance
 
     def collect(self) -> Iterator['DataDrivenTestCase']:
         yield from split_test_cases(
             parent=self,
-            suite=self.obj,
-            file=os.path.join(self.obj.data_prefix, self.name),
+            suite=self.parent.obj,
+            file=os.path.join(self.parent.obj.data_prefix, self.name),
         )
 
 

[sobolevn]

Thank you! Fixed!

[hauntsaninja]

This is great, thank you all! :-)