Switch to using unittest.main and custom runner for unittest design #22388
Description
Currently the steps to run unittest tests are broken down into individually calling:
loader = unittest.TestLoader()
tailor: unittest.TestSuite = loader.loadTestsFromNames(test_ids)
result: UnittestTestResult = runner.run(tailor) # type: ignore
This limits our ability to run tests as similar to how they are run in the command line and to move all heavy lifting over to unittest.
Instead we should use unittest.main(module='__main__', testRunner=None, ...
with a custom testRunner
that is already written to run unittest. Therefore we can more easily just run any args provided and then also gives users more flexibility. This change is similar in design to that of adding Django support with the use of the testRunner
argument.
Looking for community input on useful this change is. Additionally the issues I think it would help with are as follows:
-
support all unittest args: Unittest args doesn't support -t flag #5721
-
(unsure about this one but might fix) load_test protocol allow usage of unittest's load_tests protocol in python unit tests #12063
-
Python unittest runner doesn't respect arguments to pass to unittest runner #20425
-
supporting -s Backslashes doesn't work when specifying the paths with
python.testing.unittestArgs
#11670 -
support -s and -t Incorrect unit test paths in side panel when using both -s and -t parameters #21267