🚀 Feature: Running parallel tests in isolation

[rochdev]

Is your feature request related to a problem or a nice-to-have?? Please describe.

Right now mocha --parallel works by using a specific number of workers and running tests in parallel across these workers. This means a single worker ends up running many test suites in the same process. The main reason we wanted to parallelize tests is not for speed but to isolate tests, which is not the case even in parallel.

Describe the solution you'd like

It would be great to have a mode where the tests could be completely isolated so that a new worker is started for every file, thus avoiding side-effects from previous files.

Describe alternatives you've considered

Usually, the alternative is to write code and tests in a way that avoids side-effects so that every test can be run in the same process without issues. However, for our case, we patch and instrument many things that are irreversible, like enabling async_hooks, patching the module system, patching dependencies as they are imported, adding ESM loaders, etc. This makes it basically impossible to avoid side-effects, so our only option is to ensure test isolation.

Additional context

I wasn't able to find a way to do this with the latest version of Mocha, but please let me know if there is already a way to do this.

[rochdev]

It looks like setting the maximum number of jobs higher than the number of files result in the isolation we're looking for (or at least it looks like it). However, this means spawning dozens if not hundreds of workers simultaneously.

[JoshuaKGoldberg]

Yeah I can see why you'd want this. Newer test runners such as Jest and Vitest do this automatically, and it's a feature gap of Mocha to not be able to at least opt into this. I think this is a useful enough feature to warrant tackling eventually.

Note that per #5027 we're trying to take it slow on any big changes. Even if this ends up being a small opt-in feature, it still touches enough very tricky code that we likely won't get to it for a while. Even reviewing a PR for it would likely be blocked on us getting through many existing issues & PRs.

cc @mochajs/maintenance-crew - thoughts?