Description
Is your feature request related to a problem or a nice-to-have?? Please describe.
I use CodeceptJS which uses Mocha under the hood to programmatically run end-to-end tests for browsers. We've been increasing the amount of tests slowly and we're running into issues where tests are taking too long to run on a single machine.
Describe the solution you'd like
There is a concept of test sharding that I propose we add to Mocha. You can read more about this in a few other test runners such as Playwright and Codeception .
So if we had 100 tests, and I split into groups of 4, then tell one machine to execute group 1, it should execute 25 tests. One method of determining which test are part of which groups are simply running every 4th test.
We can then use the matrix strategy in Github Actions to execute each shard on a different machine in parallel.
Describe alternatives you've considered
- We've considered using parallel workers but the machines we're using in CI are very weak but we can spin up a lot of them.
- We've also considered marking each test with a grep such as "Shard 1: Test Name" and then targetting "Shard 1" with the --grep option. This gives us the flexibility to determine which tests are in which shard, but it is a large burden on the developer to need to decide this all the time.
Additional context
I've implemented this in a fork: master...discatech:mocha:test-sharding of Mocha already and would love to get some feedback on whether this is a good change for Mocha itself. I will probably continue to use my forked Mocha until we reach a decision. Thank you so much for considering this!
Activity