Description
I think it would be nice if cargo test would output timing information about each test it runs. That way users can see where to spend effort if one tries to make a test suite run faster.
It would also be nice if one could influence the order in which test cases are started. Most of my tests finish in an instant, but I have one that does more exhaustive testing. It would be good to have that test started first so it can crunch away in its own thread while the other tests are processed on the other cores. Put differently: I don't want this test to be started last since I'll run out of quick and fast tests to run on my other cores.
Testing... Okay, it seems that Cargo works in the following way:
- Test functions are started in alphabetical order
- The
--test-threadsflag is set to the number of CPUs
The documentation says that
By default, all tests are run in parallel. This can be altered with the
--test-threadsflag or theRUST_TEST_THREADSenvironment variable when running tests (set it to 1).
I first read that as saying "we spawn a thread per test so they all start executing in parallel". That would alleviate my problem above since the overall time for the test run would become the time for the longest running test. However, testing reveals that --test-threads is set equal to the number of CPUs (which is more sensible overall, though twice the number of CPUs could also be a good default in case the tests do even a slight amount of I/O).
If Cargo had timing information from past test runs, then it could use that to start the slowest tests first. Assuming tests are independent, it seems to me that such an order should reduce the overall test time to the time of the longest running test. I think the timing information could be saved inside the target/ directory.
Does this sound like a good idea?