Description
Currently we run pipeline tests for all packages first, followed by system tests for all packages. As the number of packages grows, as we define pipeline and system tests for more packages, and when we add a third type of test (asset loading tests), this serial ordering will slow down CI jobs.
An initial optimization might be to run each type of test for all packages concurrently. So if we have N packages and M test types, there will be M concurrent sub-jobs, one for running each type of test on the N packages.
A further optimization might be to run each type of test (pipeline, system, etc.) for each package concurrently. So if we have N packages and M test types, there will be N x M concurrent sub-jobs, one for running each type of test on each package.