Split off SuiteRunner class

[camillobruni]

Move Suite-related code to a separate SuiteRunner class.
This happens in preparation for async and remote suites where it will be cleaner to handle the code paths independently instead of adding more functionality to the BenchmarkRunner class.

[camillobruni]

There is no visible perf change, which is expected since this only moves methods to a separate class.

Before:

browser                                     Safari                    Google Chrome          Firefox
version                                     18.0 (19619.1.26.111.10)  129.0.6668.102         131.0
os                                          macos 14.7 arm64          macos 14.7 arm64       macos 14.7 arm64
cpu                                         Apple M1 Max 10 cores     Apple M1 Max 10 cores  Apple M1 Max 10 cores
runs                                        2                         2                      2
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM  38.00 ± 0.79%             47.80 ± 0.87%          46.50 ± 0.22%
TodoMVC-React-Redux                         30.950 ± 0.16%            32.90 ± 1.4%           35.95 ± 1.3%
TodoMVC-JavaScript-ES5                      29.40 ± 0.34%             38.360 ± 0.10%         38.750 ± 0.13%
NewsSite-Next                               58.70 ± 0.51%             79.01 ± 0.20%          80.5 ± 1.3%
TodoMVC-WebComponents                       13.450 ± 0.37%            15.36 ± 3.2%           19.15 ± 2.9%
NewsSite-Nuxt                               61.9 ± 1.8%               67.55 ± 1.2%           69.7 ± 1.4%
TodoMVC-Lit-Complex-DOM                     14.90 ± 0.67%             18.81 ± 4.2%           19.80 ± 1.0%
Charts-observable-plot                      39.15 ± 2.2%              44.23 ± 0.87%          47.00 ± 0.43%
TodoMVC-Preact-Complex-DOM                  13.350 ± 0.37%            13.98 ± 2.6%           14.100 ± 0.71%
TodoMVC-Angular-Complex-DOM                 32.20 ± 0.31%             28.73 ± 1.8%           36.750 ± 0.14%
TodoMVC-Svelte-Complex-DOM                  11.40 ± 0.88%             11.115 ± 0.13%         12.95 ± 1.9%
Editor-CodeMirror                           23.95 ± 3.1%              22.2 ± 4.8%            22.15 ± 1.1%
TodoMVC-Backbone                            23.45 ± 0.64%             22.16 ± 1.1%           26.85 ± 2.8%
TodoMVC-jQuery                              80.750 ± 0.062%           98.8 ± 3.0%            113.40 ± 0.53%
TodoMVC-React-Complex-DOM                   30.70 ± 1.6%              36.52 ± 0.51%          35.5
Charts-chartjs                              36.25 ± 1.5%              76.5 ± 1.7%            40.2 ± 4.2%
Editor-TipTap                               56.45 ± 0.27%             68.465 ± 0.051%        78.2 ± 1.3%
Perf-Dashboard                              33.65 ± 0.45%             35.7 ± 3.3%            36.65 ± 1.2%
React-Stockcharts-SVG                       68.50 ± 1.5%              74.0 ± 1.6%            79.7 ± 1.6%
TodoMVC-Vue                                 20.60 ± 0.97%             19.79 ± 1.1%           24.00 ± 0.42%
Score                                       32.502 ± 0.20%            28.52 ± 0.60%          27.14 ± 0.69%

After:

browser                                     Safari                    Google Chrome          Firefox
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM  38.05 ± 1.2%              47.7 ± 2.6%            46.35 ± 0.32%
TodoMVC-React-Redux                         30.75 ± 1.1%              33.6 ± 3.1%            36.00 ± 1.9%
TodoMVC-JavaScript-ES5                      29.40 ± 1.7%              38.31 ± 0.48%          38.10 ± 0.26%
NewsSite-Next                               57.700 ± 0.17%            78.04 ± 1.1%           80.2 ± 1.7%
TodoMVC-WebComponents                       13.30 ± 2.3%              15.32 ± 1.2%           20.35 ± 0.74%
NewsSite-Nuxt                               60.55 ± 1.1%              67.10 ± 0.44%          68.7 ± 1.6%
TodoMVC-Lit-Complex-DOM                     14.65 ± 1.0%              18.34 ± 1.7%           20.15 ± 0.74%
Charts-observable-plot                      38.65 ± 1.7%              41.99 ± 0.74%          48.00 ± 1.3%
TodoMVC-Preact-Complex-DOM                  13.35 ± 1.9%              12.75 ± 3.3%           14.150 ± 0.35%
TodoMVC-Angular-Complex-DOM                 32.20 ± 0.62%             29.00 ± 1.9%           36.300 ± 0.28%
TodoMVC-Svelte-Complex-DOM                  11.85 ± 5.5%              13.2 ± 14%             13.05 ± 1.1%
Editor-CodeMirror                           23.55 ± 1.9%              22.00 ± 3.1%           22.50 ± 0.44%
TodoMVC-Backbone                            23.75 ± 0.63%             21.56 ± 1.5%           27.2 ± 5.0%
TodoMVC-jQuery                              80.30 ± 0.62%             96.6 ± 1.7%            113.0 ± 1.7%
TodoMVC-React-Complex-DOM                   29.65 ± 1.2%              34.50 ± 0.67%          35.65 ± 1.3%
Charts-chartjs                              37.05 ± 0.94%             77.805 ± 0.058%        40.3 ± 4.2%
Editor-TipTap                               57.0 ± 2.4%               67.660 ± 0.030%        81.5 ± 4.0%
Perf-Dashboard                              33.00 ± 0.61%             35.00 ± 0.60%          37.30 ± 0.80%
React-Stockcharts-SVG                       68.10 ± 0.59%             73.140 ± 0.055%        78.25 ± 0.32%
TodoMVC-Vue                                 20.65 ± 2.2%              19.040 ± 0.32%         25.20 ± 0.79%
Score                                       32.69 ± 0.38%             28.83 ± 0.41%          26.91 ± 0.63%

[bgrins]

Looks like there's a test failure, FYI

[julienw]

Thanks, this looks good to me, except that loadFrame should IMO be moved as well in this PR. Tell me what you think!

(and of course like Brian mentioned, the tests are red for some reason :D)

[julienw]

thanks for the change, looks good to me

[julienw]

This looks so brittle :D but let's not fix that now

[camillobruni]

lol... yeah, generally not too happy about how many internals we test, but then again I don't have a clear idea on how to fix this.