gtest_filter slow

[anholt]

Describe the bug

Running va-api's 17,415 gtest tests is slow (~25 minutes on Chrome OS's "grunt" boards), so I'm parallelizing it by sharding to multiple threads running multiple tests per gtest process invocation (https://gitlab.freedesktop.org/anholt/deqp-runner/-/merge_requests/24). By specifying the tests to run on each shard as a gtest_filter separated by ':', we're now spending more CPU in FilterMatchesTest() than in the actual tests. On my desktop, perf says I've got 35% of my CPU spent under main() of my unit tests, and 30% under FilterMatchesTest()

If at startup googletest could parse the filter list and toss the non-glob-containing filters into a set, leaving just a vec of glob-containing filters to go through PatternMatchesString(), that should be make the filter cheap for the case of "lots of names of specific tests to run"

Steps to reproduce the bug

git clone from that deqp-runner branch
cargo install --path .

# replace test_va_api here with a handy gtest binary with 17k tests in it
perf record -g gtest-runner run --gtest ~/test_va_api --output <output_dir>
perf report -g

Does the bug persist in the most recent commit?

Tested on fd873f6, hoping that might help, but no luck (I'm not using globs, so the fixes to globbing didn't do much).

What operating system and version are you using?

debian unstable

What compiler and version are you using?

gcc version 11.2.0 (Debian 11.2.0-9)

What build system are you using?

meson "debugoptimized" buildtype (-O2 but with assertions)

[derekmauro]

We actually already support a much simpler sharding protocol. You can just pass the environment variables GTEST_SHARD_INDEX and GTEST_TOTAL_SHARDS.

For example
GTEST_SHARD_INDEX=0 GTEST_TOTAL_SHARDS=2 ./mytest
GTEST_SHARD_INDEX=1 GTEST_TOTAL_SHARDS=2 ./mytest
will run the two shards

Bazel build rules even have a shards flag that uses this protocol.

Does this work for you?

[anholt]

It would take some contortion to rework things that way -- we have to support handling of failing, flaky, and crashing tests while still tracking regressions on the passing set, and my support for those features (as well as include/skip regexes similar to gtest_filter) is based on passing around lists of test names to run.

[anholt]

#3615 did not fix this bug, this bug was about the search for non-pattern-match test names in an array instead of using a better datastructure for those.

[dinord]

Thanks for pointing it out. This was closed by a robot based on "Fixes #3614" in the description of #3615.

[IYP-Programer-Yeah]

@anholt Would appreciate it if you would also take a look at the new merge request and confirm that it is what your desired solution would be. It would be much better if you could also test it out and provide some feedback.

[anholt]

Thanks for the fix! This is an excellent improvement for Mesa CI (which is, incidentally, using google resources :) )

[dinord]

We merged #3746 (thanks @IYP-Programer-Yeah). Is gtest_filter still too slow?

[anholt]

Yeah, it fixed the problem.