Skip to content

[BUG] Minimum benchmark time with multiple threads #2117

@hahnjo

Description

@hahnjo

Describe the bug
#1836 fixed #1834 and now the reported output makes more sense. However, the minimum benchmark time doesn't seem to be respected.

System
Arch Linux, GCC 15.2.1, benchmark 1.9.5-1

To reproduce
Consider

#include <benchmark/benchmark.h>

static void BM_MultiThreaded(benchmark::State& state) {
  int a = 0;

  for (auto _ : state) {
    for (int i = 0; i < 1000; i++) {
      benchmark::DoNotOptimize(a++);
    }
  }
}
BENCHMARK(BM_MultiThreaded)->ThreadRange(1, 8);

BENCHMARK_MAIN();

The output is somewhat expected:

---------------------------------------------------------------------
Benchmark                           Time             CPU   Iterations
---------------------------------------------------------------------
BM_MultiThreaded/threads:1        275 ns          274 ns      2585850
BM_MultiThreaded/threads:2        273 ns          272 ns      2259184
BM_MultiThreaded/threads:4        279 ns          278 ns      2449204
BM_MultiThreaded/threads:8        308 ns          307 ns      2148808

However, notice that the number of iterations stays constant - it should increase as more threads are used to run the iterations. For example:

 $ /usr/bin/time -v ./benchmark-multithreaded --benchmark_min_time=1s --benchmark_filter=threads:8
[...]
	User time (seconds): 1.72
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.25

With previous versions of benchmark, this could be fixed with UseRealTime() but that doesn't seem to work anymore since (probably) #1836.

Expected behavior
Each benchmark should spend the minimum time (per thread).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions