Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add
river bench benchmarking tool producing succinct output + summary
Here, add a new benchmarking tool to the main River CLI. We had an
existing one, but it hasn't been used or updated in ages, and was
written quite quickly, without too much concern for UX.
This `river bench` is user-runnable, and designed to produce output
that's succinct and easily comprehensible. Every two seconds it produces
a new line of output with the number of jobs worked during that period,
number of jobs inserted during that period, and the rough jobs per
second being complete. When the program is interrupted via `SIGINT`, it
produces one final log line indicating similar information, but
calculated across the entire run period.
$ go run main.go bench --database-url $DATABASE_URL
bench: jobs worked [ 0 ], inserted [ 50000 ], job/sec [ 0.0 ] [0s]
bench: jobs worked [ 22445 ], inserted [ 22000 ], job/sec [ 11222.5 ] [2s]
bench: jobs worked [ 26504 ], inserted [ 28000 ], job/sec [ 13252.0 ] [2s]
bench: jobs worked [ 25919 ], inserted [ 24000 ], job/sec [ 12959.5 ] [2s]
bench: jobs worked [ 27432 ], inserted [ 28000 ], job/sec [ 13716.0 ] [2s]
bench: jobs worked [ 26068 ], inserted [ 26000 ], job/sec [ 13034.0 ] [2s]
bench: jobs worked [ 27068 ], inserted [ 28000 ], job/sec [ 13534.0 ] [2s]
bench: jobs worked [ 27876 ], inserted [ 28000 ], job/sec [ 13938.0 ] [2s]
bench: jobs worked [ 25058 ], inserted [ 24000 ], job/sec [ 12529.0 ] [2s]
^Cbench: total jobs worked [ 214356 ], total jobs inserted [ 264000 ], overall job/sec [ 13026.7 ], running 16.455185125s
It can also run with a total duration, which will be useful if we're
trying to compare runs across branches without having to try and time it
artificially:
$ go run main.go bench --database-url $DATABASE_URL --duration 30s
bench: jobs worked [ 0 ], inserted [ 50000 ], job/sec [ 0.0 ] [0s]
bench: jobs worked [ 23875 ], inserted [ 24000 ], job/sec [ 11937.5 ] [2s]
bench: jobs worked [ 27964 ], inserted [ 28000 ], job/sec [ 13982.0 ] [2s]
bench: jobs worked [ 25694 ], inserted [ 26000 ], job/sec [ 12847.0 ] [2s]
bench: jobs worked [ 26649 ], inserted [ 26000 ], job/sec [ 13324.5 ] [2s]
bench: jobs worked [ 26872 ], inserted [ 28000 ], job/sec [ 13436.0 ] [2s]
bench: jobs worked [ 26519 ], inserted [ 26000 ], job/sec [ 13259.5 ] [2s]
bench: jobs worked [ 25077 ], inserted [ 24000 ], job/sec [ 12538.5 ] [2s]
bench: jobs worked [ 24126 ], inserted [ 26000 ], job/sec [ 12063.0 ] [2s]
bench: jobs worked [ 23936 ], inserted [ 22000 ], job/sec [ 11968.0 ] [2s]
bench: jobs worked [ 26044 ], inserted [ 28000 ], job/sec [ 13022.0 ] [2s]
bench: jobs worked [ 26289 ], inserted [ 26000 ], job/sec [ 13144.5 ] [2s]
bench: jobs worked [ 23058 ], inserted [ 22000 ], job/sec [ 11529.0 ] [2s]
bench: jobs worked [ 23474 ], inserted [ 24000 ], job/sec [ 11737.0 ] [2s]
bench: jobs worked [ 25380 ], inserted [ 26000 ], job/sec [ 12690.0 ] [2s]
bench: total jobs worked [ 375743 ], total jobs inserted [ 426000 ], overall job/sec [ 12524.8 ], running 30.000017167s
Unlike the old benchmarking tool, we switch this one over to do job
accounting using a client subscribe channel instead of measuring it in
the worker. Measuring in the worker doesn't account for the time needed
to block in the job executor waiting for a goroutine to become available
in the completer to complete a job, making it less accurate and possibly
prone to memory overruns as a large backlog of jobs have been accounted
as completed but are actually waiting for a completer slot.
I'm not going to call this feature complete, but I think it's a step in
the right direction, and the hope is that it'll give us a reasonable way
to gutcheck new changes and see whether they cause an obvious regression
or improvement in total performance.- Loading branch information
