ClickHouse Cloud: collect the concurrent QPS

[alexey-milovidov]

Run the same concurrent QPS test as for other systems (10 connections, 600 s, the seeded query order of bench_concurrent_qps in lib/benchmark-common.sh) on ClickHouse Cloud services after the main run, and record concurrent_qps and concurrent_error_ratio in the results.

• concurrent-qps.py (new): the test itself, as threads in one Python process (standard library only), each connection keeping its HTTPS connection open. The shared shell version starts several processes and a new TLS connection per query; with all services tested at once from one runner, the runner became the bottleneck (fast services stuck at about 3 QPS regardless of replicas). Same query order and same counting as the shell version; wait_end_of_query=1 makes a query that fails partway through return an HTTP error. There is no watchdog, since a Cloud service can't be restarted from here.
• benchmark.sh: runs it after run.sh. The data size stays on the last line.
• collect-results.sh: puts the two values into the JSON when they are present and not null.

Tested concurrent-qps.py against a local HTTPS server (about 870 QPS from 10 kept-open connections at under half a core; a failing query and a wrong password are counted as errors), and checked that the permutations match the shared ones. The first Cloud run used the shell client; a second run with the new client is in progress. Each service stays up about 10 minutes longer.

🤖 Generated with Claude Code

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ alexey-milovidov
❌ github

---

github seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

Preview of the benchmark page with these results · Comparison with the previous results