-
Notifications
You must be signed in to change notification settings - Fork 850
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run benchmark from CI once a week #595
Comments
Are you sure about running this on every merge? My concern is the cost :) May be run once a week? |
Agree, will plan to start with once a week. Then we can run the benchmark for longer across more machines and get a more stable result. |
Should we start by exporting to a collector within GitHub actions itself? Just spam the spring-boot smoke test container? I don't know if the numbers are reliable or not, but it seems like an easier start than provisioning cloud. |
For reference, I've written this benchmark with testcontainers before for Zipkin, it's basically awk + zipkin + backend + prometheus. https://github.com/openzipkin/zipkin/blob/master/benchmarks/src/test/java/zipkin2/server/ServerIntegratedBenchmark.java |
cc: @breedx-splk ready made issue 😁 |
We now have https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/workflows/nightly-benchmark-overhead.yml, should we close this issue @trask ? |
Run both
coldstart
andruntime
benchmark from:https://github.com/trask/agent-benchmarking
These spin up a number of Azure App Service VMs and report average and median in order to get a more stable result than you can get on a single cloud machine.
On each VM, the tests alternate enabling/disabling the agent, capturing the overhead that enabling the agent incurs.
The text was updated successfully, but these errors were encountered: