Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

Microbenchmark build step #1180

Open
1 of 6 tasks
bialesdaniel opened this issue Sep 13, 2020 · 0 comments
Open
1 of 6 tasks

Microbenchmark build step #1180

bialesdaniel opened this issue Sep 13, 2020 · 0 comments
Labels
feature Adds a requested feature performance Performance related issues or changes.

Comments

@bialesdaniel
Copy link
Contributor

bialesdaniel commented Sep 13, 2020

Problem Statement:
Currently, developers are having trouble merging PRs due to microbenchmark failures in the Jenkins builds. The microbenchmark results are not stable so developers will just rerun the build until they get the microbenchmarks to pass. As a result, the microbenchmarks are not frequently used on a per build basis.

Temporary Solution
The running the microbenchmarks have been temporarily disabled in jenkinsfile-nightly.

Long-Term Solution

  • Running microbenchmarks will be removed from the normal build pipeline. The normal build pipeline will still check for compilation issues with the microbenchmarks
  • Re-enable microbenchmarks in the nightly build
  • Change the run_microbench.py so that a violation of failure thresholds does not fail the build. In fact, the failure thresholds become somewhat meaningless
  • Store the results of the microbenchmarks in TimescaleDB
  • Create Grafana visualizations that show benchmark throughput results over time for each benchmark
  • Create slack alerts for microbenchmark performance regression
@bialesdaniel bialesdaniel added performance Performance related issues or changes. feature Adds a requested feature labels Sep 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature Adds a requested feature performance Performance related issues or changes.
Projects
None yet
Development

No branches or pull requests

1 participant