Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a pull request hook to run benchmark on every PR. #1050

Open
tejal29 opened this issue Feb 13, 2020 · 0 comments
Open

Add a pull request hook to run benchmark on every PR. #1050

tejal29 opened this issue Feb 13, 2020 · 0 comments
Labels
area/internal bugs related to kaniko development workflow

Comments

@tejal29
Copy link
Member

tejal29 commented Feb 13, 2020

It would be great to make sure a PR is not degrading the build time. This will also help us identify PRs which improve the performance especially when snapshotting the whole File System.
We have seen some bugs reported by users like #970.

The idea is to

  1. Use https://developer.github.com/webhooks/ to write a service hosted on GCP which will be called on every new PR or changes to an existing PR.
  2. Run Benchmark on every PR BENCHMARK=true for a set of real dockerfiles (not all)
  3. The service, will grab the results from the current run, compare it with master and see if the PR caused a performance regression.

Changes:

  1. Refactor code to identify if ./benchmark.sh is run inside a Travis Run.
  2. Currently BENCHMARK=true uploads a file to gs://kaniko-project/benchmarks for every Dockerfile. Change this to publish just one file gs://kaniko-project/benchmarks/<pull_req_num>
  3. Finally write the webservice which
    • pull the results for master stored at gs://kaniko-project/benchmarks/master
    • pull the results for the current PR stored at ``gs://kaniko-project/benchmarks/<pull_req_num>`
    • response if diff +ve or -ve time difference.
@tejal29 tejal29 added the area/internal bugs related to kaniko development workflow label Feb 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/internal bugs related to kaniko development workflow
Projects
None yet
Development

No branches or pull requests

1 participant