Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Add pull after checking out master for Benchmark CI #460

Merged
merged 2 commits into from
Sep 21, 2022

Conversation

sergejparity
Copy link
Contributor

It appears that in sometimes master branch is out of sync.
Like here https://gitlab.parity.io/parity/mirrors/wasmi/-/jobs/1868177#L29

$ git checkout master
Previous HEAD position was 2efb5fe add missing import after merge
Switched to branch 'master'
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

This PR fixes that

@Robbepop
Copy link
Member

Thanks @sergejparity for the quick fix! Let's wait for the benchmark results.

@paritytech-cicd-pr
Copy link

paritytech-cicd-pr commented Sep 21, 2022

CRITERION BENCHMARKS

BENCHMARK MASTER PR Diff
compile_and_validate_v1 7.0249 ms 7.0592 ms ⚪ +0.4196%
execute_count_until_v1 3.0771 ms 3.0723 ms ⚪ -0.1115%
execute_factorial_iterative_v1 1.6601 µs 1.6658 µs ⚪ +0.3383%
execute_factorial_recursive_v1 2.1922 µs 2.1982 µs ⚪ -0.1444%
execute_fib_iterative_v1 10.226 ms 10.165 ms ⚪ -0.3122%
execute_fib_recursive_v1 19.578 ms 19.662 ms ⚪ -0.8181%
execute_global_bump_v1 4.0099 ms 3.9348 ms ⚪ -0.9819%
execute_host_calls_v1 35.730 µs 35.259 µs ⚪ -0.6151%
execute_memory_fill_v1 9.3910 ms 9.3927 ms ⚪ +0.0162%
execute_memory_sum_v1 9.0757 ms 9.0985 ms ⚪ +0.3758%
execute_memory_vec_add_v1 16.215 ms 16.202 ms ⚪ +0.1999%
execute_recursive_is_even_v1 3.6142 ms 3.3954 ms 🟢 -5.6863%
execute_recursive_ok_v1 412.75 µs 413.56 µs ⚪ +0.1994%
execute_recursive_scan_v1 516.68 µs 519.08 µs ⚪ +0.5401%
execute_recursive_trap_v1 40.557 µs 40.777 µs ⚪ -0.1006%
execute_regex_redux_v1 3.5115 ms 3.5201 ms ⚪ +0.2508%
execute_rev_complement_v1 3.1027 ms 3.1011 ms ⚪ +0.0856%
execute_tiny_keccak_v1 2.7674 ms 2.7048 ms ⚪ -1.6335%
execute_trunc_f2i_v1 3.9442 ms 3.9420 ms ⚪ -0.1639%
instantiate_v1 76.474 µs 74.501 µs ⚪ +0.6916%

Link to pipeline

Copy link
Member

@Robbepop Robbepop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Since you are already at it ... ^.^ Could you please make use of the more efficient (non-bench) GitLab runners? Or is there a technical/practical reason why not to use those for benchmarking even though they are faster? (Still a lot slower than my 2015 laptop ...)

@Robbepop Robbepop changed the title add pull after checking out master CI: Add pull after checking out master for Benchmark CI Sep 21, 2022
@sergejparity
Copy link
Contributor Author

technical/practical reason why not to use those for benchmarking even though they are faster?

Main reason is to deliver trustworthy results. So we can be sure that difference between tests executed against master and pr is not the result of random factors like some another jobs started on the runner at the same time.

@Robbepop
Copy link
Member

technical/practical reason why not to use those for benchmarking even though they are faster?

Main reason is to deliver trustworthy results. So we can be sure that difference between tests executed against master and pr is not the result of random factors like some another jobs started on the runner at the same time.

sounds reasonable, okay 👍

@Robbepop Robbepop merged commit 5eb4a2d into master Sep 21, 2022
@Robbepop Robbepop deleted the sk-fix-ci-git branch September 21, 2022 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants