-
Notifications
You must be signed in to change notification settings - Fork 41
Fix failing benchmarks #1949
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix failing benchmarks #1949
Conversation
Memory benchmark result| Test Name | %Δ | Master (MB) | PR (MB) | Δ (MB) | Time PR (s) | Time Master (s) |
| -------------------------------------- | ------------ | ------------------ | ------------------ | ------------ | ------------------ | ------------------ |
test_objective_jac_w7x | 4.42 % | 3.813e+03 | 3.981e+03 | 168.46 | 34.84 | 31.04 |
test_proximal_jac_w7x_with_eq_update | -3.58 % | 6.890e+03 | 6.643e+03 | -246.74 | 157.28 | 159.21 |
test_proximal_freeb_jac | -0.31 % | 1.318e+04 | 1.314e+04 | -40.89 | 76.79 | 76.72 |
test_proximal_freeb_jac_blocked | 0.15 % | 7.557e+03 | 7.569e+03 | 11.43 | 67.37 | 67.37 |
test_proximal_freeb_jac_batched | 0.44 % | 7.550e+03 | 7.583e+03 | 32.96 | 68.17 | 67.88 |
test_proximal_jac_ripple | -0.11 % | 7.616e+03 | 7.607e+03 | -8.65 | 68.69 | 68.38 |
test_proximal_jac_ripple_spline | 1.03 % | 3.488e+03 | 3.524e+03 | 35.96 | 70.79 | 70.20 |
test_eq_solve | 1.45 % | 2.057e+03 | 2.087e+03 | 29.79 | 123.53 | 123.43 |For the memory plots, go to the summary of |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1949 +/- ##
=======================================
Coverage 95.74% 95.74%
=======================================
Files 100 100
Lines 27565 27565
=======================================
Hits 26392 26392
Misses 1173 1173
🚀 New features to boost your workflow:
|
| benchmark_name | dt(%) | dt(s) | t_new(s) | t_old(s) |
| -------------------------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
test_build_transform_fft_midres | +1.75 +/- 3.70 | +1.23e-02 +/- 2.60e-02 | 7.15e-01 +/- 2.2e-02 | 7.03e-01 +/- 1.3e-02 |
test_build_transform_fft_highres | -0.09 +/- 1.84 | -8.76e-04 +/- 1.74e-02 | 9.48e-01 +/- 1.4e-02 | 9.49e-01 +/- 9.8e-03 |
test_equilibrium_init_lowres | +2.06 +/- 3.11 | +9.80e-02 +/- 1.48e-01 | 4.85e+00 +/- 1.4e-01 | 4.76e+00 +/- 4.4e-02 |
test_objective_compile_atf | -0.05 +/- 3.59 | -3.30e-03 +/- 2.23e-01 | 6.21e+00 +/- 1.9e-01 | 6.21e+00 +/- 1.1e-01 |
test_objective_compute_atf | -1.84 +/- 3.52 | -4.02e-05 +/- 7.67e-05 | 2.14e-03 +/- 5.1e-05 | 2.18e-03 +/- 5.7e-05 |
test_objective_jac_atf | +1.76 +/- 2.60 | +3.00e-02 +/- 4.42e-02 | 1.73e+00 +/- 2.9e-02 | 1.70e+00 +/- 3.3e-02 |
test_perturb_1 | -0.02 +/- 2.76 | -3.38e-03 +/- 3.98e-01 | 1.44e+01 +/- 2.7e-01 | 1.44e+01 +/- 2.9e-01 |
test_proximal_jac_atf | -0.91 +/- 1.38 | -5.14e-02 +/- 7.78e-02 | 5.57e+00 +/- 4.1e-02 | 5.62e+00 +/- 6.6e-02 |
test_proximal_freeb_compute | -0.40 +/- 2.23 | -6.67e-04 +/- 3.69e-03 | 1.65e-01 +/- 2.7e-03 | 1.66e-01 +/- 2.5e-03 |
test_solve_fixed_iter | -1.32 +/- 2.10 | -3.95e-01 +/- 6.29e-01 | 2.95e+01 +/- 3.1e-01 | 2.99e+01 +/- 5.4e-01 |
test_objective_compute_ripple | +0.01 +/- 1.22 | +3.46e-04 +/- 3.18e-02 | 2.62e+00 +/- 1.1e-02 | 2.62e+00 +/- 3.0e-02 |
test_objective_grad_ripple | -0.34 +/- 1.21 | -1.58e-02 +/- 5.71e-02 | 4.68e+00 +/- 3.2e-02 | 4.70e+00 +/- 4.7e-02 |
test_build_transform_fft_lowres | +0.24 +/- 3.72 | +1.33e-03 +/- 2.07e-02 | 5.58e-01 +/- 1.9e-02 | 5.57e-01 +/- 8.1e-03 |
test_equilibrium_init_medres | +0.39 +/- 1.51 | +1.87e-02 +/- 7.31e-02 | 4.87e+00 +/- 4.2e-02 | 4.85e+00 +/- 6.0e-02 |
test_equilibrium_init_highres | -0.24 +/- 1.33 | -1.30e-02 +/- 7.32e-02 | 5.49e+00 +/- 6.4e-02 | 5.51e+00 +/- 3.5e-02 |
test_objective_compile_dshape_current | -0.48 +/- 1.94 | -1.61e-02 +/- 6.47e-02 | 3.32e+00 +/- 5.5e-02 | 3.34e+00 +/- 3.4e-02 |
test_objective_compute_dshape_current | +6.29 +/- 9.46 | +4.67e-05 +/- 7.02e-05 | 7.89e-04 +/- 6.7e-05 | 7.42e-04 +/- 2.2e-05 |
test_objective_jac_dshape_current | -0.63 +/- 20.53 | -2.08e-04 +/- 6.74e-03 | 3.26e-02 +/- 5.2e-03 | 3.28e-02 +/- 4.3e-03 |
test_perturb_2 | -0.40 +/- 1.27 | -6.73e-02 +/- 2.12e-01 | 1.66e+01 +/- 1.4e-01 | 1.67e+01 +/- 1.6e-01 |
test_proximal_jac_atf_with_eq_update | +0.20 +/- 0.67 | +2.74e-02 +/- 9.15e-02 | 1.37e+01 +/- 5.7e-02 | 1.37e+01 +/- 7.1e-02 |
test_proximal_freeb_jac | +0.28 +/- 2.63 | +1.42e-02 +/- 1.33e-01 | 5.05e+00 +/- 7.0e-02 | 5.04e+00 +/- 1.1e-01 |
test_solve_fixed_iter_compiled | +0.29 +/- 1.08 | +5.01e-02 +/- 1.84e-01 | 1.70e+01 +/- 1.5e-01 | 1.70e+01 +/- 1.0e-01 |
test_LinearConstraintProjection_build | -0.08 +/- 2.64 | -7.02e-03 +/- 2.20e-01 | 8.31e+00 +/- 1.8e-01 | 8.32e+00 +/- 1.2e-01 |
test_objective_compute_ripple_spline | +0.81 +/- 4.84 | +2.36e-03 +/- 1.41e-02 | 2.95e-01 +/- 7.8e-03 | 2.92e-01 +/- 1.2e-02 |
test_objective_grad_ripple_spline | -0.05 +/- 4.30 | -5.96e-04 +/- 4.77e-02 | 1.11e+00 +/- 3.4e-02 | 1.11e+00 +/- 3.3e-02 |Github CI performance can be noisy. When evaluating the benchmarks, developers should take this into account. |
YigitElma
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also solve the other benchmark issue, right?
|
It will solve the failing benchmark on my open prs but I never made an issue for that. |
This PR fixes failing benchmarks as required from changes made in PlasmaControl#1946. PlasmaControl#1834 PR will rename the variable `spline` to `use_bounce1d`, and without this change, both memory and performance benchmarks fail.
This PR fixes failing benchmarks as required from changes made in #1946.
#1834 PR will rename the variable
splinetouse_bounce1d, and without this change, both memory and performance benchmarks fail.