Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable preview mode during benchmarks #7208

Merged
merged 1 commit into from
Sep 11, 2023
Merged

Enable preview mode during benchmarks #7208

merged 1 commit into from
Sep 11, 2023

Conversation

zanieb
Copy link
Member

@zanieb zanieb commented Sep 6, 2023

Split out of #7195 so benchmark changes from enabling additional rules can be reviewed separately.

@zanieb zanieb marked this pull request as draft September 6, 2023 19:07
@zanieb zanieb force-pushed the zanie/preview-benchmark branch from dad2878 to b72f962 Compare September 6, 2023 19:12
@codspeed-hq
Copy link

codspeed-hq bot commented Sep 6, 2023

CodSpeed Performance Report

Merging #7208 will degrade performances by 5.6%

Comparing zanie/preview-benchmark (32e5cab) with main (6566d00)

Summary

❌ 5 (👁 5) regressions
✅ 20 untouched benchmarks

Benchmarks breakdown

Benchmark main zanie/preview-benchmark Change
👁 linter/all-rules[numpy/globals.py] 4 ms 4.1 ms -2.58%
👁 linter/all-rules[numpy/ctypeslib.py] 33.2 ms 34.5 ms -3.93%
👁 linter/all-rules[unicode/pypinyin.py] 15.4 ms 15.7 ms -2.29%
👁 linter/all-rules[pydantic/types.py] 69.1 ms 72.4 ms -4.59%
👁 linter/all-rules[large/dataset.py] 154.5 ms 163.7 ms -5.6%

@zanieb zanieb force-pushed the zanie/rule-preview branch from 8490cf3 to 16f830f Compare September 6, 2023 21:10
@zanieb zanieb force-pushed the zanie/preview-benchmark branch 5 times, most recently from 74e7eb1 to 6116d0e Compare September 6, 2023 21:43
@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

@zanieb
Copy link
Member Author

zanieb commented Sep 6, 2023

Another option is to add new benchmarks for preview mode. Probably a good way to separate performance changes in preview and stable?

@zanieb zanieb force-pushed the zanie/rule-preview branch from 8421200 to 4ea6f17 Compare September 8, 2023 21:59
@zanieb zanieb force-pushed the zanie/preview-benchmark branch from 6116d0e to caa3a4e Compare September 8, 2023 22:01
@MichaReiser
Copy link
Member

Another option is to add new benchmarks for preview mode. Probably a good way to separate performance changes in preview and stable?

Sounds good to me. Separating preview and stable benchmarks can help narrow down a regression or catch unintentional performance regressions when supposedly only changing preview code. It could then be taken off lightly as: this is preview only when in fact, we just regressed stable.

@zanieb zanieb force-pushed the zanie/preview-benchmark branch from caa3a4e to f060bbd Compare September 11, 2023 13:54
Base automatically changed from zanie/rule-preview to main September 11, 2023 17:28
@zanieb zanieb force-pushed the zanie/preview-benchmark branch from f060bbd to 32e5cab Compare September 11, 2023 17:29
@zanieb zanieb marked this pull request as ready for review September 11, 2023 17:30
@zanieb zanieb added the preview Related to preview mode features label Sep 11, 2023
@zanieb
Copy link
Member Author

zanieb commented Sep 11, 2023

I think I'd like to include this as a general improvement and we can look for a contributor for splitting them into separate benchmarks?

@zanieb zanieb merged commit 24b848a into main Sep 11, 2023
@zanieb zanieb deleted the zanie/preview-benchmark branch September 11, 2023 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants