Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(minor): Make CI fail with regression for crashes or other run failures #211

Merged
merged 2 commits into from
Dec 20, 2023

Conversation

hassila
Copy link
Contributor

@hassila hassila commented Dec 20, 2023

Description

After the changes in #166 we no longer fail the run of the rest of the benchmarks if a single one fails, which is reasonable during normal iterative work. This is not ok for CI baseline checks though, so this PR restores failures when baseline check is performed.

How Has This Been Tested?

Manually tested.

Minimal checklist:

  • I have performed a self-review of my own code
  • I have added DocC code-level documentation for any public interfaces exported by the package
  • I have added unit and/or integration tests that prove my fix is effective or that my feature works

@hassila hassila requested a review from freef4ll December 20, 2023 13:45
Copy link
Contributor

github-actions bot commented Dec 20, 2023

Copy link

codecov bot commented Dec 20, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5fb6d7e) 70.40% compared to head (b8b4160) 70.40%.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #211   +/-   ##
=======================================
  Coverage   70.40%   70.40%           
=======================================
  Files          29       29           
  Lines        3909     3909           
=======================================
  Hits         2752     2752           
  Misses       1157     1157           

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5fb6d7e...b8b4160. Read the comment docs.

@hassila hassila changed the title feat(minor) Make CI fail with regression for crashes or other run failures feat(minor): Make CI fail with regression for crashes or other run failures Dec 20, 2023
@hassila hassila merged commit 645e475 into main Dec 20, 2023
13 of 16 checks passed
@hassila hassila deleted the fail-ci-check-on-benchmark-failures branch December 20, 2023 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants