-
-
Notifications
You must be signed in to change notification settings - Fork 8.6k
[rb] Improve test filters #16813
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: trunk
Are you sure you want to change the base?
[rb] Improve test filters #16813
Conversation
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
|||||||||||||||||||||||||||||||
PR Code Suggestions ✨Explore these optional code suggestions:
|
||||||||||||||||
311fdeb to
eb0f5e2
Compare
eb0f5e2 to
994a074
Compare
994a074 to
040b94b
Compare
CI Feedback 🧐A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
|
User description
🔗 Related Issues
Ruby implementation of #16809
We are mostly relying on RBE to tell us if there is a problem, this is supposed to be extra information that strikes a good balance between information and execution time.
💥 What does this PR do?
ci.ymlkicks off on every commit. It runs the new and improvedcheck-bazel-targets.shInstead of just checking to see if there are any
rbtargets and running the whole ci-ruby workflow, the unique set of applicable test targets are passed from the script toci-ruby.ymland browser tests are only run on thoseAdditionally, docs/steep/unit tests are only run if the PR includes "[rb]" or is executed manually (passed into
ci-ruby.ymlfromci.ymlwith arun-full-suiteargument.Even if tagged
[rb], if only a subset of ruby tests are matched, only those will be runRemoved build instead of gating all tests on it. Most of the build is executed by the applicable tests as necessary. If there is something weird that breaks building, it'll get detected in nightly run
It's now 2 jobs instead of 6, but effective changes are:
Adds chrome-beta-remote/windowsAdds firefox-beta-remote/windowsRemoves Edge Remote WindowsUpdate: fixed so it run edge remote like previously and doesn't run chrome/firefox remote
🔧 Implementation Notes
I'm picking betas instead of production browsers as a way to get additional information earlier. Failures in new browser versions will still be managed in a PR updating them, not in trunk.
💡 Additional Considerations
This probably doesn't decrease the total amount of Ruby testing by enough, so we probably need to add more filters, but I think this is a good place to start.
PR Type
Enhancement, Tests
Description
Implement selective Ruby test filtering based on changed files
Limit test universe to binding roots for faster CI execution
Remove build job and consolidate to two browser test jobs
Filter test targets by
[rb]tag and file changesDiagram Walkthrough
File Walkthrough
check-bazel-targets.sh
Restrict target universe and improve query logicscripts/github-actions/check-bazel-targets.sh
set -euo pipefailci-ruby.yml
Refactor workflow with selective test filtering.github/workflows/ci-ruby.yml
targetsandrun-full-suiteparametersfilter-targetsjob to filter Ruby-specific targetsonly
run-full-suiteinputci.yml
Wire target filtering to Ruby workflow inputs.github/workflows/ci.yml
run-full-suitelogic based on[rb]tag and event type[rb]tag