-
-
Couldn't load subscription status.
- Fork 10.8k
Disable failing GPT-OSS Eval (Blackwell) for now #25107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: mgoin <mgoin64@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request disables a failing GPT-OSS evaluation test for Blackwell GPUs in the CI pipeline by marking it as optional. While this is a reasonable temporary measure to unblock development, it introduces a risk of the test remaining disabled indefinitely. My review includes a suggestion to add a tracking issue to ensure the test is eventually fixed and re-enabled.
| working_dir: "/vllm-workspace/" | ||
| gpu: b200 | ||
| # optional: true | ||
| optional: true # disable while debugging |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While disabling a failing test is a reasonable temporary measure to unblock the CI, there's a risk it might be forgotten and left disabled permanently. This would prevent catching regressions in the GPT-OSS evaluation on Blackwell GPUs. To mitigate this, it's highly recommended to create a tracking issue to fix and re-enable this test, and reference the issue number in a comment here.
optional: true # TODO(<issue-link>): Re-enable this test after fixing the failures.Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Purpose
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.