Reject SPSA int params with C < 0.50 during workload validation#1
Conversation
There was a problem hiding this comment.
Pull request overview
This PR tightens SPSA workload parameter validation so that integer-typed parameters cannot be submitted with an invalid perturbation constant C (< 0.50), preventing later divide-by-zero/invalid scaling during workload assignment.
Changes:
- Update SPSA datatype validation error message to correctly say “int or float”.
- Add a new validation rule:
Cmust be>= 0.50whendata_typeisint. - (Unrelated/likely accidental) Modify
fetch_artifact_urlin a way that breaks the function/module.
Comments suppressed due to low confidence (1)
OpenBench/workloads/verify_workload.py:501
fetch_artifact_urlnow ends with atry:block but noexcept/finallyand no return path. This makes the module fail to import (invalidtrystatement) and the newassert len(artifacts) >= lenis also incorrect (compares to the built-inlenfunction). Restore the previous completion/return + exception handling logic and fix the assertion to compare against the intended value (likelylen(jobs)).
# All jobs finished, with at least one non-expired Artifact
assert not any(job['conclusion'] != 'success' for job in jobs)
assert not any(artifact['expired'] for artifact in artifacts)
assert len(artifacts) >= len
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Agent-Logs-Url: https://github.com/yohaann196/OpenBench/sessions/33a68a30-9a82-4a73-8d8d-a08c562c7bc8 Co-authored-by: yohaann196 <229655281+yohaann196@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
verify_spsa_inputsalready rejected float params with C <= 0.00, buthad no equivalent check for int params. A C of 0 for an int param would
pass validation, get stored, and then cause a divide-by-zero in
spsa_workload_assignment_dictwhen computingr_values = a / c ** 2.The minimum meaningful C for an int param is 0.50, which matches the
floor already applied in
spsa_utils.py:This fix rejects bad input at submission time with a clear error message,
consistent with how the rest of workload validation works.