-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: replace eval with a safer alternative #147
fix: replace eval with a safer alternative #147
Conversation
2140c82
to
9e52199
Compare
fef6e45
to
fa48a53
Compare
091ea3f
to
1c14a3e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this is a really interesting contribution - a lot of questions and one request to split the PR up a little 😄
tuning/trainercontroller/callback.py
Outdated
@@ -59,6 +67,21 @@ | |||
CONTROLLER_OPERATIONS_KEY = OPERATIONS_KEY | |||
|
|||
|
|||
def get_evaluator(metrics: dict) -> EvalWithCompoundTypes: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any thoughts on making this part of the class, e.g., as a staticmethod? It feels a bit strange to me to store it separately here like this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EvalWithCompoundTypes
is coming directly from the library.
Making it a static method would require overriding the class.
Since this is just a utility function I have moved it to a more appropriate location.
We can also remove the function entirely if necessary.
e5f8d09
to
678cf68
Compare
012447a
to
1236039
Compare
1236039
to
6f540a9
Compare
docs: update documentation with the new format for controller metrics and operations and details of rule evaluation Signed-off-by: Harikrishnan Balagopal <harikrishmenon@gmail.com>
6f540a9
to
c8a56d6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. LGTM, thanks!
docs: update documentation with the new format for controller metrics and operations and details of rule evaluation Signed-off-by: Harikrishnan Balagopal <harikrishmenon@gmail.com>
Description of the change
Use the AST module to evaluate the rules instead of
eval
.Example supported expressions: https://github.com/danthedeckie/simpleeval?tab=readme-ov-file#operators
Support for more complex types (list, dict, etc.) are also implemented:
Notes:
["hello"]*10
is allowed but limited to smaller numbers to avoid Denial of Service (DOS).9**9
is allowed but limited to smaller numbers to avoid DOS.__class__
is disallowed to avoid access to arbitrary classes likeQuitter
.int
,float
,str
,rand
andrandint
.foo.bar
. Example dict:{"foo": {"bar": 42}})
Related issue number
Fixes #148
How to verify the PR
Have added unit tests for the safe evaluator.
make test
Was the PR tested