-
Notifications
You must be signed in to change notification settings - Fork 4.3k
make sure DefaultTrainerDict is pickle-able #4842
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Without the fix, the added test fails: def test_pickle():
# Make sure RunOptions is pickle-able.
run_options = RunOptions()
p = pickle.dumps(run_options)
> pickle.loads(p)
ml-agents/mlagents/trainers/tests/test_settings.py:526:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = DefaultTrainerDict(<class 'mlagents.trainers.settings.TrainerSettings'>, {})
args = (<class 'mlagents.trainers.settings.TrainerSettings'>,)
def __init__(self, *args):
> super().__init__(TrainerSettings, *args)
E TypeError: 'type' object is not iterable
ml-agents/mlagents/trainers/settings.py:684: TypeError |
@@ -681,7 +681,12 @@ def structure(d: Mapping, t: type) -> Any: | |||
|
|||
class DefaultTrainerDict(collections.defaultdict): | |||
def __init__(self, *args): | |||
super().__init__(TrainerSettings, *args) | |||
# Depending on how this is called, args may have the defaultdict | |||
# callable at the start of the list or not. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a note about how it addresses an issue with pickle load for posterity
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
only change between the last commit and the previous one was adding a comment, so going to merge without tests finishing. |
Proposed change(s)
In the current form, the DefaultTrainerDict can't be deserialized from pickle (and thus can't be passed to a worker process from the subprocess module, which I'm trying to do in #4780). It appears that the initializer needs to be able to take both forms of arguments; I'm not sure if there's a cleaner way to do this.
Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)
Added in #4448
Types of change(s)
Checklist