-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Refactor DemonstrationStore/Recorder #3354
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/DemonstrationStore.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Demonstrations/IExperienceWriter.cs
Outdated
Show resolved
Hide resolved
com.unity.ml-agents/Runtime/Agent.cs
Outdated
/// You can also add your own DemonstrationStores; the Agent is not responsible for creating or closing the | ||
/// stores, only opening them. | ||
/// </summary> | ||
public ISet<DemonstrationStore> DemonstrationStores = new HashSet<DemonstrationStore>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can hide this a bit and add AddDemonstrationStore/RemoveDemonstrationStore methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seems like it would be great if we could just completely remove this demonstration stuff from Agent. It feels like it could happen. But there is no way to hook into Agent at the moment to know when SendInfo is called on a specific one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could put this in the policy factory maybe? This way we can wrap the policy in a recording policy that looks for demo recorder components at initialization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall the changes look good. If we could remove the demo store stuff from Agent, it feels like that would be ideal. Maybe down the road, we can figure that out.
m_DemoStore.Close(); | ||
m_DemoStore = null; | ||
} | ||
} | ||
|
||
/// <summary> | ||
/// Closes Demonstration store. | ||
/// Clean up the DemonstrationStore when shutting down. | ||
/// </summary> | ||
void OnApplicationQuit() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not onDestroy ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's how it was before :) But yeah, sounds like a better place for it.
Open to suggestions on how to do this. Maybe an IPolicy that wraps the "main" IPolicy, and forwards everything to the wrapped IPolicy and also handles the writing? Anyway, you're happy revisiting this later, right? |
@@ -10,7 +10,7 @@ namespace MLAgents | |||
/// Struct that contains all the information for an Agent, including its | |||
/// observations, actions and current status, that is sent to the Brain. | |||
/// </summary> | |||
public struct AgentInfo | |||
internal struct AgentInfo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
com.unity.ml-agents/Runtime/Agent.cs
Outdated
/// You can also add your own DemonstrationStores; the Agent is not responsible for creating or closing the | ||
/// stores, only opening them. | ||
/// </summary> | ||
public ISet<DemonstrationStore> DemonstrationStores = new HashSet<DemonstrationStore>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could put this in the policy factory maybe? This way we can wrap the policy in a recording policy that looks for demo recorder components at initialization.
Clean up the DemonstrationStore/DemonstrationRecorder API:
Other quality-of-life improvements:
I will rename DemonstrationStore -> DemonstrationWriter in a followup PR; there were too many other changes in this, and I didn't want to complicate the diffs with file moves.