Skip to content

Refactor DemonstrationStore/Recorder #3354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Feb 22, 2020
Merged

Conversation

chriselion
Copy link
Contributor

@chriselion chriselion commented Feb 5, 2020

Clean up the DemonstrationStore/DemonstrationRecorder API:

  • Agent now maintains a set of DemonstrationStores, and sends it’s AgentInfo and sensors to each of these
  • DemonstrationStore’s only public methods are a constructor that takes a Stream and an Initialization method. Users that want to add their own writing logic (e.g. chunking into different files, compression) can add the logic in a custom Stream, make a new DemonstrationStore, and pass to the Agent.
  • DemonstrationRecorder basically just makes a file Stream, makes a DemonstrationStore, and passes to the Agent.

Other quality-of-life improvements:

  • Default write directory is now "{Application.dataPath}/Demonstrations", but this can be overridden in the DemonstrationRecorder component.
  • No longer requires editor mode for recording.

I will rename DemonstrationStore -> DemonstrationWriter in a followup PR; there were too many other changes in this, and I didn't want to complicate the diffs with file moves.

/// You can also add your own DemonstrationStores; the Agent is not responsible for creating or closing the
/// stores, only opening them.
/// </summary>
public ISet<DemonstrationStore> DemonstrationStores = new HashSet<DemonstrationStore>();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can hide this a bit and add AddDemonstrationStore/RemoveDemonstrationStore methods.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like it would be great if we could just completely remove this demonstration stuff from Agent. It feels like it could happen. But there is no way to hook into Agent at the moment to know when SendInfo is called on a specific one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could put this in the policy factory maybe? This way we can wrap the policy in a recording policy that looks for demo recorder components at initialization.

@chriselion chriselion marked this pull request as ready for review February 13, 2020 21:00
Copy link
Contributor

@surfnerd surfnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the changes look good. If we could remove the demo store stuff from Agent, it feels like that would be ideal. Maybe down the road, we can figure that out.

m_DemoStore.Close();
m_DemoStore = null;
}
}

/// <summary>
/// Closes Demonstration store.
/// Clean up the DemonstrationStore when shutting down.
/// </summary>
void OnApplicationQuit()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not onDestroy ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's how it was before :) But yeah, sounds like a better place for it.

@chriselion
Copy link
Contributor Author

If we could remove the demo store stuff from Agent, it feels like that would be ideal.

Open to suggestions on how to do this. Maybe an IPolicy that wraps the "main" IPolicy, and forwards everything to the wrapped IPolicy and also handles the writing?

Anyway, you're happy revisiting this later, right?

@@ -10,7 +10,7 @@ namespace MLAgents
/// Struct that contains all the information for an Agent, including its
/// observations, actions and current status, that is sent to the Brain.
/// </summary>
public struct AgentInfo
internal struct AgentInfo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

/// You can also add your own DemonstrationStores; the Agent is not responsible for creating or closing the
/// stores, only opening them.
/// </summary>
public ISet<DemonstrationStore> DemonstrationStores = new HashSet<DemonstrationStore>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could put this in the policy factory maybe? This way we can wrap the policy in a recording policy that looks for demo recorder components at initialization.

@chriselion chriselion merged commit ef2b170 into master Feb 22, 2020
@delete-merged-branch delete-merged-branch bot deleted the develop-refactor-demo-store branch February 22, 2020 23:39
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants