Hotfix memory leak on Python #3664

vincentpierre · 2020-03-20T18:57:38Z

Proposed change(s)

The agents will be marked as Done when changing policies.
@ervteng can you see if this helps the leak?

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

Checklist

Added tests that prove my fix is effective or that my feature works
Updated the changelog (if applicable)
Updated the documentation (if applicable)
Updated the migration guide (if applicable)

Other comments

…ted when the agent is done

chriselion · 2020-03-20T21:33:40Z

com.unity.ml-agents/Runtime/Communicator/RpcCommunicator.cs

@@ -458,13 +458,16 @@ UnityRLInitializationOutputProto GetTempUnityRlInitializationOutput()
            {
                if (m_CurrentUnityRlOutput.AgentInfos.ContainsKey(behaviorName))
                {
-                    if (output == null)
+                    if (m_CurrentUnityRlOutput.AgentInfos[behaviorName].CalculateSize() > 0)


Can you explain why this is necessary? How could the AgentInfos for a behavior name be in the dictionary but the list be empty?

I am still investigating that part.

(discussed on slack) This change sounds OK, but can you add some comments to clarify what's going on (my bad for not documenting better in the first place). Something like "Check the BrainParameters that we haven't sent yet, and add them to the InitializationOutput if we have observations for that behavior this step."

chriselion · 2020-03-20T21:52:23Z

com.unity.ml-agents/Runtime/Policies/HeuristicPolicy.cs

@@ -29,7 +29,10 @@ public HeuristicPolicy(Func<float[]> heuristic)
        public void RequestDecision(AgentInfo info, List<ISensor> sensors)
        {
            StepSensors(sensors);
-            m_LastDecision = m_Heuristic.Invoke();
+            if (!info.done)


We clear the done flag right be calling RequestDecision - not sure this will have any effect.

ml-agents/com.unity.ml-agents/Runtime/Agent.cs

Lines 623 to 627 in 6a11071

m_Info.done = false;

m_Info.maxStepReached = false;

m_Info.episodeId = m_EpisodeId;

m_Brain.RequestDecision(m_Info, sensors);

Yes, I want to avoid it being called when using NotifyAgentDone().
It does have an effect in that regard...

ervteng · 2020-03-21T01:59:50Z

It's still leaking, thought for sure this would fix the issue. Gonna look into the Python side and see what's going on there

ervteng · 2020-03-23T21:04:20Z

Going to run another cloud run with the Python fix #3671. If it works we can merge this and it would likely need to be in a hotfix 0.15.1.

ervteng · 2020-03-24T04:46:27Z

Much better now, old one would have hit 8GB by now:

ervteng

👍

* Hotfix memory leak on Python * Fixing * Fixing a bug in the heuristic policy. A decision should not be requested when the agent is done * [bug-fix] Make Python able to deal with 0-step episodes (#3671) * adding some comments Co-authored-by: Ervin T <ervin@unity3d.com>

* [bug-fix] Increase height of wall in CrawlerStatic (#3650) * [bug-fix] Improve performance for PPO with continuous actions (#3662) * Corrected a typo in a name of a function (#3670) OnEpsiodeBegin was corrected to OnEpisodeBegin in Migrating.md document * Add Academy.AutomaticSteppingEnabled to migration (#3666) * Fix editor port in Dockerfile (#3674) * Hotfix memory leak on Python (#3664) * Hotfix memory leak on Python * Fixing * Fixing a bug in the heuristic policy. A decision should not be requested when the agent is done * [bug-fix] Make Python able to deal with 0-step episodes (#3671) * adding some comments Co-authored-by: Ervin T <ervin@unity3d.com> * Remove vis_encode_type from list of required (#3677) * Update changelog (#3678) * Shorten timeout duration for environment close (#3679) The timeout duration for closing an environment was set to the same duration as the timeout when waiting for a response from the still-running environment. This led to long waits for the error response when communication version wasn't matching. This change forces a timeout duration of 0 when handling errors. * Bumping the versions * handle multiple dones in a single step (#3700) * handle multiple dones in a single step * [tests] Make end-to-end tests more stable (#3697) * [bug-fix] Fix entropy computation for GaussianDistribution (#3684) * Fix how we set logging levels (#3703) * cleanup logging * comments and cleanup * pylint, gym * [skip-ci] Update changelog for logging fix. (#3707) * [skip ci] Update README * [skip ci] Fixed a typo Co-authored-by: Ervin T <ervin@unity3d.com> Co-authored-by: Adam Streck <adam.streck@gmail.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: Jonathan Harper <jharper+moar@unity3d.com>

* Bumping version on the release (#3615) * Update examples project to 2018.4.18f1 (#3618) From 2018.4.14f1. An internal package dependency was updated as a side effect. * Remove dead components from the examples scenes (#3619) (#3624) * Improve warnings and exception if using unsupported combo * add meta file * fix unit test * enforce onnx conversion (expect tf2 CI to fail) (#3600) * Update error message * Updated the release branch docs (#3621) * Updated the release branch docs * Edited the README * make sure top-level timer is closed before writing * Remove space from Product Name for examples In #2588 it was suggested that the space in the Product Name for our example environments causes confusion when using a default build because of the need to escape the space in the build filename. This change removes the space from the Product Name in the project's player settings. * [bug-fix] Increase 3dballhard and GAIL default steps (#3636) * Updating the NN models (#3632) * Updating the NN models * Update gridworld * [skip ci] Update BallHard * Update hallway * Hotfixes for Release 0.15.1 (#3698) * [bug-fix] Increase height of wall in CrawlerStatic (#3650) * [bug-fix] Improve performance for PPO with continuous actions (#3662) * Corrected a typo in a name of a function (#3670) OnEpsiodeBegin was corrected to OnEpisodeBegin in Migrating.md document * Add Academy.AutomaticSteppingEnabled to migration (#3666) * Fix editor port in Dockerfile (#3674) * Hotfix memory leak on Python (#3664) * Hotfix memory leak on Python * Fixing * Fixing a bug in the heuristic policy. A decision should not be requested when the agent is done * [bug-fix] Make Python able to deal with 0-step episodes (#3671) * adding some comments Co-authored-by: Ervin T <ervin@unity3d.com> * Remove vis_encode_type from list of required (#3677) * Update changelog (#3678) * Shorten timeout duration for environment close (#3679) The timeout duration for closing an environment was set to the same duration as the timeout when waiting for a response from the still-running environment. This led to long waits for the error response when communication version wasn't matching. This change forces a timeout duration of 0 when handling errors. * Bumping the versions * handle multiple dones in a single step (#3700) * handle multiple dones in a single step * [tests] Make end-to-end tests more stable (#3697) * [bug-fix] Fix entropy computation for GaussianDistribution (#3684) * Fix how we set logging levels (#3703) * cleanup logging * comments and cleanup * pylint, gym * [skip-ci] Update changelog for logging fix. (#3707) * [skip ci] Update README * [skip ci] Fixed a typo Co-authored-by: Ervin T <ervin@unity3d.com> Co-authored-by: Adam Streck <adam.streck@gmail.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: Jonathan Harper <jharper+moar@unity3d.com> * fix changelog * keep master gridworld Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> Co-authored-by: Jonathan Harper <jharper+moar@unity3d.com> Co-authored-by: Ervin T <ervin@unity3d.com> Co-authored-by: Adam Streck <adam.streck@gmail.com>

Hotfix memory leak on Python

7c199a3

vincentpierre requested review from ervteng and chriselion March 20, 2020 18:57

vincentpierre self-assigned this Mar 20, 2020

vincentpierre added 2 commits March 20, 2020 13:05

Fixing

cc7a4a1

Fixing a bug in the heuristic policy. A decision should not be reques…

50a8e94

…ted when the agent is done

chriselion reviewed Mar 20, 2020

View reviewed changes

[bug-fix] Make Python able to deal with 0-step episodes (#3671)

7bdaac5

adding some comments

dd71442

ervteng approved these changes Mar 24, 2020

View reviewed changes

vincentpierre merged commit 1dd6cb6 into master Mar 24, 2020

delete-merged-branch bot deleted the develop-fix-wall-jump branch March 24, 2020 17:28

ervteng mentioned this pull request Mar 24, 2020

[changelog] Update changelog for Walljump memory leak #3678

Merged

10 tasks

vincentpierre mentioned this pull request Mar 26, 2020

Hotfixes for Release 0.15.1 #3698

Merged

10 tasks

github-actions bot locked as resolved and limited conversation to collaborators May 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Hotfix memory leak on Python #3664

Hotfix memory leak on Python #3664

Uh oh!

vincentpierre commented Mar 20, 2020

Uh oh!

chriselion Mar 20, 2020

Uh oh!

vincentpierre Mar 20, 2020

Uh oh!

chriselion Mar 23, 2020

Uh oh!

chriselion Mar 20, 2020

Uh oh!

vincentpierre Mar 20, 2020

Uh oh!

ervteng commented Mar 21, 2020

Uh oh!

ervteng commented Mar 23, 2020

Uh oh!

ervteng commented Mar 24, 2020

Uh oh!

ervteng left a comment

Uh oh!

Uh oh!

	m_Info.done = false;
	m_Info.maxStepReached = false;
	m_Info.episodeId = m_EpisodeId;

	m_Brain.RequestDecision(m_Info, sensors);

Hotfix memory leak on Python #3664

Hotfix memory leak on Python #3664

Uh oh!

Conversation

vincentpierre commented Mar 20, 2020

Proposed change(s)

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

Checklist

Other comments

Uh oh!

chriselion Mar 20, 2020

Choose a reason for hiding this comment

Uh oh!

vincentpierre Mar 20, 2020

Choose a reason for hiding this comment

Uh oh!

chriselion Mar 23, 2020

Choose a reason for hiding this comment

Uh oh!

chriselion Mar 20, 2020

Choose a reason for hiding this comment

Uh oh!

vincentpierre Mar 20, 2020

Choose a reason for hiding this comment

Uh oh!

ervteng commented Mar 21, 2020

Uh oh!

ervteng commented Mar 23, 2020

Uh oh!

ervteng commented Mar 24, 2020

Uh oh!

ervteng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!