Develop new ll api #3022

vincentpierre · 2019-12-04T00:48:50Z

Todo : Documentation changes in other PR #3042

gym-unity/gym_unity/tests/test_gym.py

chriselion · 2019-12-05T23:38:10Z

ml-agents-envs/mlagents/envs/base_env.py

+        specific agent
+        """
+        try:
+            agent_index = np.where(self.agent_id == agent_id)[0][0]


What do you think about having an agent_id -> index dictionary, but only building it the first time get_agent_step_result() is called? I'm worried that this is O(num agents) per call, and if you're using this once, you'll probably use it (num agents) times.

I am unsure since BatchedStepResponse is a namedtuple, I do not think I can have references to dicts in there.

You definitely can have a reference (it's python! everything is a reference!) but I forget how to have a default value like this. If you think it would be useful I'll look into it.

Made it an object for now. We can see if we really need it to be a named tuple.

Thanks. If you really want to prematurely optimize things, you can start adding __slots__ to classes...

Also https://stackoverflow.com/questions/59205528/typing-namedtuple-with-a-dictionary-default

ml-agents-envs/mlagents/envs/base_env.py

Co-Authored-By: Chris Elion <chris.elion@unity3d.com>

ml-agents-envs/mlagents/envs/base_env.py

ml-agents-envs/mlagents/envs/brain_conversion_utils.py

* LL-API documentation changes * fixes * deleting implementation details * Update docs/Python-API.md Co-Authored-By: Chris Elion <chris.elion@unity3d.com> * edited the migrating docs * Update docs/Migrating.md Co-Authored-By: Chris Elion <chris.elion@unity3d.com>

docs/Python-API.md

harperj

Over all this PR looks good to me, but since it's rather large it would be good to hear some idea of what cases you've tested. Have you retrained all scened using cloud training for this branch? Have you tested it on Windows?

harperj · 2019-12-09T19:45:22Z

ml-agents-envs/mlagents/envs/base_env.py

+one agent in the simulation sends its observations to Python again. Since
+Agents can request decisions at different frequencies, a simulation step does
+not necessarily correspond to a fixed simulation time increment.
+Changes from ML-Agents v0.11 :


Minor but is this a good place for a changelist? I worry it will become out of date quickly

I put this here because it is major change but I can remove it.
I have not tested on windows, will do.

windows

cloud (looking at some of the logs generated, the training seems to be going fine)

Looks like it is working

ervteng · 2019-12-09T19:54:32Z

Over all this PR looks good to me, but since it's rather large it would be good to hear some idea of what cases you've tested. Have you retrained all scened using cloud training for this branch? Have you tested it on Windows?

+1 for cloud training

ervteng

As long as cloud runs seem OK,

vincentpierre added 2 commits December 3, 2019 16:47

initial commit for LL-API

a4ae6c7

fixing ml-agents-envs tests

7acd227

vincentpierre self-assigned this Dec 4, 2019

vincentpierre added 5 commits December 4, 2019 10:42

Implementing action masks

c49be07

training is fixed for 3DBall

bcce783

Tests all fixed, gym is broken and missing documentation changes

942c0d2

adding case where no vector obs

7ccf5fc

Fixed Gym

89b1d34

vincentpierre marked this pull request as ready for review December 5, 2019 00:01

vincentpierre requested review from chriselion, harperj and ervteng and removed request for chriselion December 5, 2019 00:03

vincentpierre added 4 commits December 4, 2019 16:11

fixing tests of float64

48d675d

fixing float64

9d2d70c

reverting some of brain.py

099f12b

removing old proto apis

02cae4d

vincentpierre changed the base branch from develop to master December 5, 2019 19:19

vincentpierre added 2 commits December 5, 2019 12:47

comment type fixes

782584b

added properties to AgentGroupSpec and edited the notebooks.

d0b6d7d

vincentpierre mentioned this pull request Dec 5, 2019

LL-API documentation changes #3042

Closed

clearing the notebook outputs

6d09c91

chriselion reviewed Dec 5, 2019

View reviewed changes

gym-unity/gym_unity/tests/test_gym.py Show resolved Hide resolved

chriselion reviewed Dec 5, 2019

View reviewed changes

gym-unity/gym_unity/tests/test_gym.py Outdated Show resolved Hide resolved

chriselion reviewed Dec 5, 2019

View reviewed changes

gym-unity/gym_unity/tests/test_gym.py Outdated Show resolved Hide resolved

chriselion reviewed Dec 5, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

chriselion reviewed Dec 5, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

chriselion reviewed Dec 5, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

vincentpierre added 5 commits December 5, 2019 16:20

addressing first comments

cc4456a

NaN checks for rewards are back

5a45d7b

restoring Union[int, Tuple[int, ...]] for action_shape

5b8a354

Made BatchdStepResult an object

3d0cad7

Made _agent_id_to_index private

1148f35

chriselion reviewed Dec 6, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

chriselion reviewed Dec 6, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

Update ml-agents-envs/mlagents/envs/base_env.py

d25a885

Co-Authored-By: Chris Elion <chris.elion@unity3d.com>

ervteng reviewed Dec 6, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

ervteng reviewed Dec 6, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Show resolved Hide resolved

ervteng reviewed Dec 6, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/base_env.py Outdated Show resolved Hide resolved

vincentpierre added 2 commits December 6, 2019 11:16

replacing np.array with np.ndarray in typing

329fa34

adding a new type for AgentGroup and AgentId

98e10fe

ervteng reviewed Dec 6, 2019

View reviewed changes

ml-agents-envs/mlagents/envs/brain_conversion_utils.py Outdated Show resolved Hide resolved

fixing brain_info when vec_obs == 0

6da142d

chriselion previously approved these changes Dec 9, 2019

View reviewed changes

Docs ll api (#3047)

617a768

* LL-API documentation changes * fixes * deleting implementation details * Update docs/Python-API.md Co-Authored-By: Chris Elion <chris.elion@unity3d.com> * edited the migrating docs * Update docs/Migrating.md Co-Authored-By: Chris Elion <chris.elion@unity3d.com>

vincentpierre dismissed chriselion’s stale review via 617a768 December 9, 2019 19:22

ervteng reviewed Dec 9, 2019

View reviewed changes

docs/Python-API.md Outdated Show resolved Hide resolved

chriselion previously approved these changes Dec 9, 2019

View reviewed changes

adding a period

d8b52c2

vincentpierre dismissed chriselion’s stale review via d8b52c2 December 9, 2019 19:47

harperj reviewed Dec 9, 2019

View reviewed changes

removing change log

81acfaf

ervteng approved these changes Dec 9, 2019

View reviewed changes

vincentpierre merged commit d92f528 into master Dec 9, 2019

delete-merged-branch bot deleted the develop-new-ll-api branch December 9, 2019 23:58

chriselion mentioned this pull request Dec 10, 2019

Fix setup in jupyter notebook. #3068

Merged

github-actions bot locked as resolved and limited conversation to collaborators May 17, 2021

Develop new ll api #3022

Develop new ll api #3022

Uh oh!

Conversation

vincentpierre commented Dec 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

harperj left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vincentpierre Dec 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ervteng commented Dec 9, 2019

Uh oh!

ervteng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vincentpierre commented Dec 4, 2019 •

edited

Loading

vincentpierre Dec 9, 2019 •

edited

Loading