Skip to content

speedup accessing actions one agent at a time #4261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

speedup accessing actions one agent at a time #4261

wants to merge 1 commit into from

Conversation

chriselion
Copy link
Contributor

@chriselion chriselion commented Jul 22, 2020 •
edited
Loading

Proposed change(s)

This showed up as a non-trivial portion of time (14s / 204s) when profiling with cprofile.

Real-life profiling with 3Dball:

master
2020-07-22 13:27:45 INFO [stats.py:112] 3DBall: Step: 492000. Time Elapsed: 193.277 s Mean Reward: 100.000. Std of Reward: 0.000. Training.

optimized
2020-07-22 14:56:14 INFO [stats.py:112] 3DBall: Step: 492000. Time Elapsed: 187.601 s Mean Reward: 100.000. Std of Reward: 0.000. Training.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

  • Bug fix
  • New feature
  • Code refactor
  • Breaking change
  • Documentation update
  • Other (please describe)

Checklist

  • Added tests that prove my fix is effective or that my feature works
  • Updated the changelog (if applicable)
  • Updated the documentation (if applicable)
  • Updated the migration guide (if applicable)

Other comments

@chriselion chriselion requested a review from ervteng July 22, 2020 22:40
ervteng
ervteng reviewed Jul 23, 2020
View reviewed changes
ml-agents/mlagents/trainers/policy/tf_policy.py
@@ -353,6 +353,16 @@ def retrieve_previous_action(self, agent_ids: List[str]) -> np.ndarray:
action_matrix[index, :] = self.previous_action_dict[agent_id]
return action_matrix

def retrieve_previous_action_single(self, agent_id: str) -> np.ndarray:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method needs to be moved to policy.py with the other retrieve methods (#4254).

ervteng
ervteng reviewed Jul 23, 2020
View reviewed changes
ml-agents/mlagents/trainers/agent_processor.py
@@ -137,7 +137,7 @@ def _process_step(
action_pre = None
action_probs = stored_take_action_outputs["log_probs"][idx]
action_mask = stored_decision_step.action_mask
prev_action = self.policy.retrieve_previous_action([global_id])[0, :]
prev_action = self.policy.retrieve_previous_action_single(global_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be opposed to renaming the method retrieve_previous_action and the old method to retrieve_previous_actions? Seems more descriptive and more similar to the methods in the LL-API.

@chriselion chriselion closed this Dec 8, 2020
@chriselion chriselion deleted the MLA-1173-retrieve_previous_action branch December 18, 2020 23:56
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ervteng ervteng ervteng left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@chriselion @ervteng