Skip to content

[rllib] Support batch norm layers #3369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Nov 29, 2018
Merged

[rllib] Support batch norm layers #3369

merged 10 commits into from
Nov 29, 2018

Conversation

@ericl
Copy link
Contributor

@ericl ericl commented Nov 21, 2018 •
edited
Loading

What do these changes do?

  • Pass in the right is_training tensor to build_layers_v2
  • Add the right update ops to apply grads automatically
  • Example script in test_batch_norm.py

I don't think this will work for e.g., A3C which applies gradient updates separately, but it should work fine in the other execution modes.

Related issue number

Closes: #2023

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9498/
Test FAILed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 21, 2018
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9528/
Test FAILed.

@ericl ericl removed the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 22, 2018
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9542/
Test FAILed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Nov 22, 2018
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9574/
Test FAILed.

@ericl
Copy link
Contributor Author

ericl commented Nov 27, 2018

Ping @richardliaw

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9630/
Test FAILed.

richardliaw
richardliaw reviewed Nov 28, 2018
View reviewed changes
richardliaw
richardliaw reviewed Nov 28, 2018
View reviewed changes
python/ray/rllib/evaluation/tf_policy_graph.py
if self._prev_reward_input is not None and prev_reward_batch:
builder.add_feed_dict({self._prev_reward_input: prev_reward_batch})
builder.add_feed_dict({self._is_training: is_training})
builder.add_feed_dict({self._is_training: False})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where in the code is _is_training True?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ray/python/ray/rllib/evaluation/tf_policy_graph.py

Line 227 in dedd3a8

builder.add_feed_dict({self._is_training: True})

richardliaw
richardliaw reviewed Nov 28, 2018
View reviewed changes
Copy link
Contributor

@richardliaw richardliaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9660/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9661/
Test FAILed.

@ericl ericl merged commit 07d8cbf into ray-project:master Nov 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@richardliaw richardliaw richardliaw approved these changes

Assignees

@richardliaw richardliaw

Labels

tests-ok The tagger certifies test failures are unrelated and assumes personal liability.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ericl @AmplabJenkins @richardliaw