Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). #9136

Merged
merged 22 commits into from
Jun 30, 2020
Merged

[RLlib] Tf2x preparation; part 2 (upgrading try_import_tf()). #9136

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
fac4f24
WIP.
sven1977 Jun 24, 2020
7036bd1
Fixes.
sven1977 Jun 25, 2020
7c5b55b
LINT.
sven1977 Jun 25, 2020
154991d
Merge branch 'master' of https://github.com/ray-project/ray into tf2x…
sven1977 Jun 25, 2020
c9ed12a
WIP.
sven1977 Jun 25, 2020
10f00c2
WIP.
sven1977 Jun 25, 2020
47fe0e8
Merge branch 'master' of https://github.com/ray-project/ray into tf2x…
sven1977 Jun 25, 2020
bd417da
Fixes.
sven1977 Jun 26, 2020
05fcd8d
Fixes.
sven1977 Jun 26, 2020
274fb22
Merge branch 'master' of https://github.com/ray-project/ray into tf2x…
sven1977 Jun 26, 2020
931f71c
Merge branch 'master' of https://github.com/ray-project/ray into tf2x…
sven1977 Jun 27, 2020
4f5da24
Fixes.
sven1977 Jun 27, 2020
f7e8fcf
Fixes.
sven1977 Jun 27, 2020
4feddb6
Merge branch 'master' of https://github.com/ray-project/ray into tf2x…
sven1977 Jun 28, 2020
3c5a0f2
WIP.
sven1977 Jun 28, 2020
db17270
Fixes.
sven1977 Jun 29, 2020
97d373f
Merge branch 'master' of https://github.com/ray-project/ray into tf2x…
sven1977 Jun 29, 2020
da69b5f
Test
sven1977 Jun 29, 2020
db12f2d
Fix.
sven1977 Jun 29, 2020
5cfc394
Fixes and LINT.
sven1977 Jun 29, 2020
a793647
Fixes and LINT.
sven1977 Jun 30, 2020
12562e2
LINT.
sven1977 Jun 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
LINT.
  • Loading branch information
@sven1977
sven1977 committed Jun 25, 2020
commit 7c5b55bd1d2b1165eca2ff84d61094dc7d90fed6
2 changes: 1 addition & 1 deletion rllib/agents/a3c/a3c_tf_policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def stats(policy, train_batch):

def grad_stats(policy, train_batch, grads):
return {
"grad_gnorm" : tf.linalg.global_norm(grads),
"grad_gnorm": tf.linalg.global_norm(grads),
"vf_explained_var": explained_variance(
train_batch[Postprocessing.VALUE_TARGETS],
policy.model.value_function()),
Expand Down
12 changes: 6 additions & 6 deletions rllib/agents/impala/vtrace_tf_policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,12 +224,12 @@ def stats(policy, train_batch):
drop_last=policy.config["vtrace"])

return {
"cur_lr" : tf.cast(policy.cur_lr, tf.float64),
"policy_loss" : policy.loss.pi_loss,
"entropy" : policy.loss.entropy,
"entropy_coeff" : tf.cast(policy.entropy_coeff, tf.float64),
"var_gnorm" : tf.linalg.global_norm(policy.model.trainable_variables()),
"vf_loss" : policy.loss.vf_loss,
"cur_lr": tf.cast(policy.cur_lr, tf.float64),
"policy_loss": policy.loss.pi_loss,
"entropy": policy.loss.entropy,
"entropy_coeff": tf.cast(policy.entropy_coeff, tf.float64),
"var_gnorm": tf.linalg.global_norm(policy.model.trainable_variables()),
"vf_loss": policy.loss.vf_loss,
"vf_explained_var": explained_variance(
tf.reshape(policy.loss.value_targets, [-1]),
tf.reshape(values_batched, [-1])),
Expand Down
10 changes: 5 additions & 5 deletions rllib/agents/ppo/appo_tf_policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,11 +347,11 @@ def stats(policy, train_batch):
drop_last=policy.config["vtrace"])

stats_dict = {
"cur_lr" : tf.cast(policy.cur_lr, tf.float64),
"policy_loss" : policy.loss.pi_loss,
"entropy" : policy.loss.entropy,
"var_gnorm" : tf.linalg.global_norm(policy.model.trainable_variables()),
"vf_loss" : policy.loss.vf_loss,
"cur_lr": tf.cast(policy.cur_lr, tf.float64),
"policy_loss": policy.loss.pi_loss,
"entropy": policy.loss.entropy,
"var_gnorm": tf.linalg.global_norm(policy.model.trainable_variables()),
"vf_loss": policy.loss.vf_loss,
"vf_explained_var": explained_variance(
tf.reshape(policy.loss.value_targets, [-1]),
tf.reshape(values_batched, [-1])),
Expand Down