How to print action distributions and agent_id in RLlib during training?

[c4cld]

Search before asking

• I searched the issues and found no similar issues.

Ray Component

RLlib

What happened + What you expected to happen

I'm trying to print action distributions and agent_id at each timestep to see what my agent is doing. I want to achieve this goal by adding some codes and setting breakpoints in path/to/rllib/agents/ppo/ppo_policy.py in Pycharm. However, although I set some breakpoints, these breakpoints are ignored. For some reasons, I have to use tensorflow rather than pytorch. So how can I print action distributions and agent_id at each timestep?

Versions / Dependencies

ray 0.8.2
tensorflow 1.10.0

Reproduction script

git clone https://github.com/eugenevinitsky/robust_RL_multi_adversary.git

python /path/to/robust_RL_multi_adversary/run_scripts/mujoco/run_adv_mujoco.py --train_batch_size 100000 --num_iters 500 --checkpoint_freq 100 --num_concat_states 1 --num_adv_strengths 1 --advs_per_strength 2 --advs_per_rew 1 --num_adv_rews 2 --env_name hopper --exp_title hop_2adv_concat1_str0p25 --num_cpus 7 --run_transfer_tests --local_mode --adv_strength 0.25 --adv_all_actions --concat_actions

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[gjoliver]

you can add tf.Print nodes into the policcy nn?