Optimizer setting in PPO experiments

[takasho777]

Hi! I'm using the PPO implementation for my custom environment with continuous action space. I built my custom experiments based on the PPO pendulum experiment template, where the actor and critic are defined explicitly with optimizer=ADAM(3e-4). After playing with it for a while, I realized that I have to use the optimizer defined as part of the ActorCritic type if I want to change the learning rate, etc. It looks the optimizers defined for actor and critic are not used, so it would be less confusing if the optimizer is specified only for the ActorCritic call in the template.

[findmyway]

Good catch! Thanks for raising this issue!

The definition of ActorCritic struct was changed some time ago. But the usage was not changed. I'll fix it soon.