Trained baseline models perform poorly

[zrjnz]

Hello, I have trained the baseline models for all of the tasks and the results are only good for the pushing and picking tasks, and not even that good for the picking one. As for pick and place and stacking, the trained baseline model fails consistently. Is this expected behavior?

I ran the reproduce_experiments.py script and then evaluated each trained model with the evaluation pipeline. I can post the videos and the evaluation script if needed.

[ftraeuble]

Hi zrjnz,

Thanks for trying out the package and sorry for the late reply.

You should be able to reproduce the baseline results reported in the paper as shown here https://github.com/rr-learning/CausalWorld/tree/master/scripts
We trained the baseline policies on curriculum 0 and 1 to verify this and as expected we were able to get the same results with slight variance due to the random seeds. After training the policies, the policies should behave similar to the ones here https://github.com/rr-learning/CausalWorld/tree/master/causal_world/actors.

Can you post the evaluation results using the evaluation protocols as well as the videos, so we can help you better.