-
Notifications
You must be signed in to change notification settings - Fork 6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[rllib] Document creating an ensemble of envs; also add vector_index attribute to env config #2513
Conversation
Test FAILed. |
Test FAILed. |
Test FAILed. |
Test FAILed. |
6e9aaaa
to
2a13d7d
Compare
Test FAILed. |
Test FAILed. |
Test FAILed. |
Test PASSed. |
What do these changes do?
This also removes the async resetting code in VectorEnv. While that improves benchmark performance slightly, it substantially complicates env configuration and probably isn't worth it for most envs.
This makes it easy to efficiently support setups like Joint PPO: https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/retro-contest/gotta_learn_fast_report.pdf
For example, for 188 envs, you could do something like
num_envs: 10, num_envs_per_worker: 19
.