Why split repos?

[drozzy]

Sorry for the silly question:

Why are the components of ReinforcementLearning.jl split up into different repos?
It makes is really hard to navigate and search for things as an end-user.
While it is good for the developer it's not as friendly towards the user of the package.

At least the docs should all be in the same place?
For example, I was just going to look up how to do equivalent to the following silly random loop (first thing I write when I begin coding a new project):

import gym
env = gym.make('CartPole-v0')
env.reset()
for _ in range(1000): # run for 1000 steps
    env.render()
    action = env.action_space.sampe() # pick a random action
    env.step(action) # take action

But I'm not sure where to look:

• ReinforcementLearning Guide?
• Doc? That seems to be just api doc
• ReinforcementLearningZoo.jl - for a simple cartpole?
• ReinforcementLearningEnvironments.jl? Seems like it's the same as zoo...
• ReinforcementLearningBase.jl - bacause maybe it has a simple env.step example?

Perhaps we should make a single doc with all the sections on the left hand side, so a user can quickly glance over what's available, for example: https://stable-baselines3.readthedocs.io/en/master/

[findmyway]

Actually, this is a very good question!

Why are the components of ReinforcementLearning.jl split up into different repos?

The answer is short: reuse.

We split the package into RLBase, RLCore, RLZoo, RLEnvs, so that different people may only depend on parts of them.

• For developers who want to define customized environments, they can only depend on RLBase.
• For developers who want to write their own algorithms, they may only depend on RLCore to reuse some components.
• To apply existing algorithms to customized environments, RLZoo is the only dependency.

I think some other orgs also adopt this kind of project structure. For example POMDP (@zsunberg may have more words on it), JuliaImages and so on.

However, we can make these packages more coherent by using sub-packages to move code and docs into this repo. One thing I haven't figured out with sub-packages is how to run tests for a specific package in each PR.

Perhaps we should make a single doc with all the sections on the left hand side, so a user can quickly glance over what's available, for example: https://stable-baselines3.readthedocs.io/en/master/

Thanks for this suggestion. I'll go for it in the next version.

[drozzy]

Yeah makes sense.
But if I'm looking for a specific thing like RandomPolicy I don't know where to start looking.
But if we have a single doc reference that has all the relevant docs then I could just search for "RandomPolicy" in the doc.

[drozzy]

Actually this is already possible here:
https://juliareinforcementlearning.org/ReinforcementLearning.jl/latest/search/?q=randompolicy

So I guess it's just a matter of making the docs have the section "Reference" as in the other discussion. #210

[findmyway]

All the sub-packages are moved into this repo now.