Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incorporate distributed RL framework, Ape-X and Ape-X DQN #246

Merged
merged 30 commits into from
Jun 23, 2020
Merged

Incorporate distributed RL framework, Ape-X and Ape-X DQN #246

merged 30 commits into from
Jun 23, 2020

Commits on Jun 19, 2020

  1. implement abstract classes for distributed and ApeX Learner wrapper

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    aeef276 View commit details
    Browse the repository at this point in the history

  2. Implement params2numpy method that loads torch state_dict as array of… 

    … np.ndarray.
    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    32c073d View commit details
    Browse the repository at this point in the history

  3. add __init__

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    01e5f2c View commit details
    Browse the repository at this point in the history

  4. implement worker as abstract class, not wrapper base class

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    4b7c7f6 View commit details
    Browse the repository at this point in the history

  5. Change apex_learner file name to learner.

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    bbe0236 View commit details
    Browse the repository at this point in the history

  6. Implement Ape-X worker and learner base classes

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    069b361 View commit details
    Browse the repository at this point in the history

  7. implement Ape-X DQN worker

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    7e19bca View commit details
    Browse the repository at this point in the history

  8. Create base class for distributed architectures

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    1cc872e View commit details
    Browse the repository at this point in the history

  9. Take context as init_communication input; all processes share the sam… 

    …e context.
    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    8c4ecf5 View commit details
    Browse the repository at this point in the history

  10. Implement and test Ape-X DQN working on Pong 

    * Accept current change (master) for PongNoFrameskip-v4 dqn config
* Make env_info more explicit in run_pong script (accept incoming change)
* Make learner return cpu state_dict (accept incoming change)
    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    f2969e1 View commit details
    Browse the repository at this point in the history

  11. Fix minor errors

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    4a73778 View commit details
    Browse the repository at this point in the history

  12. Implement ApeXWorker as a wrapper ApeXWorkerWrapper 

    Implement Logger and test wandb functionality
Add worker and logger render in argparse
Implement load_param() method in logger and worker
    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    dcbb857 View commit details
    Browse the repository at this point in the history

  13. Move num_workers to hyperparams, and add logger_interval to hyperparams.

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    22b2d21 View commit details
    Browse the repository at this point in the history

  14. Implement safe exit condition for all ray actors.

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    610f0f6 View commit details
    Browse the repository at this point in the history

  15. Change _init_communication -> init_communication and call outside of … 

    …__init__ for all ApeX actors

Implement test() in distributed architectures (load from checkpoint and run logger test())
    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    86da701 View commit details
    Browse the repository at this point in the history

  16. * Add documentation 

    * Move collect_data from worker class to ApeX Wrapper
* Change hyperparameters around
* Add worker-verbose as argparse flag
    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    4342b78 View commit details
    Browse the repository at this point in the history

  17. * Move num_worker to hyper_param cfg

    @cyoon1729
    cyoon1729 committed Jun 19, 2020
    Configuration menu
    Copy the full SHA
    d36212a View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2020

  1. * Add author 

    * Add separate integration test for ApeX
* Add integration test flag to pong
    @cyoon1729
    cyoon1729 committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    0388ac3 View commit details
    Browse the repository at this point in the history

  2. argparse integration test flag store_false->store_true

    @cyoon1729
    cyoon1729 committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    1ab5e35 View commit details
    Browse the repository at this point in the history

  3. Change default config to dqn.

    @cyoon1729
    cyoon1729 committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    b954f4d View commit details
    Browse the repository at this point in the history

  4. * Log worker scores per update step on Wandb.

    @cyoon1729
    cyoon1729 committed Jun 22, 2020
    Configuration menu
    Copy the full SHA
    a7be46b View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2020

  1. Modify integration test

    khkim committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    ab93eb6 View commit details
    Browse the repository at this point in the history

  2. Modify apex buffer config for integration test

    khkim committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    15bce10 View commit details
    Browse the repository at this point in the history

  3. Change distributed directory structure

    khkim committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    ec150b5 View commit details
    Browse the repository at this point in the history

  4. Add documentation

    khkim committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    cd0af75 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into feature/add_apex_rebase

    @MrSyee
    MrSyee authored Jun 23, 2020
    Configuration menu
    Copy the full SHA
    f3c0646 View commit details
    Browse the repository at this point in the history

  6. Modify readme.md

    khkim committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    edd0219 View commit details
    Browse the repository at this point in the history

  7. Modify readme.md

    khkim committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    e8ccea4 View commit details
    Browse the repository at this point in the history

  8. Add Ape-X to README.

    @cyoon1729
    cyoon1729 committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    64afd45 View commit details
    Browse the repository at this point in the history

  9. Add description about args flags for distributed training.

    @cyoon1729
    cyoon1729 committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    5c09012 View commit details
    Browse the repository at this point in the history