Skip to content

Add R2D1 agents #248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 26 commits into from
Jun 23, 2020
Merged

Add R2D1 agents #248

merged 26 commits into from
Jun 23, 2020

Conversation

jinPrelude
Copy link
Collaborator

@jinPrelude jinPrelude commented Jun 23, 2020 •
edited
Loading

Add R2D1 agents which can be combined with q-learning algorithms(DQN, C51, IQN).

  1. IQN(blue) vs R2D1IQN(red)
    pong_r2d1iqn

  2. C51(blue) vs R2D1C51(red)
    pong_r2d1c51

  3. DQN(blue) vs R2D1DQN(red)
    pong_r2d1dqn

  4. IQN ResNet(blue) vs R2D1DQN ResNet(red)
    pong_resnet

  5. lunarlander C51(blue) vs R2D1C51(red)
    lunarlander_r2d1

jinPrelude and others added 21 commits June 17, 2020 16:30
@jinPrelude
Add R2D1 DQNAgent
be144ba 
Change grandiosely used functions

Fix zero-padding & torch contiguous

Fix zero-padding & Change indices sampling function

Change hyperparameters

Remove redundant codes

Add CNN compatibility to R2D1Agent

Remove redundant code

Implement rlpyt forward style

Add previous_action & previous_reward GRU input structure

Fix error

Fix prev_action bug & Use make_one_hot function

Fix error

Update descriptions & move leading_dims functions to helper_functions.py

Move valid_from_done from R2D1Loss to helper_functions.py

Fix parameters

r2d1_iqn loss & agent

Fix GRUBrain compatible with c51

Add R2D1C51Loss

Add r2d1_c51 configs

Fix priority > 0 assert error

Change parameters
@jinPrelude
Fix descriptions & Rename R2D1 to R2D1DQN
720926e 
Change parameters

Add total_step to wandb log

Add upndown env & configs

Fix test score

Fix test score sum to mean

Add total step to recurrent dqn_agent

Fix test log position

Add framestack argument

Remove upndown environment

Fix no_framestack argument

Add r2d1 resnet configs

Delete lunarlander iqn & Fix R2D1C51 lunarlander config description

Fix configs

Change total_step count startpoint after warmup

Chage test startpoint

Fix epsilon decay

Change r2d1 agent epsilon_decay

Fix several issues commented
@jinPrelude
Rebase commit to a302479
10393f1
@jinPrelude
Rebase commit to 990c78a
22bc2f0
@jinPrelude
Fix several issues commented
f96e963
@jinPrelude
Delete PrioritizedRecurrentReplayBuffer
a5df8f1
@jinPrelude
Resolved issues commented
264e611
@jinPrelude
Fix issues commented
cfd194e
@jinPrelude
Change R2D1Learner parent class & Add descriptions
8e428e0
@jinPrelude
Fix issues commented
5826c01
@jinPrelude
Change descriptions
d58a3b2
@jinPrelude
Delete unnecessary configs
8add670
@jinPrelude
Change descriptions due to the length limit
ce8a112
@jinPrelude
Fix several issues commented
4b9b1b8
@jinPrelude
Fix contiguous issues
1be2c3f
@jinPrelude
Add __init__.py in recurrent
eb231e4
@jinPrelude
Rebase commit to 815a1ca
bb37068
@jinPrelude
Use no grad tensor to select action
4066dbf
Modify documentation
a3725c1
Add torch.no_grad to select action in other class
b7f8201
@jinPrelude
Add R2D1DQN ResNet config
1d3cf20
@jinPrelude jinPrelude added the enhancement New feature or request label Jun 23, 2020
@jinPrelude jinPrelude requested review from MrSyee and cyoon1729 June 23, 2020 01:08
@jinPrelude jinPrelude self-assigned this Jun 23, 2020
@lgtm-com
Copy link

lgtm-com bot commented Jun 23, 2020

This pull request introduces 8 alerts when merging 1d3cf20 into 815a1ca - view on LGTM.com

new alerts:

  • 4 for Mismatch between signature and use of an overridden method
  • 2 for Signature mismatch in overriding method
  • 1 for Module is imported with 'import' and 'import from'
  • 1 for __init__ method calls overridden method

@jinPrelude jinPrelude mentioned this pull request Jun 23, 2020
Closed
@lgtm-com
Copy link

lgtm-com bot commented Jun 23, 2020

This pull request introduces 8 alerts when merging d3a2e59 into 815a1ca - view on LGTM.com

new alerts:

  • 4 for Mismatch between signature and use of an overridden method
  • 2 for Signature mismatch in overriding method
  • 1 for Module is imported with 'import' and 'import from'
  • 1 for __init__ method calls overridden method

@lgtm-com
Copy link

lgtm-com bot commented Jun 23, 2020

This pull request introduces 8 alerts when merging 083a447 into 815a1ca - view on LGTM.com

new alerts:

  • 4 for Mismatch between signature and use of an overridden method
  • 2 for Signature mismatch in overriding method
  • 1 for Module is imported with 'import' and 'import from'
  • 1 for __init__ method calls overridden method

@lgtm-com
Copy link

lgtm-com bot commented Jun 23, 2020

This pull request introduces 8 alerts when merging c3056b3 into 815a1ca - view on LGTM.com

new alerts:

  • 4 for Mismatch between signature and use of an overridden method
  • 2 for Signature mismatch in overriding method
  • 1 for Module is imported with 'import' and 'import from'
  • 1 for __init__ method calls overridden method

MrSyee
MrSyee reviewed Jun 23, 2020
View reviewed changes
@jinPrelude jinPrelude merged commit 07743f6 into master Jun 23, 2020
@lgtm-com
Copy link

lgtm-com bot commented Jun 23, 2020

This pull request introduces 8 alerts when merging abcc34b into 815a1ca - view on LGTM.com

new alerts:

  • 4 for Mismatch between signature and use of an overridden method
  • 2 for Signature mismatch in overriding method
  • 1 for Module is imported with 'import' and 'import from'
  • 1 for __init__ method calls overridden method

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MrSyee MrSyee MrSyee approved these changes

@cyoon1729 cyoon1729 Awaiting requested review from cyoon1729

Assignees

@jinPrelude jinPrelude

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jinPrelude @MrSyee