Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] New ConnectorV2 API #08: Full multi-agent support. #42373

Closed
wants to merge 35 commits into from
Closed

[RLlib] New ConnectorV2 API #08: Full multi-agent support. #42373

wants to merge 35 commits into from

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Jan 12, 2024 •
edited
Loading

WIP: DO NOT MERGE;

New ConnectorV2 API #8: Full multi-agent support.

TODO:

IMPORTANT NOTE: This PR concludes the feature-completion effort for the new RLlib API stack.

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@sven1977
wip
e7ae52a 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
fe640de 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into env_…
b340ddc 
…runner_support_connectors_04_learner_api_changes
@sven1977
wip
9492b7a 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
merge
52d5e72 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
6437d7e 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
merge
242d40a 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
61be702 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
bf802fc 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fixes
16f2c38 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into env_…
ad047a7 
…runner_support_connectors_04_learner_api_changes
@sven1977
wip
083388d 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into env_…
8e02889 
…runner_support_connectors_04_learner_api_changes
@sven1977
LINT
10b0700 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fix
e439fc8 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fix
4633659 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
cce2c66 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
7dff78f 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fix
bdb20dc 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fix
bcdb92f 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into env_…
03fe431 
…runner_support_connectors_04_learner_api_changes
@sven1977
fix
7dd8f3f 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fix
b769c05 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into env_…
f5ffe83 
…runner_support_connectors_04_learner_api_changes
@sven1977
Merge branch 'env_runner_support_connectors_04_learner_api_changes' i…
0b78c50 
…nto env_runner_support_connectors_05_ppo_w_connectorv2s

# Conflicts:
#	rllib/algorithms/ppo/ppo.py
#	rllib/core/learner/learner.py
#	rllib/utils/minibatch_utils.py
@sven1977
fix
c96c7ae 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
fix bug in single-agent-env-runner that drops all finished episodes f…
7ecc594 
…rom being returned from `_sample_timesteps()`.

Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
merge
13bcb1d 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
merge
d28ee91 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
db16556 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
790a537 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977 sven1977 added do-not-merge Do not merge this PR! rllib RLlib related issues rllib-multi-agent An RLlib multi-agent related problem. rllib-newstack labels Jan 12, 2024
@sven1977
wip
f2a6450 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
d525de3 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
wip
d728a08 
Signed-off-by: sven1977 <svenmika1977@gmail.com>
@sven1977
Merge branch 'env_runner_support_connectors_05_ppo_w_connectorv2s' in…
9da81f6 
…to env_runner_support_connectors_08_multi_agent_support
@sven1977 sven1977 closed this Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@avnishn avnishn Awaiting requested review from avnishn

@ArturNiederfahrenhorst ArturNiederfahrenhorst Awaiting requested review from ArturNiederfahrenhorst

@smorad smorad Awaiting requested review from smorad

@maxpumperla maxpumperla Awaiting requested review from maxpumperla

@kouroshHakha kouroshHakha Awaiting requested review from kouroshHakha

Assignees
No one assigned
Labels
do-not-merge Do not merge this PR! rllib RLlib related issues rllib-multi-agent An RLlib multi-agent related problem. rllib-newstack
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sven1977