Update wandb to 0.13.11

[pseudo-rnd-thoughts]

Description

The indention of following code is inconsistent between files, I am assuming that ddpg_contiuous_action is correct and a fixing these but it might be the other way around

    if args.track and args.capture_video:
        for filename in os.listdir(f"videos/{run_name}"):
            if filename not in video_filenames and filename.endswith(".mp4"):
                wandb.log({f"videos": wandb.Video(f"videos/{run_name}/{filename}")})
                video_filenames.add(filename)

Types of changes

• Bug fix
• New feature
• New algorithm
• Documentation

Checklist:

• I've read the CONTRIBUTION guide (required).
• I have ensured pre-commit run --all-files passes (required).
• I have updated the tests accordingly (if applicable).
• I have updated the documentation and previewed the changes via mkdocs serve.
  • I have explained note-worthy implementation details.
  • I have explained the logged metrics.
  • I have added links to the original paper and related papers.

If you need to run benchmark experiments for a performance-impacting changes:

• I have contacted @vwxyzjn to obtain access to the openrlbenchmark W&B team.
• I have used the benchmark utility to submit the tracked experiments to the openrlbenchmark/cleanrl W&B project, optionally with --capture-video.
• I have performed RLops with python -m openrlbenchmark.rlops.
  • For new feature or bug fix:
    • I have used the RLops utility to understand the performance impact of the changes and confirmed there is no regression.
  • For new algorithm:
    • I have created a table comparing my results against those from reputable sources (i.e., the original paper or other reference implementation).
  • I have added the learning curves generated by the python -m openrlbenchmark.rlops utility to the documentation.
  • I have added links to the tracked experiments in W&B, generated by python -m openrlbenchmark.rlops ....your_args... --report, to the documentation.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
cleanrl	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 15, 2023 10:47am

[vwxyzjn]

This should be at the beginning of the training loop

[pseudo-rnd-thoughts]

@vwxyzjn I realised that the easier solution is to update wandb to 0.13.11 with wandb/wandb#5008 to support gymnasium rather than adding the code to each of the implementations

[vwxyzjn]

@vwxyzjn I realised that the easier solution is to update wandb to 0.13.11 with wandb/wandb#5008 to support gymnasium rather than adding the code to each of the implementations

It probably won’t work because we have both gymnasium and gym installed.

[sdpkjc]

cleanrl/requirements/requirements-atari.txt

Line 73 in e19f858

wandb==0.13.11 ; python_full_version >= "3.7.1" and python_version < "3.10"

In fact, our code is already using wandb==0.13.11, we need to update it officially in the pyproject.toml file and run poetry lock.

cleanrl/cleanrl/dqn.py

Line 137 in e19f858

monitor_gym=True,

Current dqn.py has updated the way it records videos, and it still works. (I've tested it.) Because wandb preferentially uses the gymnasium library.

However, the current part of the code that uses gym may have bugs due to wandb updates, which may be fixed by setting _gym_version_lt_0_26=True or creating a PR in wandb.

https://github.com/wandb/wandb/blob/134cf1c69200b7211167a50455e32642e1089fcf/wandb/integration/gym/__init__.py#L43-L86

[vwxyzjn]

@sdpkjc thanks for the comment. @pseudo-rnd-thoughts I see what’s the issue. Currently, because wandb version in the lock file is already 0.13.11 and its monitor_gym will work for gymnasium scripts. I also confirmed that dqn.py works https://wandb.ai/costa-huang/cleanRL/runs/mco6owqv?workspace=user-costa-huang.

However, ppo.py with --capture-video no longer works, because it still uses gym and now monitor_gym only works with gymnasium.

Let's just ignore the capture video problem in ppo.py and continue our gymnasium migration. @pseudo-rnd-thoughts can you update the lock file in this PR with poetry lock --no-update? We can merge after that.

[vwxyzjn]

This also LGTM. Feel free to merge after #386. Thanks so much @pseudo-rnd-thoughts