Skip to content

Commit 18d10e8

Browse files
author
Vincent Moens
committed
Merge remote-tracking branch 'origin/main' into consolidate
2 parents 5fffe1e + eb6c85d commit 18d10e8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+1924
-323
lines changed

.github/unittest/linux/scripts/run_all.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ set -v
99

1010
if [[ $OSTYPE != 'darwin'* ]]; then
1111
apt-get update && apt-get upgrade -y
12-
apt-get install -y vim git wget
12+
apt-get install -y vim git wget libsdl2-dev libsdl2-2.0-0
1313

1414
apt-get install -y libglfw3 libgl1-mesa-glx libosmesa6 libglew-dev
1515
apt-get install -y libglvnd0 libgl1 libglx0 libegl1 libgles2

.github/unittest/linux_libs/scripts_brax/environment.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,5 @@ dependencies:
1717
- pyyaml
1818
- scipy
1919
- hydra-core
20+
- jax[cuda12]
2021
- brax

.github/unittest/linux_libs/scripts_gym/batch_scripts.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ do
5252
echo "Testing gym version: ${GYM_VERSION}"
5353
# handling https://github.com/openai/gym/issues/3202
5454
pip3 install wheel==0.38.4
55+
pip3 install "pip<24.1"
5556
pip3 install gym==$GYM_VERSION
5657
$DIR/run_test.sh
5758

@@ -70,6 +71,7 @@ do
7071

7172
echo "Testing gym version: ${GYM_VERSION}"
7273
pip3 install wheel==0.38.4
74+
pip3 install "pip<24.1"
7375
pip3 install 'gym[atari]'==$GYM_VERSION
7476
pip3 install ale-py==0.7
7577
$DIR/run_test.sh
@@ -88,6 +90,7 @@ do
8890

8991
echo "Testing gym version: ${GYM_VERSION}"
9092
pip3 install 'gym[atari]'==$GYM_VERSION
93+
pip3 install pip -U
9194
$DIR/run_test.sh
9295

9396
# delete the conda copy

.github/unittest/linux_libs/scripts_gym/install.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,9 @@ git submodule sync && git submodule update --init --recursive
3737

3838
printf "Installing PyTorch with %s\n" "${CU_VERSION}"
3939
if [ "${CU_VERSION:-}" == cpu ] ; then
40-
conda install pytorch==2.0 torchvision==0.15 cpuonly -c pytorch
40+
conda install pytorch==2.0 torchvision==0.15 cpuonly -c pytorch -y
4141
else
42-
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
42+
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia -y
4343
fi
4444

4545
# Solving circular import: https://stackoverflow.com/questions/75501048/how-to-fix-attributeerror-partially-initialized-module-charset-normalizer-has

.github/unittest/linux_olddeps/scripts_gym_0_13/install.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,9 @@ git submodule sync && git submodule update --init --recursive
3737

3838
printf "Installing PyTorch with %s\n" "${CU_VERSION}"
3939
if [ "${CU_VERSION:-}" == cpu ] ; then
40-
conda install pytorch==2.0 torchvision==0.15 cpuonly -c pytorch
40+
conda install pytorch==2.0 torchvision==0.15 cpuonly -c pytorch -y
4141
else
42-
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
42+
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia -y
4343
fi
4444

4545
# Solving circular import: https://stackoverflow.com/questions/75501048/how-to-fix-attributeerror-partially-initialized-module-charset-normalizer-has

README.md

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -550,45 +550,59 @@ including `pip3` or other special installation instructions.
550550
**Torchrl**
551551

552552
You can install the **latest stable release** by using
553-
```
553+
```bash
554554
pip3 install torchrl
555555
```
556556
This should work on linux, Windows 10 and OsX (Intel or Silicon chips).
557557
On certain Windows machines (Windows 11), one should install the library locally (see below).
558558

559559
The **nightly build** can be installed via
560-
```
561-
pip install torchrl-nightly
560+
```bash
561+
pip3install torchrl-nightly
562562
```
563563
which we currently only ship for Linux and OsX (Intel) machines.
564564
Importantly, the nightly builds require the nightly builds of PyTorch too.
565565

566566
To install extra dependencies, call
567-
```
567+
```bash
568568
pip3 install "torchrl[atari,dm_control,gym_continuous,rendering,tests,utils,marl,checkpointing]"
569569
```
570570
or a subset of these.
571571

572572
One may also desire to install the library locally. Three main reasons can motivate this:
573573
- the nightly/stable release isn't available for one's platform (eg, Windows 11, nightlies for Apple Silicon etc.);
574574
- contributing to the code;
575-
- install torchrl with a previous version of PyTorch (note that this should also be doable via a regular install followed
576-
by a downgrade to a previous pytorch version -- but the C++ binaries will not be available.)
575+
- install torchrl with a previous version of PyTorch (any version >= 2.0) (note that this should also be doable via a regular install followed
576+
by a downgrade to a previous pytorch version -- but the C++ binaries will not be available so some feature will not work,
577+
such as prioritized replay buffers and the like.)
577578

578579
To install the library locally, start by cloning the repo:
579-
```
580+
```bash
580581
git clone https://github.com/pytorch/rl
581582
```
583+
and don't forget to check out the branch or tag you want to use for the build:
584+
```bash
585+
git checkout v0.4.0
586+
```
582587

583588
Go to the directory where you have cloned the torchrl repo and install it (after
584589
installing `ninja`)
585-
```
590+
```bash
586591
cd /path/to/torchrl/
587-
pip install ninja -U
592+
pip3install ninja -U
588593
python setup.py develop
589594
```
590595

591-
(unfortunately, `pip install -e .` will not work).
596+
One can also build the wheels to distribute to co-workers using
597+
```bash
598+
python setup.py bdist_wheel
599+
```
600+
Your wheels will be stored there `./dist/torchrl<name>.whl` and installable via
601+
```bash
602+
pip install torchrl<name>.whl
603+
```
604+
605+
**Warning**: Unfortunately, `pip3 install -e .` does not currently work. Contributions to help fix this are welcome!
592606

593607
On M1 machines, this should work out-of-the-box with the nightly build of PyTorch.
594608
If the generation of this artifact in MacOs M1 doesn't work correctly or in the execution the message

docs/requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ numpy
33
sphinx-copybutton
44
sphinx-gallery
55
sphinx===5.0.0
6-
Jinja2==3.1.3
6+
Jinja2==3.1.4
77
sphinx-autodoc-typehints
88
sphinx-serve==1.0.1
99
git+https://github.com/vmoens/aafig@4319769eae88fff8e3464858f3cf8c277f35335d

docs/source/reference/data.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -902,6 +902,7 @@ Check the :obj:`torchrl.envs.utils.check_env_specs` method for a sanity check.
902902
UnboundedDiscreteTensorSpec
903903
LazyStackedTensorSpec
904904
LazyStackedCompositeSpec
905+
NonTensorSpec
905906

906907
Reinforcement Learning From Human Feedback (RLHF)
907908
-------------------------------------------------

docs/source/reference/envs.rst

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,9 @@ Each env will have the following attributes:
4646
all the output keys (:obj:`"full_observation_spec"`, :obj:`"full_reward_spec"` and :obj:`"full_done_spec"`).
4747
It is locked and should not be modified directly.
4848

49+
If the environment carries non-tensor data, a :class:`~torchrl.data.NonTensorSpec`
50+
instance can be used.
51+
4952
Importantly, the environment spec shapes should contain the batch size, e.g.
5053
an environment with :obj:`env.batch_size == torch.Size([4])` should have
5154
an :obj:`env.action_spec` with shape :obj:`torch.Size([4, action_size])`.
@@ -765,6 +768,7 @@ to be able to create this other composition:
765768

766769
Transform
767770
TransformedEnv
771+
ActionDiscretizer
768772
ActionMask
769773
AutoResetEnv
770774
AutoResetTransform
@@ -776,17 +780,16 @@ to be able to create this other composition:
776780
CenterCrop
777781
ClipTransform
778782
Compose
783+
DTypeCastTransform
779784
DeviceCastTransform
780785
DiscreteActionProjection
781786
DoubleToFloat
782-
DTypeCastTransform
783787
EndOfLifeTransform
784788
ExcludeTransform
785789
FiniteTensorDictCheck
786790
FlattenObservation
787791
FrameSkipTransform
788792
GrayScale
789-
gSDENoise
790793
InitTracker
791794
KLRewardTransform
792795
NoopResetEnv
@@ -796,13 +799,13 @@ to be able to create this other composition:
796799
PinMemoryTransform
797800
R3MTransform
798801
RandomCropTensorDict
802+
RemoveEmptySpecs
799803
RenameTransform
800804
Resize
805+
Reward2GoTransform
801806
RewardClipping
802807
RewardScaling
803808
RewardSum
804-
Reward2GoTransform
805-
RemoveEmptySpecs
806809
SelectTransform
807810
SignTransform
808811
SqueezeTransform
@@ -812,11 +815,12 @@ to be able to create this other composition:
812815
TimeMaxPool
813816
ToTensorImage
814817
UnsqueezeTransform
815-
VecGymEnvTransform
816-
VecNorm
817818
VC1Transform
818819
VIPRewardTransform
819820
VIPTransform
821+
VecGymEnvTransform
822+
VecNorm
823+
gSDENoise
820824

821825
Environments with masked actions
822826
--------------------------------

sota-implementations/cql/cql_offline.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,8 @@ def main(cfg: "DictConfig"): # noqa: F821
162162

163163
pbar.close()
164164
torchrl_logger.info(f"Training time: {time.time() - start_time}")
165+
if not eval_env.is_closed:
166+
eval_env.close()
165167

166168

167169
if __name__ == "__main__":

0 commit comments

Comments
 (0)