Skip to content

Commit

Permalink
Describe --no-instr and --no-mem params
Browse files Browse the repository at this point in the history
  • Loading branch information
lcswillems committed Aug 10, 2018
1 parent aecdbf8 commit 8a8d6ff
Showing 1 changed file with 13 additions and 5 deletions.
18 changes: 13 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,18 +89,24 @@ An example of use of `torch_rl.DictList` and an example of a `preprocess_obss` f
export OMP_NUM_THREADS=1
```

For your own purposes, you will probabily need to change:
- the model in `model.py`,
- the `ObssPreprocessor.__call__` method in `utils.format`.

## `model.py`

Along with the `torch_rl` package is provided a model that:
- has a memory. This can be disabled by setting `use_memory` to `False` in the constructor.
- understands instructions. This can be disabled by setting `use_instr` to `False` in the constructor.

## `scripts`

Along with the `torch_rl` package are provided 3 general reinforcement learning scripts:
- `train.py` for training an actor-critic model with A2C or PPO.
- `enjoy.py` for visualizing your trained model acting.
- `evaluate.py` for evaluating the performances of your trained model over X episodes.

For your own purposes, you will probabily need to change:
- the model in `model.py`,
- the `ObssPreprocessor.__call__` method in `utils.format`.

They were designed especially for the [MiniGrid environments](https://github.com/maximecb/gym-minigrid). These environments give an observation containing an image and a textual instruction to the agent and a reward of 1 if it successfully executes the instruction, 0 otherwise. They are used in what follows for illustrating purposes.
These scripts were designed especially for the [MiniGrid environments](https://github.com/maximecb/gym-minigrid). These environments give an observation containing an image and a textual instruction to the agent and a reward of 1 if it successfully executes the instruction, 0 otherwise. They are used in what follows for illustrating purposes.

These scripts assume that you have already installed the `gym` package (with `pip3 install gym` for example). By default, models and logs are stored in the `storage` folder. You can define a different folder in the environment variable `TORCH_RL_STORAGE`.

Expand All @@ -115,6 +121,8 @@ These scripts assume that you have already installed the `gym` package (with `pi
and a bunch of optional arguments are available among which:
- `--model MODEL`: name of the model, used for loading and saving it. If not specified, it is the `_`-concatenation of the environment name and algorithm name.
- `--frames-per-proc FRAMES_PER_PROC`: number of frames per process before updating parameters.
- `--no-instr`: disable the understanding of instructions of the original model in `model.py`. If your model is trained on an environment where there is no need to understand instructions, it is advised to disable it for faster training.
- `--no-mem`: disable the memory of the original model in `model.py`. If your model is trained on an environment where there is no need to remember something, it is advised to disable it for faster training.
- ... (see more using `--help`)

Here is an example of command:
Expand Down

0 comments on commit 8a8d6ff

Please sign in to comment.