Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
f651e7f
added daemon=True for multi-process rollout, policy manager and infer…
Oct 21, 2021
6de5582
removed obsolete files
Oct 21, 2021
3e38eb7
[REDO][PR#406]V0.2 rl refinement taskq (#408)
buptchan Oct 29, 2021
7e226cc
RL policy redesign (V2) (#405)
lihuoran Nov 1, 2021
f0db8b5
refined messy workflow code
Nov 1, 2021
e8deb75
merged 'scenario_dir' and 'scenario' in rl config
Nov 1, 2021
bc58b72
1. refined env_sampler and agent_wrapper code; 2. added docstrings fo…
Nov 3, 2021
7b449eb
1. temporarily renamed RLPolicy from polivy_v2 to RLPolicyV2; 2. merg…
Nov 3, 2021
c75e387
merged cim and cim_v2
Nov 3, 2021
f624fc8
lint issue fix
Nov 3, 2021
62e2b01
refined logging logic
Nov 5, 2021
148164e
lint issue fix
Nov 5, 2021
169262a
reversed unwanted changes
Nov 5, 2021
7eb89e6
.
lihuoran Nov 7, 2021
b478e99
LearnerManager
lihuoran Nov 7, 2021
b3f50ec
Use batch to transfer data & add SHAPE_CHECK_FLAG
lihuoran Nov 8, 2021
b0de228
Rename learner to trainer
lihuoran Nov 8, 2021
c1e8653
Add property for policy._is_exploring
lihuoran Nov 8, 2021
e3a811a
CIM test scenario for V3. Manual test passed. Next step: run it, make…
lihuoran Nov 9, 2021
98ceacf
env_sampler.py could run
lihuoran Nov 9, 2021
dc8f279
env_sampler refine on the way
lihuoran Nov 9, 2021
4c51575
First runnable version done
lihuoran Nov 10, 2021
84ad7f0
AC could run, but the result is bad. Need to check the logic
lihuoran Nov 10, 2021
3a4f78e
Refine abstract method & shape check error info.
lihuoran Nov 11, 2021
65bc88b
Docs
lihuoran Nov 11, 2021
3f10fd7
Very detailed compare. Try again.
lihuoran Nov 11, 2021
5192868
AC done
lihuoran Nov 11, 2021
eaced75
DQN check done
lihuoran Nov 12, 2021
5faa8ff
Minor
lihuoran Nov 12, 2021
27d9371
DDPG, not tested
lihuoran Nov 12, 2021
5769369
Minors
lihuoran Nov 12, 2021
eaee7fa
A rough draft of MAAC
lihuoran Nov 12, 2021
bf37ab3
Cannot use CIM as the multi-agent scenario.
lihuoran Nov 14, 2021
6cb42d3
Minor
lihuoran Nov 14, 2021
545521f
MAAC refinement on the way
lihuoran Nov 15, 2021
30e6aea
Remove ActionWithAux
lihuoran Nov 15, 2021
0823933
Refine batch & memory
lihuoran Nov 15, 2021
85062ff
MAAC example works
lihuoran Nov 15, 2021
fd09abf
Reproduce-able fix. Policy share between env_sampler and trainer_mana…
lihuoran Nov 15, 2021
b2c4c97
Detail refinement
lihuoran Nov 16, 2021
ce696d5
Simplify the user configed workflow
lihuoran Nov 16, 2021
67ce3f2
Minor
lihuoran Nov 16, 2021
5b57bea
Refine example codes
lihuoran Nov 16, 2021
1fc1c3d
Minor polishment
lihuoran Nov 17, 2021
f9c66bb
Migrate rollout_manager to V3
lihuoran Nov 17, 2021
dc408c5
Error on the way
lihuoran Nov 17, 2021
990dfc3
Redesign torch.device management
lihuoran Nov 18, 2021
8e47b8e
Rl v3 maddpg (#418)
buptchan Dec 1, 2021
8e4ad49
Preparation for data parallel (#420)
lihuoran Dec 21, 2021
f80dcc3
[DRAFT] distributed training pipeline based on RL Toolkit V3 (#450)
ysqyang Jan 5, 2022
9634ba4
Miner bug fix
lihuoran Jan 5, 2022
8ecbcea
Coroutine-related bug fix ("get_policy_state") (#452)
ysqyang Jan 6, 2022
dc2b9ab
Quick fix
lihuoran Jan 6, 2022
0bdc230
Redesign experience recording logic (#453)
lihuoran Jan 10, 2022
1610a3f
Calculating advantages / returns (#454)
lihuoran Jan 11, 2022
4428151
Rl v3 hanging issue fix (#455)
ysqyang Jan 14, 2022
d2e69d4
Rl v3 parallel rollout (#457)
ysqyang Jan 20, 2022
6634e8f
MADDPG performance bug fix (#459)
lihuoran Jan 21, 2022
d7093b3
Rl v3 example update (#461)
ysqyang Jan 25, 2022
4faa8f1
Done (#462)
lihuoran Jan 27, 2022
680bb52
Rl v3 load save (#463)
ysqyang Jan 27, 2022
f77d052
RL Toolkit data parallelism revamp & config utils (#464)
ysqyang Feb 10, 2022
101baeb
RL doc string (#465)
lihuoran Feb 24, 2022
7b366b1
RL online doc (#469)
lihuoran Mar 3, 2022
79e7dd7
Merge branch 'v0.3' into rl_v3
lihuoran Mar 3, 2022
3cbca27
Rl v3 pr comment resolution (#474)
ysqyang Mar 7, 2022
7f1713b
Merge branch 'v0.3' into rl_v3
lihuoran Mar 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,5 @@ data/
maro_venv/
pyvenv.cfg
htmlcov/
*supply_chain_*/
examples/supply_chain/docker-compose.yml
.coverage
.coveragerc
27 changes: 0 additions & 27 deletions docker-compose.yml

This file was deleted.

9 changes: 3 additions & 6 deletions docker_files/dev.df
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM ubuntu:18.04
FROM python:3.7-buster
WORKDIR /maro

# Install Apt packages
Expand All @@ -9,11 +9,11 @@ RUN apt-get install -y gcc
RUN apt-get install -y libcurl4 libcurl4-openssl-dev libssl-dev curl
RUN apt-get install -y libzmq3-dev
RUN apt-get install -y python3-pip
RUN apt-get install -y python3-dev libpython3.6-dev python-numpy
RUN apt-get install -y python3-dev libpython3.7-dev python-numpy
RUN rm -rf /var/lib/apt/lists/*

# Install Python packages
RUN pip3 install --upgrade pip
RUN pip install --upgrade pip
RUN pip install --no-cache-dir Cython==0.29.14
RUN pip install --no-cache-dir pyaml==20.4.0
RUN pip install --no-cache-dir pyzmq==19.0.2
Expand All @@ -31,9 +31,6 @@ COPY setup.py /maro/
RUN bash /maro/scripts/install_maro.sh
RUN pip cache purge

RUN rm -r /maro/maro/rl
RUN rm -r /maro/maro/simulator/scenarios/supply_chain

ENV PYTHONPATH=/maro

CMD ["/bin/bash"]
Loading