Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rllib] Added evaluation script to RLLib #1295

Merged
merged 12 commits into from
Dec 11, 2017
Merged

[rllib] Added evaluation script to RLLib #1295

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c25094b
Added eval script
pschafhalter Dec 6, 2017
ec16345
Added tests for eval script
pschafhalter Dec 6, 2017
d0e1de3
Fixed paths
pschafhalter Dec 6, 2017
0d2012d
Add eval test to run tests script
pschafhalter Dec 6, 2017
ebd9ef3
Fix linting
pschafhalter Dec 6, 2017
938fc9f
Add future imports
pschafhalter Dec 7, 2017
35ddce2
Address PR comments
pschafhalter Dec 7, 2017
a7ed5e2
Fix bug and linting
pschafhalter Dec 8, 2017
d21d236
Fix indent in running test
pschafhalter Dec 8, 2017
b915195
Updated test script to match fixed checkpoints (#1294)
pschafhalter Dec 8, 2017
3e4e1b8
Add documentation for eval.py
pschafhalter Dec 9, 2017
76664e5
Created subsection for eval.py
pschafhalter Dec 11, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added tests for eval script
  • Loading branch information
@pschafhalter
pschafhalter committed Dec 8, 2017
commit ec16345241cd085bc37fd1cab38aacad7d132089
41 changes: 41 additions & 0 deletions test/jenkins_tests/multi_node_tests/test_rllib_eval.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/sh

GYM_ENV='CartPole-v0'

# TODO: Test AC3

# Test for DQN
ALG='DQN'
EXPERIMENT_NAME=$GYM_ENV'_'$ALG
python ~/workspace/ray/python/ray/rllib/train.py --run $ALG --env $GYM_ENV \
--stop '{"training_iteration": 2}' --experiment-name $EXPERIMENT_NAME \
--checkpoint-freq 1

EXPERIMENT_PATH='/tmp/ray/'$EXPERIMENT_NAME
CHECKPOINT_FOLDER=$(ls $EXPERIMENT_PATH)
CHECKPOINT=$EXPERIMENT_PATH'/'$CHECKPOINT_FOLDER'/checkpoint-0'

python ~/workspace/ray/python/ray/rllib/eval.py $CHECKPOINT --run $ALG \
--env $GYM_ENV --hide

# Clean up
rm -rf $EXPERIMENT_PATH

# Test for PPO
ALG='PPO'
EXPERIMENT_NAME=$GYM_ENV'_'$ALG
python ~/workspace/ray/python/ray/rllib/train.py --run $ALG --env $GYM_ENV \
--stop '{"training_iteration": 2}' --experiment-name $EXPERIMENT_NAME \
--checkpoint-freq 1

EXPERIMENT_PATH='/tmp/ray/'$EXPERIMENT_NAME
CHECKPOINT_FOLDER=$(ls $EXPERIMENT_PATH)
CHECKPOINT=$EXPERIMENT_PATH'/'$CHECKPOINT_FOLDER'/checkpoint-1'

echo $CHECKPOINT
python ~/workspace/ray/python/ray/rllib/eval.py $CHECKPOINT --run $ALG \
--env $GYM_ENV --hide

# Clean up
rm -rf $EXPERIMENT_PATH