Adds checkpoint frequencies for serial and batch Agents.

[prabhatnagarajan]

We may want to checkpoint intermediate networks during training. This PR modifies one example, and the experiments API to allow that for serial agents.

One use case of this update could be T-REX (https://arxiv.org/abs/1904.06387), which generates synthetic demonstrations by taking different checkpoints of a neural network and using them to generate demonstrations, which may be ranked according to the checkpoint order.

[prabhatnagarajan]

Testing:

I ran python train_ppo_ale.py --gpu -1 --checkpoint-frequency 20 --steps 100

and got the following output:

INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/40_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/60_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/80_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/100_checkpoint
INFO:chainerrl.experiments.train_agent_batch:Saved the agent to results/20190820T214258.089467/104_finish

I also checked some of the directories, and they indeed exist.

[prabhatnagarajan]

I also ran on train_dqn_ale.py. I ran the following:

python examples/atari/train_dqn_ale.py --gpu -1 --checkpoint-frequency 20 --steps 100

with the following output:

INFO:chainerrl.experiments.train_agent:Saved the agent to results/20190820T214138.012278/40_checkpoint
INFO:chainerrl.experiments.train_agent:Saved the agent to results/20190820T214138.012278/60_checkpoint
INFO:chainerrl.experiments.train_agent:Saved the agent to results/20190820T214138.012278/80_checkpoint
INFO:chainerrl.experiments.train_agent:outdir:results/20190820T214138.012278 step:100 episode:0 R:2.0```

I checked that the directories were created, and that there were files inside them.

[ummavi]

/test

[pfn-ci-bot]

Successfully created a job for commit 649b3c1:

• Dashboard for commit 649b3c1