Skip to content

Latest commit

 

History

History
59 lines (46 loc) · 4.42 KB

README.md

File metadata and controls

59 lines (46 loc) · 4.42 KB
Raw

Reinforcement Learning: An Introduction

Python code for Sutton & Barto's book Reinforcement Learning: An Introduction (2nd Edition)

Contents

Click to view the sample output

Chapter 1

  1. Tic-Tac-Toe

Chapter 2

  1. Figure 2.2: Average performance of epsilon-greedy action-value methods on the 10-armed testbed
  2. Figure 2.3: Optimistic initial action-value estimates
  3. Figure 2.4: Average performance of UCB action selection on the 10-armed testbed
  4. Figure 2.5: Average performance of the gradient bandit algorithm

Chapter 3

  1. Figure 3.5: Grid example with random policy
  2. Figure 3.8: Optimal solutions to the gridworld example

Chapter 4

  1. Figure 4.1: Convergence of iterative policy evaluation on a small gridworld
  2. Figure 4.2: Jack’s car rental problem
  3. Figure 4.3: The solution to the gambler’s problem

Chapter 5

  1. Figure 5.1: Approximate state-value functions for the blackjack policy
  2. Figure 5.4: Weighted importance sampling
  3. Figure 5.5: Ordinary importance sampling with surprisingly unstable estimates

Chapter 6

  1. Figure 6.2: Random walk
  2. Figure 6.3: Batch updating
  3. Figure 6.4: Sarsa applied to windy grid world
  4. Figure 6.5: The cliff-walking task
  5. Figure 6.7: Interim and asymptotic performance of TD control methods
  6. Figure 6.8: Comparison of Q-learning and Double Q-learning

Chapter 7

  1. Figure 7.2: Performance of n-step TD methods on 19-state random walk

Chapter 8

  1. Figure 8.3: Average learning curves for Dyna-Q agents varying in their number of planning steps
  2. Figure 8.5: Average performance of Dyna agents on a blocking task
  3. Figure 8.6: Average performance of Dyna agents on a shortcut task
  4. Figure 8.7: Prioritized sweeping significantly shortens learning time on the Dyna maze task

Chapter 9

  1. Figure 9.1: Gradient Monte Carlo algorithm on the 1000-state random walk task
  2. Figure 9.2: Semi-gradient n-steps TD algorithm on the 1000-state random walk task
  3. Figure 9.5: Fourier basis vs polynomials on the 1000-state random walk task
  4. Figure 9.8: Example of feature width’s effect on initial generalization and asymptotic accuracy
  5. Figure 9.10: Single tiling and multiple tilings on the 1000-state random walk task

Environment

  • Python 2.7
  • Numpy
  • Matplotlib