README.md

Parametric Q learning to solve the Cart Pole

There exists everywhere a medium in things, determined by equilibrium.

-- Dmitri Mendeleev

Table of Contents

• Welcome 🤗
• Lecture transcripts
• Quick setup
• Notebooks
• Let's connect

Welcome 🤗

In today's lecture we enter new territory...

A territory where function approximation (aka supervised machine learning) meets good old Reinforcement Learning.

And this is how Deep RL is born.

We will solve the Cart Pole environment of OpenAI using parametric Q-learning.

Today's lesson is split into 3 parts.

Lecture transcripts

📝 1. Parametric Q learning
📝 2. Deep Q learning
📝 3. Hyperparameter search

Quick setup

Make sure you have Python >= 3.7. Otherwise, update it.

1. Pull the code from GitHub and cd into the 01_taxi folder:

   $ git clone https://github.com/Paulescu/hands-on-rl.git
   $ cd hands-on-rl/01_taxi
   

2. Make sure you have the virtualenv tool in your Python installation

   $ pip3 install virtualenv
   

3. Create a virtual environment and activate it.

   $ virtualenv -p python3 venv
   $ source venv/bin/activate
   

   From this point onwards commands run inside the virtual environment.

4. Install dependencies and code from src folder in editable mode, so you can experiment with the code.

   $ (venv) pip install -r requirements.txt
   $ (venv) export PYTHONPATH="."
   

5. Open the notebooks, either with good old Jupyter or Jupyter lab

   $ (venv) jupyter notebook
   

   $ (venv) jupyter lab
   

   If both launch commands fail, try these:

   $ (venv) jupyter notebook --NotebookApp.use_redirect_file=False
   

   $ (venv) jupyter lab --NotebookApp.use_redirect_file=False
   

6. Play and learn. And do the homework 😉.

Notebooks

Parametric Q-learning

• Explore the environment
• Random agent baseline
• Linear Q agent with bad hyper-parameters
• Linear Q agent with good hyper-parameters
• Homework

Deep Q-learning

• Crash course on neural networks
• Deep Q agent with bad hyper-parameters
• Deep Q agent with good hyper-parameters
• Homework

Hyperparameter search

• Hyperparameter search
• Homework

Let's connect!

Do you wanna become a PRO in Machine Learning?

👉🏽 Subscribe to the datamachines newsletter.

👉🏽 Follow me on Medium.