README

Note

This repository has been archived. This setup required changes on every major JupyterHub release and was broken again when Notebook v7 arrived. Our currently maintained workshop environment is built on top of The Littlest JupyterHub (tljh) and should be available here: https://github.com/GoSecure/jupyterhub-workshop-environment

Opionated JupyterHub deployment for workshops relying on GitHub for Authentication

Features:

• Supports JupyterLab and Jupyter Notebook (Python and Terminals)

• Read-only shared files to all users (in users' homes)

• Users have their own copies of notebook, created on first login from template directory

• Each user is in its own container which can be customized

• Optional: Expose the users' containers on the host’s network (enables connect-back or XXE workflows)

• Server deployed and managed by Terraform

Forked from https://github.com/jupyterhub/jupyterhub-deploy-docker in 2017

Setup Instructions

Prepare for Deployment

These steps should be done locally before provisioning the JupterHub server to facilitate deployment.

Setup GitHub Authentication

Create a GitHub application reflecting the purpose of your JupyterHub workshop. It will be used to allow users to login and gain access to the workshop servers. You should know the following information ahead of time:

• Workshop Name (Application Name)

• Workshop Server URL (Doesn’t need to resolve yet)

• Workshop Description

• Workshop Icon if desired.

Follow the instructions under the Authenticator setup section of README.upstream.md and create secrets/oauth.env with the following content:

GITHUB_CLIENT_ID=<github_client_id>
GITHUB_CLIENT_SECRET=<github_client_secret>
OAUTH_CALLBACK_URL=https://<myhost.mydomain>/hub/oauth_callback

NOTE: Applications for workshops should be created under your organization at https://github.com/organizations/<orgname>/settings/applications/new

Ask organisation admin to create the application with the required information.

By default our setup is open to anyone to create an account like for an open workshop with no pre-registrations (different options available below). Setup the users with admin privileges in advance by writing the github usernames to the secrets/admins file (one per line).

Setup Administrator Accounts

Add one or more accounts in secrets/admins. These accounts will be granted full access to the JupyterHub.

Workshop Files: Overlay Filesystem

There are two ways to expose files to the workshop attendees.

Read-only shared with all participants

All users will have access to a directory in their own homes (/home/jovyan/) called workshop/ that will be mapped to the servers' /srv/workshop/ via a docker volume.

Changes made in /srv/workshop/* will be reflected instantly for all participants.

Files for participants (writable and executable)

On container creation, a directory called labs/ will be created and populated from the servers' /srv/workshop/labs-source/ content.

This will happen only once per user. If you need to re-populate it, you need to delete the user' files in his volume. These are visible under: /var/lib/docker/volumes/jupyterhub-user-<username>/_data/

Deploying the Server

Ensure that you have a recent version of Terraform installed and get it ready:

terraform init

Configure your Digital Ocean credentials and other settings in terraform.tfvars:

# the following values are used as tags in digital ocean so only lowercase, dash and underscore allowed
workshop_name = "<workshop-name>"
tag_owner = "<username>"
tag_event = "<event-name>"
do_token = "<your DO token>"
# default is small 1 CPU and 2GB RAM
#instance_size = "c-4"

Spawn and provision the droplet. It might take a while to setup and provision, so be patient. The repository should be synchronized to /srv

terraform validate
terraform plan
terraform apply

Server-Side Configuration

To connect to the server via SSH, run:

./bin/ssh-connect.sh

Build Notebook Image

If you are using a custom notebook, you need to build it first and rebuild the jupyterhub-user image. The base image that jupyterhub-user will use is configured with the DOCKER_NOTEBOOK_IMAGE key in .env.

cd examples/custom-notebook-server/
# <change what you need or use another example>
docker build -t workshop-notebook .
cd ../../
make notebook_image

Configure TLS and HTTPS with Certbot

• Ensure that your domain name resolves to the created droplet public address.

• Create certificates using certbot (make sure docker-compose is not running)

  certbot certonly --standalone -d <domain-name>
  cp /etc/letsencrypt/live/<domain-name>/fullchain.pem secrets/jupyterhub.crt
  cp /etc/letsencrypt/live/<domain-name>/privkey.pem secrets/jupyterhub.key

Launch the Jupyter Server

After everything is configured, you can spawn the jupyter server with the following commands:

cd /srv/jupyterhub/
make
docker-compose up

Whenever changing the secrets or environment variables, rebuilding and relaunching the images is required:

docker-compose down
make
docker-compose up

Components

Once authenticated the user has access to a JupyterLab environment by default. A Jupyter Notebook environment is available by changing the URL from /user/<username>/lab to /user/<username>/tree.

The administrative interface is available at /hub/admin for users who are allowed to use it.

Optional: Only allow specific users

If you create a secrets/users file with a list of allowed GitHub usernames, only these usernames will be able to sign-up to the server.

When you change that list, you need to restart the JupyterHub container. Easiest way to achieve this is to kill the docker-compose environment, run make and restart docker-compose.

<ctrl-d>
make
docker-compose up

User lists can optionally have an admin tag to identify admin users. Ex:

linus admin
obilodeau
masarah admin

Optional: Stop Open Registrations

Optional: User Instances Exposed to the Internet

Uncomment the following line in the .env file and make sure to rebuild the hub container.

DOCKER_NOTEBOOK_EXPOSE_NETWORK=true

Please double check that secrets/context.env has the correct HOST_IP in it. The Makefile is a little bit brittle in that regard.

Operational Procedures

Add a user to a running instance

Add GitHub username to secrets/users. Restart the jupyterhub with:

docker-compose up -d --build hub

Upgrading JupyterHub

Read the changelog and the upgrade notes.

Tell the users to save and shutdown their notebooks and that they could lose data if they don’t. Shutdown all notebooks from the JupyterHub admin interface. Shutdown all containers with docker-compose down.

Make sure to backup the volumes (/var/lib/docker/volumes/jupyter*) and your docker-compose directory just in case.

Change the JUPYTERHUB_VERSION in .env and adjust your jupyterhub_config.py if needed.

Rebuild your user container (usually one in examples/), follow README.md instructions. Make sure to do a make notebook_image to create the jupyterhub wrapper container for it.

Rebuild the JupyterHub container: docker-compose build hub

If a database schema change is required (changelog should tell), run: docker-compose run hub jupyterhub upgrade-db

Then start the environment as usual with: docker-compose up

Debugging

Logging

In jupyter_config.py, setting log_level to 'DEBUG' will give more information in the docker-compose logs:

c.JupyterHub.log_level = 'DEBUG'

Troubleshooting User Container Failure

If you get strange errors when the user container tries to start, make sure that it can start properly. Errors can be as obscure as: "traitlets.traitlets.TraitError: The 'ip' trait of a Server instance must be a unicode string, but a value of None <class 'NoneType'> was specified". Disable the container removal and inspect the failed container logs with the following steps:

• Disable container removal by making sure that the DockerSpawner is configured properly. In jupyter_config.py ensure that this configuration is present (and not overridden after):

  c.DockerSpawner.remove = False

• Load the new config:

  docker-compose down
  make
  docker-compose up

• Upon container creation failure now you can see the containers being left dangling:

  docker ps
  docker logs jupyter-obilodeau