Skip to content
This repository has been archived by the owner on Sep 6, 2024. It is now read-only.

Spirals-Team/docker-g5k

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Build Status

docker-g5k

A tool to create a Docker Swarm cluster for Docker Machine on the Grid5000 testbed infrastructure.

Requirements

You need a Grid5000 account to use this tool. See this page to create an account.

VPN

You need to be connected to the Grid5000 VPN to create and access your Docker nodes.
Do not forget to configure your DNS or use OpenVPN DNS auto-configuration.
Please follow the instructions from the Grid5000 Wiki.

Installation

Installation from GitHub releases

Binary releases for Linux, MacOS and Windows using x86/x86_64 CPU architectures are available in the releases page.
You can use the following commands to install or upgrade the tool:

# download the binary for your OS and CPU architecture :
sudo curl -L -o /usr/local/bin/docker-g5k "<link to release>"

# grant execution rigths for everyone :
sudo chmod 755 /usr/local/bin/docker-g5k

Installation from sources

This procedure was only tested on Ubuntu 16.04 and Fedora 25

To use the Go tools, you need to set your GOPATH variable environment.
To get the code and compile the binary, run:

go get -u github.com/Spirals-Team/docker-g5k

Then, either put the 'docker-g5k' binary in a directory filled in your PATH environment variable, or run:

export PATH=$PATH:$GOPATH/bin

How to use

Command line flags

Flags marked with [ ] can be given multiple times and the values will be added.

Flags marked with { } support brace expansion (same format as sh/bash shells) to generate combinations.
Please refer to the flags format in "Flags usage" section of the command.

For create-cluster command

Flags description
  • --g5k-username : Your Grid5000 account username (required)
  • --g5k-password : Your Grid5000 account password (required)
  • --g5k-reserve-nodes : Reserve nodes on a site (required)
  • --g5k-walltime : Timelife of the nodes (format: "hh:mm:ss")
  • --g5k-image : Name of the image to deploy on the nodes
  • --g5k-resource-properties : Resource selection with OAR properties (SQL format)
  • --engine-install-url : Custom URL to use for Docker engine installation
  • --engine-opt : Specify flags to include on the selected node(s) engine
  • --engine-label : Specify labels for the selected node(s) engine
  • --swarm-master : Select node(s) to be promoted to Swarm Master
  • --swarm-mode-enable : Create a Swarm mode cluster
  • --swarm-standalone-enable : Create a Swarm standalone cluster
  • --swarm-standalone-discovery : Discovery service to use with Swarm
  • --swarm-standalone-image : Specify the Docker image to use for Swarm
  • --swarm-standalone-strategy : Define a default scheduling strategy for Swarm
  • --swarm-standalone-opt : Define arbitrary global flags for Swarm master
  • --swarm-standalone-join-opt : Define arbitrary global flags for Swarm join
  • --weave-networking : Use Weave for networking (Only with Swarm standalone)
Flags usage
Option Environment Default value { } [ ]
--g5k-username G5K_USERNAME No No
--g5k-password G5K_PASSWORD No No
--g5k-reserve-nodes G5K_RESERVE_NODES Yes Yes
--g5k-walltime G5K_WALLTIME "1:00:00" No No
--g5k-image G5K_IMAGE "jessie-x64-min" No No
--g5k-resource-properties G5K_RESOURCE_PROPERTIES No No
--engine-install-url ENGINE_INSTALL_URL "https://get.docker.com" No No
--engine-opt ENGINE_OPT Yes Yes
--engine-label ENGINE_LABEL Yes Yes
--swarm-master SWARM_MASTER Yes Yes
--swarm-mode-enable SWARM_MODE_ENABLE No No
--swarm-standalone-enable SWARM_STANDALONE_ENABLE No No
--swarm-standalone-discovery SWARM_STANDALONE_DISCOVERY Deploy a local ZooKeeper No No
--swarm-standalone-image SWARM_STANDALONE_IMAGE "swarm:latest" No No
--swarm-standalone-strategy SWARM_STANDALONE_STRATEGY "spread" No No
--swarm-standalone-opt SWARM_STANDALONE_OPT No Yes
--swarm-standalone-join-opt SWARM_STANDALONE_JOIN_OPT No Yes
--weave-networking WEAVE_NETWORKING No No

Flag --g5k-reserve-nodes format is site:numberOfNodes and brace expansion are supported.
For example, lille-16, {lille,nantes}-16.

Engine flags --engine-opt and --engine-label format is node-name:key=val and brace expansion are supported.
For example, lille-0:mykey=myval, lille-{0..5}:mykey=myval, lille-{0,2,4}:mykey=myval.

For --engine-opt flag, please refer to Docker documentation for supported parameters.
Test your parameters on a single node before deploying a cluster ! If your flags are incorrect, Docker wont start and you should redeploy the entire cluster !

For list-cluster command

This command does not take any arguments, and will print jobs reservations in the following form:

JOB ID    NUMBER OF MACHINE(S)   MACHINE(S) NAME
111111    3                      luxembourg-0, luxembourg-1, luxembourg-2
222222    5                      lyon-0, lyon-1, lyon-2, lyon-3, lyon-4
333333    1                      test-lille

For remove-cluster command

Flags description
  • --no-confirm : Disable confirmation before removing machines
Flags usage
Option Environment Default value { } [ ]
--no-confirm G5K_RM_NO_CONFIRM False No Yes

Examples

Cluster creation

An example of a 16 nodes Docker reservation (Only Docker, Swarm is not configured):

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "lille:16" \

An example of a 16 nodes Docker reservation with Engine options and labels:

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "lille:16" \
--engine-opt "lille-{0..15}:graph=/tmp" \
--engine-label "lille-0:mylabelname1=mylabelvalue1" \
--engine-label "lille-{1..5}:mylabelname2=mylabelvalue2"

An example of a 16 nodes Docker reservation using resource properties (nodes with more thant 8GB of RAM and at least 4 CPU cores):

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "lille:16" \
--g5k-resource-properties "memnode > 8192 and cpucore >= 4"

An example of multi-sites cluster creation:

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "lille:16" \
--g5k-reserve-nodes "nantes:8"

An example of multi-sites cluster creation using brace expansion:

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "{lille,nantes}:16"

An example of a 16 nodes Docker Swarm mode cluster creation using the first 3 nodes as Swarm Master:

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "lille:16" \
--swarm-mode-enable \
--swarm-master "lille-{0..2}"

An example of a 16 nodes Docker Swarm standalone cluster creation using the first node as Swarm Master and Weave Networking:

docker-g5k create-cluster \
--g5k-username "user" \
--g5k-password "********" \
--g5k-reserve-nodes "lille:16" \
--swarm-standalone-enable \
--swarm-master "lille-0" \
--weave-networking

Cluster deletion

An example of deleting only nodes related to a job ID:

docker-g5k remove-cluster 1234

An example of deleting only nodes related to a job ID without confirmation:

docker-g5k remove-cluster --no-confirm 1234

An example of multi-jobs deletion:

docker-g5k remove-cluster 1234 5678 9012

Normal use

After creating a cluster, you should be able to use it with usual Docker Machine commands.
By example, to list all nodes in the cluster :

docker-machine ls

If you remove a node with Docker Machine 'rm' command, the job will be deleted and ALL nodes related to this job will become unavailable

Use with Weave networking (Only with Swarm standalone)

First, you need to configure your Docker client to use the Swarm mode (You can get the Swarm master hostname with 'docker-machine ls'):

eval $(docker-machine env --swarm swarm-master-node-name)

Then run a container using Weave networking:

docker run --net=weave -h foo.weave.local --name foo --dns=172.17.0.1 --dns-search=weave.local. -td your-image:version

Your containers can now communicate with each other using theirs short ('foo') or long ('foo.weave.local') name.
The name used NEED to be the one given in parameter '-h'. The name of the container (parameter '--name') is not used by Weave.