Autopilot pattern for Couchbase

Read the blog post and the Docker Compose yaml to better understand how to use this repo.

This is a Dockerfile, Docker Compose file and shell script that will deploy a Couchbase cluster that can be scaled easily using docker compose scale couchbase=$n.

Prep your environment

1. Get a Joyent account and add your SSH key.
2. Install and configure the Joyent CloudAPI CLI tools.
3. Install Docker and Docker Compose.
4. Configure your Docker CLI and Compose for use with Joyent:

curl -O https://raw.githubusercontent.com/joyent/sdc-docker/master/tools/sdc-docker-setup.sh && chmod +x sdc-docker-setup.sh
 ./sdc-docker-setup.sh -k us-east-1.api.joyent.com <ACCOUNT> ~/.ssh/<PRIVATE_KEY_FILE>

Easy instructions

1. Clone or download this repo.
2. cd into the cloned or downloaded directory.
3. Execute bash start.sh to start everything up.
4. The Couchbase dashboard should automatically open. Sign in with the user/pass printed in the output of bash start.sh to see the working, one-node cluster.
5. Scale the cluster using docker-compose --project-name=ccic scale up couchbase=$n and watch the node(s) join the cluster in the Couchbase dashboard.

Detailed instructions

The start.sh script automatically does the following:

docker-compose pull
docker-compose --timeout=120 --project-name=ccic up -d --no-recreate

Those Docker Compose commands read the docker-compose.yml, which describes the three services in the app. The second command, we can call it docker-compose up for short, provisions a single container for each of the services.

The three services include:

• Couchbase, the database at the core of this application
• Consul, to support service discovery and health checking among the different services
• Couchbase Cloud Benchmarks, a benchmarking container to round out the picture

Consul is running in it's default configuration as delivered in Jeff Lindsay's excellent image, but Couchbase is wrapped with a custom start script that enables the magic here.

Once the first set of containers is running, the start.sh script bootstraps the Couchbase container with the following command:

docker exec -it ccic_couchbase_1 triton-bootstrap bootstrap benchmark

Details of what that command does are described below, but the short story is that it initializes the cluster and creates a bucket, then registers this one node Couchbase service with the Consul container.

Because one Couchbase container can get lonely, it's best to scale it using the following command:

docker-compose --timeout=120 --project-name=ccic scale couchbase=$COUNT

Docker Compose will create new Couchbase containers according to the definition in the docker-compose.yml, and when those containers come online they'll check with Consul to see if there's an established cluster. When they find there is, they'll join that cluster and rebalance the data across the new nodes.

Bootstrapping Couchbase

The Couchbase bootstrap script does the following:

1. Set some environmental variables
2. Wait for the Couchbase daemon to be responsive
3. Check if Couchbase is already configured
   1. The boostrap will exit if so
4. Check if Consul is responsive
   1. The bootstrap will exit if Consul is unreachable
5. Initializes the Couchbase node
6. Check for any arguments passed to the bootstrap script
   1. If the script is manually called with the bootstrap argument, it does the following:
      1. Initializes the Couchbase cluster
      2. Creates a data bucket
   2. Otherwise, it will...
      1. Check Consul for an established Couchbase cluster
      2. Join the cluster
      3. Rebalance the cluster
7. Check that the cluster is healthy.
8. Register the service with Consul

Consul notes

Bootstrapping, Consul clusters, and the details about adding and removing nodes. The CLI and HTTP API are also documented.

Check for registered instances of a named service

curl -v http://consul:8500/v1/catalog/service/couchbase | json -aH ServiceAddress

Register an instance of a service

export MYIP=$(ip addr show eth0 | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}')
curl http://consul:8500/v1/agent/service/register -d "$(printf '{"ID": "couchbase-%s","Name": "couchbase","Address": "%s"}' $MYIP $MYIP)"