Updating docs to show consul auto join and nomad automatic bootstrap

docs/bootstrap-nomad-google-compute-engine.md

@@ -32,7 +32,13 @@ cd envoy-consul-sds
 
 ## Bootstrap a Nomad Cluster
 
-First we will install Nomad, Consul and dnsmasq
+In this section we will create `nomad-1`, `nomad-2` and `nomad-3` instances.
+
+First we will install Consul on them and [auto join](https://www.consul.io/docs/agent/options.html#google-compute-engine) it using GCE metadata.
+
+Nomad will then [auto bootstrap](https://www.nomadproject.io/guides/cluster/automatic.html) itself using existing Consul cluster.
+
+We will also install `dnsmasq` in order to use Consul DNS interface.
 
 ```bash
 gcloud compute instances create nomad-1 nomad-2 nomad-3 \
@@ -41,6 +47,8 @@ gcloud compute instances create nomad-1 nomad-2 nomad-3 \
   --boot-disk-size 300GB \
   --machine-type n1-standard-1 \
   --can-ip-forward \
+  --scopes default,compute-ro \
+  --tags "gce-envoy-consul-sds" \
   --metadata-from-file startup-script=scripts/bootstrap-server.sh
 ```
 
@@ -54,8 +62,6 @@ nomad-2  us-east1-b  n1-standard-1               10.x.0.4     x.x.x.x         RU
 nomad-3  us-east1-b  n1-standard-1               10.x.0.3     x.x.x.x         RUNNING
 ```
 
-Join Nomad nodes
-
 ```bash
 gcloud compute ssh nomad-1
 ```
@@ -72,8 +78,17 @@ This tool needs to create the directory
 Do you want to continue (Y/n)? 
 ```
 
+List Consul members
+
 ```bash
-nomad server-join nomad-2 nomad-3
+consul members
+```
+
+```bash
+Node     Address          Status  Type    Build  Protocol  DC
+nomad-1  10.x.0.2:8301  alive   server  0.9.2  2         dc1
+nomad-2  10.x.0.3:8301  alive   server  0.9.2  2         dc1
+nomad-3  10.x.0.4:8301  alive   server  0.9.2  2         dc1
 ```
 
 Check Nomad cluster status
@@ -89,25 +104,6 @@ nomad-2.global  10.x.0.3  4648  alive   false   2         0.6.2  dc1         glo
 nomad-3.global  10.x.0.4  4648  alive   false   2         0.6.2  dc1         global
 ```
 
-Join Consul servers
-
-```bash
-consul join nomad-2 nomad-3
-```
-
-List Consul members
-
-```bash
-consul members
-```
-
-```bash
-Node     Address          Status  Type    Build  Protocol  DC
-nomad-1  10.x.0.2:8301  alive   server  0.9.2  2         dc1
-nomad-2  10.x.0.3:8301  alive   server  0.9.2  2         dc1
-nomad-3  10.x.0.4:8301  alive   server  0.9.2  2         dc1
-```
-
 Logout of `nomad-1` back to your Cloud Shell or local terminal
 
 ```bash
@@ -116,7 +112,7 @@ exit
 
 ## Bootstrap a Nomad Workers
 
-We will install Nomad, dnsmasq and docker
+In this section we will create `nomad-worker-1`, `nomad-worker-2`, `nomad-worker-3`, `nomad-worker-4` and `nomad-worker-5` instances. We will install Nomad, Consul, dnsmasq and docker on the workers.
 
 ```bash
 gcloud compute instances create nomad-worker-1 nomad-worker-2 nomad-worker-3 nomad-worker-4 nomad-worker-5 \
@@ -125,6 +121,8 @@ gcloud compute instances create nomad-worker-1 nomad-worker-2 nomad-worker-3 nom
   --boot-disk-size 200GB \
   --machine-type n1-standard-1 \
   --can-ip-forward \
+  --scopes default,compute-ro \
+  --tags "gce-envoy-consul-sds" \
   --metadata-from-file startup-script=scripts/bootstrap-client.sh
 ```
 
@@ -163,53 +161,6 @@ ID        DC   Name            Class   Drain  Status
 2c66ca1c  dc1  nomad-worker-5  <none>  false  ready
 ```
 
-Clone [envoy-consul-sds](https://github.com/anubhavmishra/envoy-consul-sds) git repo on `nomad-1`
-
-```bash
-git clone https://github.com/anubhavmishra/envoy-consul-sds.git
-```
-
-Deploy Consul system job
-
-```bash
-cd envoy-consul-sds
-```
-
-```bash
-nomad plan jobs/consul.nomad
-```
-
-```bash
-+ Job: "consul"
-+ Task Group: "consul-agent" (5 create)
-  + Task: "consul-agent" (forces create)
-Scheduler dry-run:
-- All tasks successfully allocated.
-Job Modify Index: 0
-To submit the job with version verification run:
-nomad run -check-index 0 jobs/consul.nomad
-When running the job with the check-index flag, the job will only be run if the
-server side version matches the job modify index returned. If the index has
-changed, another user has modified the job and the plan's results are
-potentially invalid.
-```
-
-```bash
-nomad run jobs/consul.nomad
-```
-
-Check Consul system job status
-
-```bash
-nomad status consul
-```
-
-Join Consul clients to the cluster
-
-```bash
-consul join nomad-worker-1 nomad-worker-2 nomad-worker-3 nomad-worker-4 nomad-worker-5
-```
-
 List Consul members
 
 ```bash

docs/deploy-envoy-consul-sds.md

@@ -4,8 +4,6 @@ The `envoy-consul-sds` service will run as a Nomad service anywhere in the Nomad
 
 ## Prerequisites
 
-*These steps might have been completed in the previous section. Hence skip to [Deploy Envoy Consul Service Discovery Service](#deploy-envoy-consul-service-discovery-service-1)*
-
 SSH into a Nomad Server
 
 ```bash 

scripts/bootstrap-client.sh

@@ -3,10 +3,46 @@
 export IP_ADDRESS=$(curl -s -H "Metadata-Flavor: Google" \
   http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip)
 
+export PROJECT_NAME=$(curl -s -H "Metadata-Flavor: Google" \
+  http://metadata.google.internal/computeMetadata/v1/project/project-id)
+
 apt-get update
 # Install unzip and dnsmasq
 apt-get install -y unzip dnsmasq
 
+## Setup consul
+mkdir -p /var/lib/consul
+
+wget https://releases.hashicorp.com/consul/0.9.2/consul_0.9.2_linux_amd64.zip
+unzip consul_0.9.2_linux_amd64.zip
+mv consul /usr/local/bin/consul
+rm consul_0.9.2_linux_amd64.zip
+
+cat > consul.service <<'EOF'
+[Unit]
+Description=consul
+Documentation=https://consul.io/docs/
+
+[Service]
+ExecStart=/usr/local/bin/consul agent \
+  -advertise=ADVERTISE_ADDR \
+  -bind=0.0.0.0 \
+  -retry-join "provider=gce project_name=PROJECT_NAME tag_value=gce-envoy-consul-sds" \
+  -data-dir=/var/lib/consul
+  
+ExecReload=/bin/kill -HUP $MAINPID
+LimitNOFILE=65536
+
+[Install]
+WantedBy=multi-user.target
+EOF
+
+sed -i "s/ADVERTISE_ADDR/${IP_ADDRESS}/" consul.service
+sed -i "s/PROJECT_NAME/${PROJECT_NAME}/" consul.service
+mv consul.service /etc/systemd/system/consul.service
+systemctl enable consul
+systemctl start consul
+
 # Download and install Nomad
 wget https://releases.hashicorp.com/nomad/0.6.2/nomad_0.6.2_linux_amd64.zip
 unzip nomad_0.6.2_linux_amd64.zip
@@ -68,6 +104,8 @@ EOF
 
 systemctl enable dnsmasq
 systemctl start dnsmasq
+# Force restart for adding consul dns
+systemctl restart dnsmasq
 
 # Install docker
 apt-get install -y docker.io

scripts/bootstrap-server.sh

@@ -27,6 +27,7 @@ Documentation=https://consul.io/docs/
 ExecStart=/usr/local/bin/consul agent \
   -advertise=ADVERTISE_ADDR \
   -bind=0.0.0.0 \
+  -bootstrap-expect 3 \
   -retry-join "provider=gce project_name=PROJECT_NAME tag_value=gce-envoy-consul-sds" \
   -client=0.0.0.0 \
   -data-dir=/var/lib/consul \
@@ -46,20 +47,6 @@ mv consul.service /etc/systemd/system/consul.service
 systemctl enable consul
 systemctl start consul
 
-# Wait until consul comes up
-sleep 2
-
-# Configure dnsmasq
-mkdir -p /etc/dnsmasq.d
-cat > /etc/dnsmasq.d/10-consul <<'EOF'
-server=/consul/127.0.0.1#8600
-EOF
-
-systemctl enable dnsmasq
-systemctl start dnsmasq
-# Force restart for adding consul dns
-systemctl restart dnsmasq
-
 # Download and install Nomad
 wget https://releases.hashicorp.com/nomad/0.6.2/nomad_0.6.2_linux_amd64.zip
 unzip nomad_0.6.2_linux_amd64.zip
@@ -115,3 +102,14 @@ mv nomad.service /etc/systemd/system/nomad.service
 
 systemctl enable nomad
 systemctl start nomad
+
+# Configure dnsmasq
+mkdir -p /etc/dnsmasq.d
+cat > /etc/dnsmasq.d/10-consul <<'EOF'
+server=/consul/127.0.0.1#8600
+EOF
+
+systemctl enable dnsmasq
+systemctl start dnsmasq
+# Force restart for adding consul dns
+systemctl restart dnsmasq