Cloud trace demo app (#2716)

* Initial commit for cloud trace demo app

* Unsubmodule cloud trace demo app

* Remove yaml files for b and c; Addressed comments in setup.sh

* Add comments for app.py

* Modify cloud shell button

* Update README file, the remaining comments need further discussions.

* add comment for trace middleware

* shorten dockerfile and add comments on app.py

* Remove unused makefile

* fix indentation and import

* More indentation fixes

* Attempt to add httppretty to resolve python test failure

* Change to python 3.6

* Add opencensus flask requirement to parent dir requirements

* Add httppretty dependency

* Update readme file

* Update readme file

* Change image demo name to cloud-trace-demo

* Delete unneeded license, update readme file with the correct image name and remove requirements

* Refactor directory structure

* Resolve comments on readme

* Resolve comments on readme

* Resolve comments on readme file

* Resolve comments on readme file

* Update step 10

* Add a screenshot of tracelist page

* Replace example image and update Readme file

* Update new line

* Modify readme file

* Update example image

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: Duke Nguyen <nguyentranduc1@gmail.com>

trace/cloud-trace-demo-app/README.md

@@ -0,0 +1,50 @@
+# cloud-trace-demo-app
+
+Open this demo app in [Google Cloud Shell](https://cloud.google.com/shell/docs/). This includes necessary tools.
+
+We provide a public image for the services in this demo app. You could also build
+your own following steps 4 - 6.
+
+[![Open Cloud Trace Demo APP in Cloud Shell](http://gstatic.com/cloudssh/images/open-btn.svg)](https://console.cloud.google.com/cloudshell/open?cloudshell_git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=trace/cloud-trace-demo-app/README.md&cloudshell_tutorial=README.md)
+
+#### Demo Requirements
+If you are using Cloud Shell, skip to the next section.
+
+1. Install gcloud <https://cloud.google.com/sdk/install>
+2. Install kubectl <https://kubernetes.io/docs/tasks/tools/install-kubectl/>
+3. Install docker <https://docs.docker.com/install/>
+
+#### Google Container Registry Image Setup
+If you are using the provided image, skip to the next section.
+
+4. Get default project id and set environment variable:
+    `PROJECT_ID=$(gcloud config get-value project)`
+5. Build Image:
+    `docker build -t gcr.io/${PROJECT-ID}/cloud-trace-demo .`
+6. Upload Image to Container Registry:
+    `gcloud docker -- push gcr.io/${PROJECT-ID}/cloud-trace-demo-test:v1`
+
+#### Create a GKE cluster
+7. Enable Google Cloud and set up region and zone.
+    `gcloud init`
+8. Enable the GKE API & billing:
+    `gcloud services enable container.googleapis.com`
+9. Create a GKE cluster named "demo":
+    `gcloud container clusters create demo`
+
+#### Send Requests to See Generated Traces
+
+10. If you are using your own image, please change the image variable in the following files:
+    * [YAML](./app/demo-service-a.yaml)
+    * [template B](./app/demo-service-b.yaml.template)
+    * [template C](./app/demo-service-c.yaml.template)
+11. Run setup.sh to apply the YAML files.
+    `./setup.sh`
+12. Send request to the last service:
+
+    `curl $(kubectl get svc cloud-trace-demo-a -ojsonpath='{.status.loadBalancer.ingress[0].ip}')`
+13. Visit [Trace List](https://pantheon.corp.google.com/traces/list) to check traces generated.
+    Click on any trace in the graph to see the Waterfall View.
+    ![Screenshot](./example.png)
+14. Clean up GKE cluster/pods/services
+    `gcloud container clusters delete demo`

trace/cloud-trace-demo-app/app/Dockerfile

@@ -0,0 +1,19 @@
+FROM python:3.6.0
+
+RUN pip3 install --upgrade pip
+
+RUN apt-get update \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+COPY requirements.txt /app/
+
+WORKDIR /app
+RUN pip3 install -r requirements.txt
+
+ENV APP_HOME /app
+WORKDIR $APP_HOME
+COPY . ./
+
+ENTRYPOINT ["python"]
+CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 main:app

trace/cloud-trace-demo-app/app/app.py

@@ -0,0 +1,76 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+A sample app demonstrating Stackdriver Trace
+"""
+
+from flask import Flask
+from opencensus.trace import execution_context
+from opencensus.trace.propagation import google_cloud_format
+from opencensus.trace.samplers import AlwaysOnSampler
+from opencensus.ext.stackdriver.trace_exporter import StackdriverExporter
+from opencensus.ext.flask.flask_middleware import FlaskMiddleware
+
+import requests
+import argparse
+import time
+import random
+
+app = Flask(__name__)
+
+propagator = google_cloud_format.GoogleCloudFormatPropagator()
+
+
+def createMiddleWare(exporter):
+    # Configure a flask middleware that listens for each request and applies automatic tracing.
+    # This needs to be set up before the application starts.
+    middleware = FlaskMiddleware(
+        app,
+        exporter=exporter,
+        propagator=propagator,
+        sampler=AlwaysOnSampler())
+    return middleware
+
+
+@app.route('/')
+def template_test():
+    # Sleep for a random time to imitate a random processing time
+    time.sleep(random.uniform(0, 0.5))
+    # Keyword that gets passed in will be concatenated to the final output string.
+    output_string = app.config['keyword']
+    # If there is no endpoint, return the output string.
+    url = app.config['endpoint']
+    if url == "":
+        return output_string, 200
+    # Endpoint is the next service to send string to.
+    data = {'body': output_string}
+    trace_context_header = propagator.to_header(execution_context.get_opencensus_tracer().span_context)
+    response = requests.get(
+        url,
+        params=data,
+        headers={
+          'X-Cloud-Trace-Context' : trace_context_header}
+    )
+    return response.text + app.config['keyword']
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--keyword",  default="", help="name of the service.")
+    parser.add_argument("--endpoint", default="", help="endpoint to dispatch appended string, simply respond if not set")
+    args = parser.parse_args()
+    app.config['keyword'] = args.keyword
+    app.config['endpoint'] = args.endpoint
+    createMiddleWare(StackdriverExporter())
+    app.run(debug=True, host='0.0.0.0', port=8080)

trace/cloud-trace-demo-app/app/app_test.py

@@ -0,0 +1,56 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+A sample app demonstrating Stackdriver Trace
+"""
+
+import app
+import mock
+import httpretty
+
+
+def test_send_response():
+    service_keyword = "Hello"
+    app.app.testing = True
+    app.app.config['keyword'] = service_keyword
+    app.app.config['endpoint'] = ""
+    client = app.app.test_client()
+    resp = client.get('/')
+    assert resp.status_code == 200
+    assert service_keyword in resp.data.decode('utf-8')
+
+
+@httpretty.activate
+def test_request_url_with_trace_context():
+    service1_keyword = "World"
+    service2_url = "http://example.com"
+    service2_keyword = "Hello"
+    app.app.testing = True
+    app.app.config['keyword'] = service1_keyword
+    app.app.config['endpoint'] = service2_url
+
+    def request_callback(request, uri, response_headers):
+        # Assert that the request is sent with a trace context
+        assert request.headers.get("X-Cloud-Trace-Context")
+        return [200, response_headers, service2_keyword]
+
+    httpretty.register_uri(httpretty.GET, service2_url, body=request_callback)
+    exporter = mock.Mock()
+    app.createMiddleWare(exporter)
+
+    client = app.app.test_client()
+    resp = client.get('/')
+    assert resp.status_code == 200
+    # Assert that the response is a concatenation of responses from both services
+    assert service2_keyword + service1_keyword in resp.data.decode('utf-8')

trace/cloud-trace-demo-app/app/demo-service-a.yaml

@@ -0,0 +1,40 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: cloud-trace-demo-a
+  labels:
+    app: cloud-trace-demo-app-a
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: cloud-trace-demo-app-a
+  template:
+    metadata:
+      name: cloud-trace-demo-a
+      labels:
+        app: cloud-trace-demo-app-a
+    spec:
+      containers:
+      - name: cloud-trace-demo-container
+        image: gcr.io/cloud-trace-demo/demo-image:latest
+        command:
+        - python
+        args:
+        - app.py
+        - --keyword=Hello
+        ports:
+        - containerPort: 8080
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: cloud-trace-demo-a
+spec:
+  selector:
+    app: cloud-trace-demo-app-a
+  ports:
+  - protocol: TCP
+    port: 80
+    targetPort: 8080
+  type: LoadBalancer

trace/cloud-trace-demo-app/app/demo-service-b.yaml.template

@@ -0,0 +1,41 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: cloud-trace-demo-b
+  labels:
+    app: cloud-trace-demo-app-b
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: cloud-trace-demo-app-b
+  template:
+    metadata:
+      name: cloud-trace-demo-b
+      labels:
+        app: cloud-trace-demo-app-b
+    spec:
+      containers:
+      - name: cloud-trace-demo-container
+        image: gcr.io/cloud-trace-demo/demo-image:latest
+        command:
+        - python
+        args:
+        - app.py
+        - --keyword=world
+        - --endpoint=http://{{ endpoint }}
+        ports:
+        - containerPort: 8080
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: cloud-trace-demo-b
+spec:
+  selector:
+    app: cloud-trace-demo-app-b
+  ports:
+  - protocol: TCP
+    port: 80
+    targetPort: 8080
+  type: LoadBalancer

trace/cloud-trace-demo-app/app/demo-service-c.yaml.template

@@ -0,0 +1,41 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: cloud-trace-demo-c
+  labels:
+    app: cloud-trace-demo-app-c
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: cloud-trace-demo-app-c
+  template:
+    metadata:
+      name: cloud-trace-demo-c
+      labels:
+        app: cloud-trace-demo-app-c
+    spec:
+      containers:
+      - name: cloud-trace-demo-container
+        image: gcr.io/cloud-trace-demo/demo-image:latest
+        command:
+        - python
+        args:
+        - app.py
+        - --keyword=!\n
+        - --endpoint=http://{{ endpoint }}
+        ports:
+        - containerPort: 8080
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: cloud-trace-demo-c
+spec:
+  selector:
+    app: cloud-trace-demo-app-c
+  ports:
+  - protocol: TCP
+    port: 80
+    targetPort: 8080
+  type: LoadBalancer

trace/cloud-trace-demo-app/app/requirements.txt

@@ -0,0 +1,9 @@
+google-cloud-monitoring==0.33.0
+google-cloud-trace==0.23.0
+opencensus==0.7.5
+Flask==1.1.1
+opencensus-ext-stackdriver==0.7.2
+opencensus-ext-flask==0.7.3
+opencensus-context==0.1.1
+grpcio==1.25.0
+httpretty==0.9.7

trace/cloud-trace-demo-app/setup.sh

@@ -0,0 +1,51 @@
+#!/bin/bash
+################## Set up service a ###########################
+
+echo "Creating service a"
+kubectl apply -f app/demo-service-a.yaml
+
+################## Set up service b ###########################
+echo "Fetching the external IP of service a"
+endpoint=""
+for run in {1..20}
+do
+  sleep 5
+  endpoint=`kubectl get svc cloud-trace-demo-a -ojsonpath='{.status.loadBalancer.ingress[0].ip}'`
+  if [[ "$endpoint" != "" ]]; then
+    break
+  fi
+done
+
+if [[ "$endpoint" == "" ]]; then
+  echo "Unable to get external IP for service cloud-trace-demo-a"
+  exit 1
+fi
+
+echo "Passing external IP for the first service ${endpoint} to the second service template"
+sed "s/{{ endpoint }}/${endpoint}/g" app/demo-service-b.yaml.template > app/demo-service-b.yaml
+kubectl apply -f app/demo-service-b.yaml
+rm app/demo-service-b.yaml
+
+################## Set up service c ###########################
+echo "Fetching the external IP of the service b"
+endpoint=""
+for run in {1..20}
+do
+  sleep 5
+  endpoint=`kubectl get svc cloud-trace-demo-b -ojsonpath='{.status.loadBalancer.ingress[0].ip}'`
+  if [[ "$endpoint" != "" ]]; then
+    break
+  fi
+done
+
+if [[ "$endpoint" == "" ]]; then
+  echo "Unable to get external IP for service cloud-trace-demo-a"
+  exit 1
+fi
+
+echo "Passing external IP for the service b ${endpoint} to the service c"
+sed "s/{{ endpoint }}/${endpoint}/g" app/demo-service-c.yaml.template > app/demo-service-c.yaml
+kubectl apply -f app/demo-service-c.yaml
+rm app/demo-service-c.yaml
+
+echo "Successfully deployed all services"