Add samples for Cloud Tasks

appengine/flexible/tasks/README.md

@@ -0,0 +1,21 @@
+# Google Cloud Tasks Samples
+
+Sample program for interacting with the Google Cloud Tasks API.
+
+The `pull_queues` directory contains command line samples for listing queues, creating
+tasks, pulling tasks, and acknowledging them.
+
+The `appengine_queues` directory contains an App Engine app and command line
+samples for creating tasks to be pushed to the App Engine app.
+
+See the respective READMEs for detailed instructions.
+
+## Contributing changes
+
+Contributions are not accepted during Alpha.
+
+## Licensing
+
+* See [LICENSE](LICENSE)
+
+

appengine/flexible/tasks/appengine_queues/README.md

@@ -0,0 +1,133 @@
+# Google Cloud Tasks App Engine Queue Samples
+
+Sample command-line program for interacting with the Cloud Tasks API
+using App Engine queues.
+
+App Engine queues push tasks to an App Engine HTTP target. This directory
+contains both the App Engine app to deploy, as well as the snippets to run
+locally to push tasks to it, which could also be called on App Engine.
+
+`app_engine_queue_snippets.py` is a simple command-line program to create tasks
+to be pushed to the App Engine app.
+
+`main.py` is the main App Engine app. This app serves as an endpoint to receive
+App Engine task attempts.
+
+`app.yaml` configures the App Engine app.
+
+
+## Prerequisites to run locally:
+
+The samples require a Python environment with
+[pip](https://pypi.python.org/pypi/pip) installed.
+[virtualenv](https://virtualenv.readthedocs.org/en/latest/) is also recommended.
+
+All samples require a Google Cloud Project whitelisted for the Cloud Tasks API.
+
+* Enable the Cloud Tasks API
+* Enable the Cloud Datastore API (used to demonstrate storing payloads)
+
+To create a project and enable the API, go to the [Google Developers
+Console](https://console.developer.google.com). You must also create an API key.
+This can be done under API Manager -> Credentials.
+
+To install the Python application dependencies, run the following commands:
+
+    * pip install -r requirements.txt
+
+## Authentication
+
+To set up authentication locally, download the
+[Cloud SDK](https://cloud.google.com/sdk), and run
+
+    gcloud auth application-default login
+
+On App Engine, authentication credentials will be automatically detected.
+
+On Compute Engine and Container Engine, authenticatino credentials will be
+automatically detected, but the instances must have been created with the
+necessary scopes.
+
+In any other environment, for example Compute Engine instance without the
+necessary scopes, you should set `GOOGLE_APPLICATION_CREDENTIALS` environment
+variable to a JSON key file for a service account.
+
+See the [authentication guide](https://cloud.google.com/docs/authentication)
+for more information.
+
+## Creating a queue
+
+Queues can not currently be created by the API. To create a queue using the
+Cloud SDK, use the provided queue.yaml:
+
+    gcloud app deploy queue.yaml
+
+## Deploying the App Engine app
+
+First, vendor the dependencies into the project:
+
+    pip install -r requirements.txt
+
+Next, deploy the App Engine app
+
+    gcloud app deploy
+
+Verify the index page is serving:
+
+    gcloud app browse
+
+The App Engine app serves as a target for the push requests. It has an
+endpoint `/set_payload` that that stores the payload from the HTTP POST data in
+Cloud Datastore. The payload can be accessed in your browser at the
+ `/get_payload` endpoint with a GET request.
+
+## Running the Samples
+
+The project ID must be specified either as a command line argument using
+`--project-id`, or by editing `DEFAULT_PROJECT_ID` within `task_snippets.py`.
+
+Set the environment variables:
+
+    export PROJECT_ID=my-project-id
+    export LOCATION_ID=us-central1
+    export QUEUE_ID=my-appengine-queue # From queue.yaml
+
+View all queues:
+
+     python app_engine_queue_snippets.py --api_key=$API_KEY list-queues --project_id=$PROJECT_ID --location_id=$LOCATION_ID
+
+Set the queue name as an environment variable:
+
+    export QUEUE_NAME=projects/$PROJECT_ID/locations/$LOCATION_ID/queues/$QUEUE_ID
+
+Create a task, targeted at the `set_payload` endpoint with a payload specified:
+
+   python app_engine_queue_snippets.py --api_key=$API_KEY create-task --queue_name=$QUEUE_NAME --payload=hello
+
+Now view that the payload was received and verify the count and payload:
+
+    http://your-app-id.appspot.com/get_payload
+
+Create a task that will be scheduled for a few seconds in the future using
+the `--in_seconds` flag:
+
+    python app_engine_queue_snippets.py --api_key=$API_KEY create-task --queue_name=$QUEUE_NAME --payload=hello --in_seconds=30
+
+Since `--in_seconds` was set to 30, it will take 30 seconds for the new
+payload to be pushed to the `/get_payload` endpoint, which can then be viewed at:
+
+    http://your-app-id.appspot.com/get_payload
+
+It might also be helpful to view the request logs of your App Engine app:
+
+    https://console.cloud.google.com/logs
+
+## Testing the Samples
+
+Install the testing dependencies:
+
+    pip install -r requirements-test.txt
+
+Run pytest:
+
+    pytest

appengine/flexible/tasks/appengine_queues/app.yaml

@@ -0,0 +1,6 @@
+runtime: python
+env: flex
+entrypoint: gunicorn -b :$PORT main:app
+
+runtime_config:
+  python_version: 3

appengine/flexible/tasks/appengine_queues/app_engine_queue_snippets.py

@@ -0,0 +1,144 @@
+# Copyright 2016 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import base64
+import datetime
+
+from googleapiclient import discovery
+
+
+def format_rfc3339(datetime_instance):
+    """Format a datetime per RFC 3339."""
+    return datetime_instance.isoformat("T") + "Z"
+
+
+def get_seconds_from_now_rfc3339(seconds):
+    """Return seconds from the current time as a RFC 3339 string."""
+    d = datetime.datetime.utcnow() + datetime.timedelta(seconds=seconds)
+    return format_rfc3339(d)
+
+
+def list_queues(api_key, project_id, location_id):
+    """List the queues in the location."""
+    client = get_client(api_key)
+    parent = 'projects/{}/locations/{}'.format(project_id, location_id)
+    queues = []
+    next_page_token = None
+
+    while True:
+        response = client.projects().locations().queues().list(
+            parent=parent, pageToken=next_page_token).execute()
+        queues += response['queues']
+        if next_page_token is None:
+            break
+
+    print('Listing queues for location {}'.format(location_id))
+
+    for queue in response['queues']:
+        print queue['name']
+    return response
+
+
+def create_task(api_key, queue_name, payload=None, in_seconds=None):
+    """Create a task for a given queue with an arbitrary payload."""
+    client = get_client(api_key)
+
+    url = '/set_payload'
+    task = {
+        'task': {
+            'app_engine_task_target': {
+                'http_method': 'POST',
+                'relative_url': url
+            }
+        }
+    }
+
+    if payload is not None:
+        task['task']['app_engine_task_target']['payload'] = base64.b64encode(
+            payload)
+
+    if in_seconds is not None:
+        scheduled_time = get_seconds_from_now_rfc3339(int(in_seconds))
+        task['task']['schedule_time'] = scheduled_time
+
+    print('Sending task {}'.format(task))
+
+    response = client.projects().locations().queues().tasks().create(
+        parent=queue_name, body=task).execute()
+
+    # By default CreateTaskRequest.responseView is BASIC, so not all
+    # information is retrieved by default because some data, such as payloads,
+    # might be desirable to return only when needed because of its large size
+    # or because of the sensitivity of data that it contains.
+    print('Created task {}'.format(response['name']))
+    return response
+
+
+def get_client(api_key):
+    """Build an authenticated http client."""
+    DISCOVERY_URL = 'https://cloudtasks.googleapis.com/$discovery/rest?version=v2beta2&key={}'.format(
+        api_key)
+    client = discovery.build('cloudtasks', 'v2beta2',
+                             discoveryServiceUrl=DISCOVERY_URL)
+    return client
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter)
+
+    parser.add_argument('--api_key', help='API Key', required=True)
+
+    subparsers = parser.add_subparsers(dest='command')
+
+    list_queues_parser = subparsers.add_parser(
+        'list-queues',
+        help=list_queues.__doc__)
+
+    list_queues_parser.add_argument(
+        '--project_id',
+        help='Project ID you want to access.',
+        required=True)
+
+    list_queues_parser.add_argument(
+        '--location_id',
+        help='Location of the queues.',
+        required=True)
+
+    create_task_parser = subparsers.add_parser(
+        'create-task',
+        help=create_task.__doc__)
+    create_task_parser.add_argument(
+        '--queue_name',
+        help='Fully qualified name of the queue to add the task to.'
+    )
+
+    create_task_parser.add_argument(
+        '--payload',
+        help='Optional payload to attach to the push queue.'
+    )
+
+    create_task_parser.add_argument(
+        '--in_seconds',
+        help='The number of seconds from now to schedule task attempt.'
+    )
+
+    args = parser.parse_args()
+
+    if args.command == 'list-queues':
+        list_queues(args.api_key, args.project_id, args.location_id)
+    if args.command == 'create-task':
+        create_task(args.api_key, args.queue_name, args.payload, args.in_seconds)

appengine/flexible/tasks/appengine_queues/app_engine_queue_snippets_test.py

@@ -0,0 +1,46 @@
+# Copyright 2016 Google Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import mock
+
+import app_engine_queue_snippets
+
+TEST_PROJECT_ID = 'mock-project'
+TEST_LOCATION = 'us-central1'
+API_KEY = 'mock-api-key'
+TEST_QUEUE_NAME = 'projects/{}/locations/{}/queues/{}'.format(
+    TEST_PROJECT_ID, TEST_LOCATION, 'my-push-queue')
+
+
+@mock.patch('app_engine_queue_snippets.get_client')
+def test_list_queues(get_client):
+    locations = get_client.return_value.projects.return_value.locations
+    queues = locations.return_value.queues
+    execute_function = queues.return_value.list.return_value.execute
+    execute_function.return_value = {'name': 'task_name', 'queues': []}
+    app_engine_queue_snippets.list_queues(API_KEY, TEST_PROJECT_ID,
+                                          TEST_LOCATION)
+    assert execute_function.called
+
+
+@mock.patch('app_engine_queue_snippets.get_client')
+def test_create_task(get_client):
+    projects = get_client.return_value.projects.return_value
+    locations = projects.locations.return_value
+    create_function = locations.queues.return_value.tasks.return_value.create
+    execute_function = create_function.return_value.execute
+    execute_function.return_value = {'name': 'task_name'}
+    app_engine_queue_snippets.create_task(API_KEY, TEST_QUEUE_NAME)
+    assert execute_function.called
+