Skip to content

Commit 55045bb

Browse files
committed
Update the celery guide
Modified the section about the flower authentication, as the auth.py module in tornado was deprecated and the example code app had been adjusted. Add some text to serve more informations about flower.
1 parent 8a50b66 commit 55045bb

File tree

1 file changed

+31
-29
lines changed

1 file changed

+31
-29
lines changed

Guides/Python/Celery.md

Lines changed: 31 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
# Deploying Celery on cloudControl
2-
[Celery] is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
2+
[Celery] is an asynchronous task queue/job queue based on distributed message
3+
passing. It is focused on real-time operation, but supports scheduling as well.
34

4-
In this tutorial we're going to show you how to deploy an example Celery app using the [CloudAMQP Add-on] and a [Worker] on [cloudControl].
5+
In this tutorial we're going to show you how to deploy an example Celery app
6+
using the [CloudAMQP Add-on], a [Worker] and [Flower] on [cloudControl].
57

68
## The Example App Explained
7-
First, lets clone the example code from Github. It is based on the official [first steps with Celery guide][celeryguide] and also includes [Flower] the Celery web interface.
9+
First, lets clone the example code from Github. It is based on the official [first steps with Celery guide][celeryguide] and also includes [Flower] the Celery web interface for monitoring your application.
810

911
~~~bash
1012
$ git clone git://github.com/cloudControl/python-celery-example-app.git
@@ -17,21 +19,26 @@ The code from the example repository is ready to be deployed. Lets still go thro
1719
The [Python buildpack] tracks dependencies via pip and the `requirements.txt` file. It needs to be placed in the root directory of your repository. Our example app requires both `celery` itself aswell as `flower` Celery's monitoring web app. The `requirements.txt` you cloned as part of the example app looks like this:
1820

1921
~~~pip
20-
celery==3.0.15
21-
flower==0.4.2
22+
celery==3.1.18
23+
flower==0.8.3
2224
~~~
2325

2426
### Process Type Definition
2527
cloudControl uses a [Procfile] to know how to start the app's processes.
2628

27-
The example code also already includes a file called `Procfile` at the top level of your repository. It looks like this:
29+
The example code also already includes a file called `Procfile` at the top level
30+
of your repository. It looks like this:
2831

2932
~~~
30-
web: celery flower --port=$PORT --broker=$CLOUDAMQP_URL --auth=$FLOWER_AUTH_EMAIL
33+
web: celery flower --port=$PORT --broker=$CLOUDAMQP_URL --basic_auth=$AUTH_USER:$AUTH_PW
3134
worker: celery -A tasks worker --loglevel=info
3235
~~~
3336

34-
We have specified two process types here. One called `web` to start the web interface and additionally one called `worker` used to start the actual Celery worker.
37+
We have specified two process types here. One called `web` to start the web
38+
interface and additionally one called `worker` used to start the actual Celery
39+
worker.
40+
41+
*Note: Checkout the Flower docs for other [authentication methods](https://flower.readthedocs.org/en/latest/auth.html)*
3542

3643
### The Celery Task
3744

@@ -69,11 +76,11 @@ $ cctrlapp APP_NAME/default addon.add cloudamqp.lemur
6976

7077
Since we are reading the AMQP URL for the broker from the environment in both, the `Procfile` and the Python code we have to enable providing Add-on credentials as environment variables which is disabled per default for Python apps.
7178

72-
We also set another environment variable called `FLOWER_AUTH_EMAIL` that is passed to the Flower web process for authentication purposes. Without this, the web interface would be public showing your secret AMQP credentials and allowing people to stop your workers.
79+
We also set another environment variables called `AUTH_USER` and `AUTH_PW` that are passed to the Flower web process for authentication purposes. Without this, the web interface would be public showing your secret AMQP credentials and allowing people to stop your workers.
80+
7381

7482
~~~bash
75-
$ cctrlapp APP_NAME/default addon.add config.free --SET_ENV_VARS --FLOWER_AUTH_EMAIL=YOUR_EMAIL_HERE
76-
# seperate multiple emails by comma
83+
$ cctrlapp APP_NAME/default addon.add config.free --SET_ENV_VARS --AUTH_USER=YOUR_FLOWER_USER_HERE --AUTH_PW=YOUR_FLOWER_PW
7784
~~~
7885

7986
This is it. The example code will now find all necessary credentials to connect to the AMQP service automatically in the runtime environment.
@@ -86,28 +93,23 @@ The first push will take a couple of seconds, because it will download and compi
8693
~~~bash
8794
$ cctrlapp APP_NAME/default push
8895
Counting objects: 6, done.
89-
Delta compression using up to 4 threads.
96+
Delta compression using up to 8 threads.
9097
Compressing objects: 100% (4/4), done.
91-
Writing objects: 100% (6/6), 605 bytes, done.
92-
Total 6 (delta 0), reused 0 (delta 0)
98+
Writing objects: 100% (6/6), 577 bytes | 0 bytes/s, done.
99+
Total 6 (delta 2), reused 0 (delta 0)
93100

94101
-----> Receiving push
95-
-----> Preparing Python interpreter (2.7.2)
96-
-----> Creating Virtualenv version 1.7.2
97-
New python executable in .heroku/venv/bin/python2.7
98-
Also creating executable in .heroku/venv/bin/python
99-
Installing distribute..................................................................................................................................................................................................done.
100-
Installing pip................done.
101-
Running virtualenv with interpreter /usr/bin/python2.7
102-
-----> Activating virtualenv
103-
-----> Installing dependencies using pip version 1.2.1
104-
105-
[...]
106-
107-
Successfully installed celery flower billiard python-dateutil kombu tornado anyjson amqp
102+
-----> No runtime.txt provided; assuming python-2.7.8.
103+
-----> Preparing Python runtime (python-2.7.8)
104+
-----> Installing Distribute (0.6.36)
105+
-----> Installing Pip (1.3.1)
106+
-----> Installing dependencies using Pip (1.3.1)
107+
Downloading/unpacking celery==3.1.18 (from -r requirements.txt (line 1))
108+
...
109+
Successfully installed celery flower tornado pytz billiard kombu babel futures certifi backports.ssl-match-hostname anyjson amqp
108110
Cleaning up...
109111
-----> Building image
110-
-----> Uploading image (4.3M)
112+
-----> Uploading image (30.3 MB)
111113

112114
To ssh://APP_NAME@cloudcontrolled.com/repository.git
113115
* [new branch] master -> master
@@ -153,7 +155,7 @@ $ cctrlapp APP_NAME/default log worker
153155
[TIMESTAMP] WRK_ID [TIMESTAMP: INFO/MainProcess] Events enabled by remote.
154156
~~~
155157

156-
If you refresh the web interface at `http://APP_NAME.cloudcontrolled.com` you should be able to see the worker now.
158+
Congratulations, you can now see your Celery application with the worker in the Flower web interface at `http://APP_NAME.cloudcontrolled.com`
157159

158160
To handle more tasks simultaneously you can always just add more workers. (Please note that only the first worker is free, adding additional workers requires a billing account.)
159161

0 commit comments

Comments
 (0)