Skip to content

Commit 3d82afd

Browse files
authored
updates (#12)
1 parent 4c1cdbf commit 3d82afd

File tree

5 files changed

+48
-61
lines changed

5 files changed

+48
-61
lines changed

Cargo.lock

Lines changed: 11 additions & 16 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

book.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,6 @@ src = "src"
66
title = "Rusty Celery"
77

88
[output.html]
9-
default-theme = "Rust"
9+
default-theme = "Coal"
1010
no-section-label = true
1111
git-repository-url = "https://github.com/rusty-celery/rusty-celery.github.io"

src/coming-from-python/index.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ In Python you can register tasks by dynamically importing them at runtime throug
1111
```rust,no_run,noplaypen
1212
# #![allow(non_upper_case_globals)]
1313
# use exitfailure::ExitFailure;
14+
# use celery::TaskResult;
1415
# #[tokio::main]
1516
# async fn main() -> Result<(), ExitFailure> {
1617
# let my_app = celery::app!(
@@ -19,8 +20,8 @@ In Python you can register tasks by dynamically importing them at runtime throug
1920
# task_routes = [],
2021
# );
2122
#[celery::task]
22-
fn add(x: i32, y: i32) -> i32 {
23-
x + y
23+
fn add(x: i32, y: i32) -> TaskResult<i32> {
24+
Ok(x + y)
2425
}
2526
2627
my_app.register_task::<add>().await.unwrap();
@@ -32,18 +33,19 @@ my_app.register_task::<add>().await.unwrap();
3233

3334
While Python Celery provides a CLI that you can use to run a worker, in Rust you'll have to implement your own worker binary. However this is a lot easier than it sounds. At a minimum you just need to initialize your [`Celery`](https://docs.rs/celery/*/celery/struct.Celery.html) application, define and register your tasks, and run the [`Celery::consume`](https://docs.rs/celery/*/celery/struct.Celery.html#method.consume) method within your `main` function.
3435

35-
Note that `Celery::consume` is an `async` method though, which means you need an async runtime to execute it. Luckily this is provided by [`tokio`](https://docs.rs/tokio/*/tokio/) and is as simple as declaring your `main` function `async` and decorating it with the `tokio::main` macro.
36+
Note that `Celery::consume` is an `async` method, which means you need an async runtime to execute it. Luckily this is provided by [`tokio`](https://docs.rs/tokio/*/tokio/) and is as simple as declaring your `main` function `async` and decorating it with the `tokio::main` macro.
3637

3738
Here is a complete example of a worker application:
3839

3940
```rust,no_run,noplaypen
4041
#![allow(non_upper_case_globals)]
4142
43+
use celery::TaskResult;
4244
use exitfailure::ExitFailure;
4345
4446
#[celery::task]
45-
fn add(x: i32, y: i32) -> i32 {
46-
x + y
47+
fn add(x: i32, y: i32) -> TaskResult<i32> {
48+
Ok(x + y)
4749
}
4850
4951
#[tokio::main]

src/guide/defining-tasks.md

Lines changed: 16 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,16 @@ A **task** represents a unit of work that a `Celery` app can produce or consume.
55
The recommended way to define a task is by decorating a function with the [`task`](https://docs.rs/celery/*/celery/attr.task.html) attribute macro:
66

77
```rust,noplaypen
8+
use celery::TaskResult;
9+
810
#[celery::task]
9-
fn add(x: i32, y: i32) -> i32 {
10-
x + y
11+
fn add(x: i32, y: i32) -> TaskResult<i32> {
12+
Ok(x + y)
1113
}
1214
```
1315

16+
If the function has a return value the return type must be a [`TaskResult<T>`](https://docs.rs/celery/*/celery/task/type.TaskResult.html).
17+
1418
Under the hood a task is just a struct that implements the [`Task`](https://docs.rs/celery/*/celery/task/trait.Task.html) trait. When you decorate a function with the task macro, this creates a struct and implements the `Task` trait so that [`Task::run`](https://docs.rs/celery/*/celery/task/trait.Task.html#method.run) calls the function you've defined.
1519

1620
The macro accepts a number of [optional parameters](https://docs.rs/celery/*/celery/attr.task.html#parameters).
@@ -22,49 +26,30 @@ use tokio::time::{self, Duration};
2226
2327
#[celery::task(name = "sleep", timeout = 5)]
2428
async fn delay(secs: u64) {
25-
time::delay_for(Duration::from_secs(secs)).await
29+
time::delay_for(Duration::from_secs(secs)).await;
2630
}
2731
```
2832

2933
## Error handling
3034

31-
When a task executes, i.e. when the `Task::run` method is called, it returns a [`Result<T, TaskError>`](https://docs.rs/celery/*/celery/task/type.TaskResult.html) where `T` is whatever type the function you define returns (like `i32` in the `add` task above, or `()` in the `delay` task). So when the `add` task executes, an `Ok(i32)` will be returned.
32-
33-
The reason `Task::run` has to return a `Result` is so the worker executing the task can know when the task has failed. When an `Err(TaskError)` is returned, the worker considers the task failed and may send it back to the broker to be retried.
35+
When a task executes, i.e. when the `Task::run` method is called, it returns a [`TaskResult<T>`](https://docs.rs/celery/*/celery/task/type.TaskResult.html) which is just a `Result<T, TaskError>`. When an `Err(TaskError)` is returned, the worker considers the task failed and may send it back to the broker to be retried.
3436

35-
A worker will generally treat certain [`TaskError`](https://docs.rs/celery/*/celery/error/enum.TaskError.html) variants differently. So when your task has points of failure, such as in the `read_some_file` example below, you'll need to coerce those possible error types to the appropriate `TaskError` variant and propogate them upwards:
37+
A worker treats certain [`TaskError`](https://docs.rs/celery/*/celery/error/enum.TaskError.html) variants differently. So when your task has points of failure, such as in the `read_some_file` example below, you'll need to coerce those possible error types to the appropriate `TaskError` variant and propogate them upwards:
3638

3739
```rust,noplaypen
38-
use celery::error::TaskResultExt;
40+
use celery::{TaskResult, TaskResultExt};
3941
4042
#[celery::task]
41-
async fn read_some_file() -> String {
43+
async fn read_some_file() -> TaskResult<String> {
4244
tokio::fs::read_to_string("some_file")
4345
.await
44-
.with_unexpected_err("File does not exist")?
46+
.with_unexpected_err("File does not exist")
4547
}
4648
```
4749

4850
Here `tokio::fs::read_to_string("some_file").await` produces a [tokio::io::Result](`https://docs.rs/tokio/0.2.13/tokio/io/type.Result.html`), so we use the helper method `.with_unexpected_err` from the [`TaskResultExt`](https://docs.rs/celery/*/celery/error/trait.TaskResultExt.html) trait to convert this into a `TaskError::UnexpectedError` and then apply the [`?`](https://doc.rust-lang.org/book/ch09-02-recoverable-errors-with-result.html#propagating-errors) operator to propogate it upwards.
4951

50-
> There are two error kinds in particular that are meant as catch-alls for any other type of error that could arise in your task: [`TaskError::UnexpectedError`](https://docs.rs/celery/*/celery/error/enum.TaskError.html#variant.UnexpectedError) and [`TaskError::ExpectedError`](https://docs.rs/celery/*/celery/error/enum.TaskError.html#variant.ExpectedError). The latter should be used for errors that will occasionally happen due to factors outside of your control - such as a third party service being temporarily unavailable - while `UnexpectedError` should be reserved to indicate a bug or that a critical resource is missing.
51-
52-
It's important to note that the return type of the `read_some_file` function is not a `Result` type. In fact, **the return type of the decorated function should never be a `Result` type.** The return type should always be the type that would result from a *successful* execution, and so your function should always return that bare type instead of an `Ok` or `Err`.
53-
54-
If you're familiar with the `?` operator, you may be wondering how we can use this within a function that is marked as returning `String` and not `Result<String, _>`. The reason this works is because the `task` attribute macro modifies the body of function by wrapping it in `Ok({ ... })` and changing the return type to a `Result`.
55-
56-
So in this example the `read_some_file` function is turned into something like this:
57-
58-
```rust,noplaypen
59-
# use celery::error::{TaskResultExt, TaskError};
60-
async fn read_some_file() -> Result<String, TaskError> {
61-
Ok({
62-
tokio::fs::read_to_string("some_file")
63-
.await
64-
.with_unexpected_err("File does not exist")?
65-
})
66-
}
67-
```
52+
There are two error kinds in particular that are meant as catch-alls for any other type of error that could arise in your task: [`TaskError::UnexpectedError`](https://docs.rs/celery/*/celery/error/enum.TaskError.html#variant.UnexpectedError) and [`TaskError::ExpectedError`](https://docs.rs/celery/*/celery/error/enum.TaskError.html#variant.ExpectedError). The latter should be used for errors that will occasionally happen due to factors outside of your control - such as a third party service being temporarily unavailable - while `UnexpectedError` should be reserved to indicate a bug or that a critical resource is missing.
6853

6954
## Positional vs keyword parameters
7055

@@ -88,7 +73,7 @@ use tokio::time::{self, Duration};
8873
#[celery::task]
8974
async fn delay(secs: Option<u64>) {
9075
let secs = secs.unwrap_or(10);
91-
time::delay_for(Duration::from_secs(secs)).await
76+
time::delay_for(Duration::from_secs(secs)).await;
9277
}
9378
```
9479

@@ -114,7 +99,7 @@ use tokio::time::{self, Duration};
11499
on_success = success_callback,
115100
)]
116101
async fn sleep(secs: u64) {
117-
time::delay_for(Duration::from_secs(secs)).await
102+
time::delay_for(Duration::from_secs(secs)).await;
118103
}
119104
120105
async fn failure_callback<T: Task>(task: &T, err: &TaskError) {
@@ -131,6 +116,6 @@ async fn success_callback<T: Task>(task: &T, _ret: &T::Returns) {
131116

132117
## Summary
133118

134-
In summary, tasks are easily defined by decorating a function with the `#[celery::task]` macro. Internally the function is wrapped in a struct that implements the `Task` trait, and the return value of the function is wrapped in a `Result<T, celery::error::TaskError>`. This makes it valid to use `?` directly within your function.
119+
In summary, tasks are easily defined by decorating a function with the `#[celery::task]` macro. If the function returns anything the return type has to be a `TaskResult<T>`. Internally the function is wrapped in a struct that implements the `Task` trait.
135120

136121
The quickest way to propogate expected or unexpected errors from within your task is by using `.with_expected_err("...")?` or `.with_unexpected_err("...")?`, respectively, on the `Result`.

src/quick-start.md

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,21 @@
33
Rusty Celery is provided as the [`celery`](https://crates.io/crates/celery) library on crates.io. To get started, add `celery` as a dependency to your project. Then you can define tasks by decorating functions with the [`task`](https://docs.rs/celery/*/celery/attr.task.html) attribute:
44

55
```rust,noplaypen
6+
use celery::TaskResult;
7+
68
#[celery::task]
7-
fn add(x: i32, y: i32) -> i32 {
8-
x + y
9+
fn add(x: i32, y: i32) -> TaskResult<i32> {
10+
Ok(x + y)
911
}
1012
```
1113

1214
And create a [`Celery`](https://docs.rs/celery/*/celery/struct.Celery.html) app with the [`app`](https://docs.rs/celery/*/celery/macro.app.html) macro:
1315

1416
```rust,no_run,noplaypen
17+
# use celery::TaskResult;
1518
# #[celery::task]
16-
# fn add(x: i32, y: i32) -> i32 {
17-
# x + y
19+
# fn add(x: i32, y: i32) -> TaskResult<i32> {
20+
# Ok(x + y)
1821
# }
1922
let my_app = celery::app!(
2023
broker = AMQP { std::env::var("AMQP_ADDR").unwrap() },
@@ -27,9 +30,10 @@ The Celery app can be used as either a producer or consumer (worker). To send ta
2730
queue for a worker to consume, use the [`Celery::send_task`](https://docs.rs/celery/*/celery/struct.Celery.html#method.send_task) method:
2831

2932
```rust,no_run,noplaypen
33+
# use celery::TaskResult;
3034
# #[celery::task]
31-
# fn add(x: i32, y: i32) -> i32 {
32-
# x + y
35+
# fn add(x: i32, y: i32) -> TaskResult<i32> {
36+
# Ok(x + y)
3337
# }
3438
# #[tokio::main]
3539
# async fn main() -> Result<(), exitfailure::ExitFailure> {
@@ -47,9 +51,10 @@ And to act as worker and consume tasks sent to a queue by a producer, use the
4751
[`Celery::consume`](https://docs.rs/celery/*/celery/struct.Celery.html#method.consume) method:
4852

4953
```rust,no_run,noplaypen
54+
# use celery::TaskResult;
5055
# #[celery::task]
51-
# fn add(x: i32, y: i32) -> i32 {
52-
# x + y
56+
# fn add(x: i32, y: i32) -> TaskResult<i32> {
57+
# Ok(x + y)
5358
# }
5459
# #[tokio::main]
5560
# async fn main() -> Result<(), exitfailure::ExitFailure> {

0 commit comments

Comments
 (0)