Skip to content

Commit

Permalink
docs: rabbitmq example added (#546)
Browse files Browse the repository at this point in the history
  • Loading branch information
marcosschroh authored Feb 16, 2024
1 parent 240e27b commit cc59d3a
Show file tree
Hide file tree
Showing 8 changed files with 306 additions and 1 deletion.
10 changes: 10 additions & 0 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,13 @@ Minimal redis [example](https://github.com/marcosschroh/dataclasses-avroschema/t
```

*(This script is complete, it should run "as is")*

## RabbitMQ examples

Minimal rabbitmq [example](https://github.com/marcosschroh/dataclasses-avroschema/blob/master/examples/rabbitmq-examples/rabbitmq-pika/rabbitmq_pika/app.py) with [pika](https://github.com/pika/pika) driver.

```python title="rabbitmq example"
--8<-- "examples/rabbitmq-examples/rabbitmq-pika/rabbitmq_pika/app.py"
```

*(This script is complete, it should run "as is")*
48 changes: 48 additions & 0 deletions examples/rabbitmq-examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Examples of dataclasses-avroschema with RabbitMQ

You will find an example of how to use [dataclasses-avroschema](https://github.com/marcosschroh/dataclasses-avroschema) and the integration with the python drivers [pika](https://github.com/pika/pika)

## Requirements

`python 3.8+ poetry docker docker-compose`

### Cluster setup

1. `cd rabbitmq-examples`
1. `docker-compose up` to start the rabbitmq cluster

## Examples

As examples you will find how to serialize/deserialize python dataclasses using the `AvroModel` provided by `dataclasses-avroschema` and how to produce/consume events using specifics python kafka drivers.

The model is for the examples is the following:

```python
class FavoriteColor(enum.Enum):
BLUE = "BLUE"
YELLOW = "YELLOW"
GREEN = "GREEN"


@dataclass
class UserModel(AvroModel):
"An User"
name: str
age: int
favorite_colors: FavoriteColor = FavoriteColor.BLUE
country: str = "Argentina"
address: Optional[str] = None

class Meta:
namespace = "User.v1"
aliases = ["user-v1", "super user"]
```

### dataclasses-avroschema and redis streams with walrus

In the file [app.py](https://github.com/marcosschroh/dataclasses-avroschema/blob/master/examples/rabbitmq-examples/rabbitmq-pika/rabbitmq_pika/app.py) you will find a the simplest use case of `rabbitmq streams` using the driver `pike`.

1. `cd rabbitmq-examples/rabbitmq-pika`
1. `poetry install`
1. `poetry run app`
1. To stop the producer/consumer use `Ctrl + c`
9 changes: 9 additions & 0 deletions examples/rabbitmq-examples/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
version: '3'
services:
rabbitmq:
image: "rabbitmq:3.10"
hostname: rabbitmq
container_name: rabbitmq
ports:
- 5672:5672

139 changes: 139 additions & 0 deletions examples/rabbitmq-examples/rabbitmq-pika/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions examples/rabbitmq-examples/rabbitmq-pika/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[tool.poetry]
name = "rabbitmq-pika"
version = "0.1.0"
description = ""
authors = ["marcosschroh <schrohm@gmail.com>"]
packages = [{include = "rabbitmq_pika"}]

[tool.poetry.dependencies]
python = "^3.8"
dataclasses-avroschema = { path = "../../../.", develop = true }
pika = "^1.3.2"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
app = "rabbitmq_pika.app:main"

Empty file.
79 changes: 79 additions & 0 deletions examples/rabbitmq-examples/rabbitmq-pika/rabbitmq_pika/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
import enum
import random
from dataclasses import dataclass
from typing import Optional

import pika

from dataclasses_avroschema import AvroModel


class FavoriteColor(enum.Enum):
BLUE = "BLUE"
YELLOW = "YELLOW"
GREEN = "GREEN"


@dataclass
class UserModel(AvroModel):
"An User"

name: str
age: int
favorite_colors: FavoriteColor = FavoriteColor.BLUE
country: str = "Argentina"
address: Optional[str] = None

class Meta:
namespace = "User.v1"
aliases = ["user-v1", "super user"]


def consume(*, channel, queue_name: str):
def callback(channel: str, method, properties, body):
print(f"Message received with body: {body} in channel {method} \n")
user = UserModel.deserialize(body)
print(f"Message deserialized: {user} \n")

channel.basic_consume(on_message_callback=callback, queue=queue_name)
channel.start_consuming()


def produce(*, channel, queue_name: str, total_events: int = 10):
for event_number in range(1, total_events + 1):
# Produce message
print(f"Sending event number {event_number}")

user = UserModel(
name=random.choice(
[
"Juan",
"Peter",
"Michael",
"Moby",
"Kim",
]
),
age=random.randint(1, 50),
)

# create the message
message = user.serialize()

# publish
channel.basic_publish(exchange="", routing_key=queue_name, body=message)


def main():
rabbitmq_url = "amqp://guest:guest@localhost:5672/%2f"
queue_name = "test-stream"
params = pika.URLParameters(rabbitmq_url)

connection = pika.BlockingConnection(params)
channel = connection.channel()
channel.queue_declare(queue=queue_name, durable=True)

produce(channel=channel, queue_name=queue_name)
consume(channel=channel, queue_name=queue_name)

connection.close()
2 changes: 1 addition & 1 deletion examples/redis-examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ You will find an example of how to use [dataclasses-avroschema](https://github.c
### Cluster setup

1. `cd redis-examples`
1. `make redis-cluster` to start the kafka cluster (kafka and zookeeper)
1. `make redis-cluster` to start the redis cluster

## Examples

Expand Down

0 comments on commit cc59d3a

Please sign in to comment.