rapids

Reducing Microservice Complexity with Kafka and Reactive Streams

Inspirations

1. Reducing Microservice Complexity with Kafka and Reactive Streams - by Jim Riecken
2. Øredev 2013 - Fred George - Implementing Micro Service Architecture
3. Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines)
4. Understanding Akka Streams, Back Pressure and Asynchronous Architectures

Run in Docker Swarm

Init

docker swarm init

docker service create --name portainer -p 9090:9000 --constraint 'node.role == manager' --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock portainer/portainer -H unix:///var/run/docker.sock

mkdir -p ~/mongodbdata
mkdir -p ~/kafkadata

Start infra

docker stack deploy --compose-file infra-dc.yml rapids

Start app (as monolith)

docker stack deploy --compose-file app-dc.yml rapids

Start app (as microservices)

docker stack deploy --compose-file ms-dc.yml rapids

Stop

docker stack rm rapids

Show (consume) kafka messages:

docker exec -it `docker container ls | grep rapids_kafka | cut -d ' ' -f1` kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic blog-event --from-beginning --property print.key=true

Used topics

blog-command, client-commands, blog-event, performance, discussion-event, web-app, user, discussion-command, error

Use docker for development without kafka

Start MongoDB:

docker stack deploy --compose-file mongo-dc.yml rapids

In sbt console:

~runMain Main -t

Use docker for development with kafka

Start infra:

docker stack deploy --compose-file infra-dc.yml rapids

In sbt console:

~runMain Main -d