Distributed gRPC Task Orchestrator

A distributed task queue for asynchronous fractal image generation jobs using gRPC for interservice communication and Protobuf for serialization. The system implements a centralized coordinator pattern where a Python server manages task distribution through a priority queue while worker nodes concurrently handle fractal computation. The system features automatic task timeouts and resilience to worker failures.

The project demonstrates concepts widely applicable to distributed systems and data infrastructure, like asynchronous task queue management, service-oriented communication, and fault tolerance and resilience.

Below, you'll find exact outputs of these image generation jobs, generated by running client.py.

Mandelbrot Set

Julia Set

Coordinator initialized.
Coordinator running on port 50051!
Worker registered: worker-807144 at localhost
Worker registered: worker-ae4712 at localhost
Task submitted: a1aacdd7-eac6-417a-8a98-976c02cb32ee (Priority: 10)
Task submitted: ac71377d-048a-4cfe-80f8-9b8bf6a979e2 (Priority: 10)
Task submitted: e806e91e-8fab-47f1-8e89-19b976288020 (Priority: 10)
Task submitted: 553ef198-2b87-4543-9813-fb95bb0b5f25 (Priority: 10)
Sending task a1aacdd7-eac6-417a-8a98-976c02cb32ee to worker worker-807144
Task completed: a1aacdd7-eac6-417a-8a98-976c02cb32ee by worker-807144
Sending task ac71377d-048a-4cfe-80f8-9b8bf6a979e2 to worker worker-807144
Sending task e806e91e-8fab-47f1-8e89-19b976288020 to worker worker-ae4712
Task completed: ac71377d-048a-4cfe-80f8-9b8bf6a979e2 by worker-807144
...

Architecture

• coordinator.py - Central manager handling priority heap, worker registry, and task dispatch
• worker.py - Compute node that polls for tasks, generates Mandelbrot/Julia set chunks, and returns results
• client.py - CLI tool for job submission and final image assembly
• proto/task.proto - Protocol Buffer definition for service contracts and messages

Techniques

gRPC & Protobuf

High-performance inter-process communication:

• Serialization: Binary Protocol Buffers for efficient payload transfer
• Transport: HTTP/2 multiplexing via gRPC
• Methods like SubmitTask, GetTask, CompleteTask, RegisterWorker

Priority-Based Queuing

Tasks are processed based on priority order using a min-heap structure:

• Heap Management: heapq-based queue for O(log n) insertion and extraction
• Ordering: Lower priority numbers execute first, and timestamps break ties
• Dataclass: Python @dataclass(order=True) enables native heap comparison

Building and Running

Prerequisites

• grpcio, grpcio-tools, protobuf
• numpy, Pillow

(See requirements.txt)

Setup

Generate the gRPC stubs:

# Windows
mkdir generated
python -m grpc_tools.protoc -I./proto --python_out=./generated --grpc_python_out=./generated ./proto/task.proto

Execution

Start Coordinator:

python coordinator.py

Start Workers (Multiple Terminals):

python worker.py

Submit Job:

# Generates a 1200x800 Mandelbrot set using 8 parallel regions
python client.py --mandelbrot --width 1200 --height 800 --regions 8

# Alternatively, generates a Julia set
python client.py --julia --width 1200 --height 800 --regions 8

Technical Details

Fractal Generation

• Supported Types: Mandelbrot sets and Julia sets (with configurable constants)
• Parallelization: Image is sliced into horizontal bands (each worker computes one band)
• Output: Workers return raw byte arrays, and client reassembles them into a PNG file

Worker Protocol

• Registration: Workers generate a UUID and register capabilities on startup
• Polling: Pull-based model (GetTask) allows stateless load balancing
• Resilience: Failed or timed-out tasks are automatically re-injected into the priority queue

Data Storage

• Workers save result data to a local .storage folder instead of sending large payloads through gRPC
• The client reads from these files and cleans them up after processing