This code implements a distributed message broker service that follows a simplified version of the AMQP protocol. The system is designed to support multiple message brokers running simultaneously, each registering itself with a custom DNS service to maintain an updated view of the broker cluster.
A message broker acts as an intermediary between clients, handling their requests and ensuring reliable message delivery. It manages exchanges and queues, which are core components of the message routing process. Clients can publish messages to an exchange, which then routes these messages to one or more queues based on predefined rules and routing keys. Other clients can bind to these queues to receive messages asynchronously, enabling scalable and decoupled communication between producers and consumers.
To ensure high availability and fault tolerance, the system includes a mechanism for automatic leader election. If a message broker fails, the remaining brokers detect the failure and initiate a leader election process using one of three algorithms: Raft, Ring, or Bully. Once a new leader is elected, it takes over the coordination responsibilities and updates the DNS service to reflect the current state of the broker cluster. This dynamic leadership transition ensures that message routing and delivery continue without interruption, maintaining system stability and reliability.
By distributing the workload across multiple brokers and implementing robust failure recovery mechanisms, this service provides a scalable and resilient messaging infrastructure for distributed applications.
Link to client application: [https://github.com/Konstiu/Distributed-Message-Broker-Client]
Use Java JDK-21. The Github-runner uses Oracle Java JDK 21 to run your tests. Make sure your code is compatible
# First compile the project with Maven
mvn compile
# Start the client application with the following command where componentId is one of client-0, client-1 or client-2.
mvn exec:java@<componentId>
# You can also combine both commands into one
mvn compile exec:java@<componentId>You need to replace <componentId> with the corresponding component ID of the server: broker-0, broker-1,
broker-2 and dns-0 are available.
You may need multiple terminal windows to start multiple servers.
The default configuration for the servers can be started using:
mvn compilemvn package -DskipTests- Run selected server
mvn exec:java@broker-0mvn exec:java@broker-1mvn exec:java@broker-2mvn exec:java@monitoring-0mvn exec:java@dns-0
Consider using Java Virtual Threads for short-lived threads and client-handlers for better i/o performance and less expensive thread creation.
Upon connecting to a Broker leader election port, the server sends the greeting ok LEP.
Notifies that the broker with the given id is up for election.
| State | Response |
|---|---|
| success (ring/bully) | ok |
| success (raft) | vote <sender-id> <candidate-id> |
| error syntax | error usage: elect <id> |
Declares the broker with the given ìd as the new leader.
| State | Response |
|---|---|
| success | ack <sender-id> |
| error syntax | error usage: declare <id> |
Sent by the leader to the followers as a heartbeat If the followers do not receive a heartbeat message within a set timeout, a new leader election is started.
| State | Response |
|---|---|
| success | pong |
| error syntax | error protocol error |
If no matching command of the protocol is found, then the broker sends error protocol error and closes the connection
The sender with sender-id votes the for the candidate with candidate-id
This paragraph describes the default testing configuration for the available server (message-broker, dns-server, monitoring-server).
The names of the servers start with 0 and are incremented by 1 every new node.
- Message Broker:
broker-# - DNS Server:
dns-# - Monitoring Server:
monitoring-#
For example the available servers for a test-case verifying that after a raft-leader election, the leader registers its domain at the dns-server could look like this:
dns-0broker-0broker-1broker-2broker-3
The broker-0 listens for publisher/subscribers on TCP port 20000 and for election updates from other brokers on
TCP port 20001. These 2 ports are incremented by 10 for every new broker instance. So for example for the first 3
brokers we would have:
broker-0: messaging=20000, election=20001broker-1: messaging=20010, election=20011broker-2: messaging=20020, election=20021
The dns-0 listens per default on TCP port 18000.
The monitoring-0 listens per default on UDP port 17000.
You can use netcat (nc) to connect to the server and send commands manually. To start a netcat client that connects to
the server at localhost on port 20000, you can use the following commands:
# For Linux and macOS
# Open the terminal. The following command starts a netcat client that connects to the server at localhost on port 20000.
nc localhost 20000# For Windows first ensure that you have installed netcat (ncat). If not, you can download it from the following link: https://nmap.org/download.html#windows
# Open CMD or PowerShell. The following command starts a netcat client that connects to the server at localhost on port 20000.
ncat -C localhost 20000