Skip to content

Add blocking mode with auto-reconnection and some other features. #632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

Xfaider48
Copy link

@Xfaider48 Xfaider48 commented Jun 8, 2025

I'm glad to introduce a lot of features that were made in the driver working on other projects.

Here is the short list:

  1. The blocking mode option [Consumer]
  2. The auto-reconnection option [Consumer]
  3. The initializing queue option [Consumer]
  4. The alive-check option [Consumer]
  5. More debug/error messages [Consumer]
  6. The verbose-messages option [Consumer]
  7. Retry options [Queue]
  8. Another expiration strategy [Queue]
  9. Support ttl less than a secong [Queue]
  10. Option to cache declared [Queue]
  11. Option to declare full route [Queue]

Some of the features were added to improve a perfomance of hanling messages using Consumer, and the others were added to improve durability.

There options are production ready (I migrated them from the v13, so there could be some bugs because of that).
These changes don't break anything and disabled by default (except the point 5 about new messages)

Now let me explain why here a lof ot options added.

Improving the Consumer

Increasing perfomance

In the default mode (non-blocking) when you don't have any messages in a queue the consumer tries to get a job, and if gets nothing it sleeps, but while it sleeps we can get a job in a queue and that job will wait, so we lose a lot of perfomance and the consumer consumes resources like CPU. To fix the problem the blocking mode was added.

When you use blocking mode the worker just waits for some time (or forever) for new messages, and while it waits it consumes no resources and when a job comes the worker gets it ASAP and starts working on it.

And the last one about perfomance: in the production we don't need any simple logs about successfuly processed jobs, but about errors. To fix it the verbose-messages option was added, you can use it to write messages ONLY if you start worker in the verbose mode (-v)

Increasing durability

We don't live in the perfect world, shit happens sometime and we need to try to handle as much as we can.

To increase the Consumber durability we need to enable the auto-reconnection option.
When something is wrong with the connection the consumer will automatically reconnect. Also we can use initializing queue option, because if the problem happens because of the RabbitMQ and the RabbitMQ restarted (you might don't have anything in it) you will get an error after reconnecting becasue the queus doesn't exist anymore.

When you use the blocking mode the connection can stuck in some strange state (the consumer works, the RabbitMQ sees the consumer, but messages don't reach the consumer at all), it could happen when you use a complex architecture with vurtial machines and/or proxies. To check that the connection is stuck we must call some action and wait for the response, for this the alive-check option was added. Just set N seconds and the consumer will check that all is OK with the connection every N seconds of working (it happens when NO messages comes in N seconds, so this doesn't affect the perfomance a lot).
Also the keepalive option must be enabled for the blocking mode, that's why it's added to the options.

And the last thing for the consumer: I added more logs in it to simplify an investigation about what happened, because sometimes the consumer dies and you don't get anything about what was going on (maybe it was stopped because of some options, maybe it was killed by the system)

Improving the Queue

First of all I want to speak about the option to cache declared. The origial code doesn't have it and it caches all declared exchanges and queues in the memory, but it's completely WRONG (see the section about auto-reconnection option above), because the original code doesn't handle any problems with the connection and don't clear them in the memory. Now it's fixed and the option was added just in case to simplify any investigations of some problems of caching (useful with the Laravel Octane).

The retry options was added to automatically reconnect and try again if we get some problems while we send messages, it helps to avoid problems during short temporary network problems. (Maybe it's better to make another class for it, but now it's included in the Queue class)

The another expiration strategy was added to impove latering messages. The original strategy creates a lot of queues (one queue per TTL), but the new one just uses the expiration option of messages, and the latering messages will become more effective with this option enabled.

The option to declare full route was added to improve durability and stability. If you face some problems with connection and the RabbitMQ lost everything you might want to restore the full route of message, right? So, you can enable this option and before first sending the Queue class will declare exchange + queue + bind them, and you will get all things work.

What else can be improved

Affected PRs

Affected issues

Xfaider48 added 8 commits June 7, 2025 19:32
Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
It allows to adjust durable/autodelete options

Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
It allows to use another expiration strategy

Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
It allows to do auto retries

Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
It allows to init full route (exchange + queue + bind)

Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
Signed-off-by: Alexander Dylevskiy <x.faider48@gmail.com>
@iamfarhad
Copy link

also you can contribute on my package
https://github.com/iamfarhad/LaravelRabbitMQ

@Xfaider48
Copy link
Author

@iamfarhad If there is no any activities from maintainers I can adapt this for your version

@iamfarhad
Copy link

@iamfarhad If there is no any activities from maintainers I can adapt this for your version

I would be happy to have your collaboration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants