Add support for configurable dequeue size#3031
Add support for configurable dequeue size#3031schueffi wants to merge 1 commit intopostalserver:mainfrom
Conversation
When sending messages to remote MTAs, the messages get dequeued in batches from the local queue. As the batch-key is the given remote MX server, those messages will be delivered to this remote MTA in one SMTP session. Although this is good for performance (to reuse the same SMTP session for many mails), many of the real-world MTAs do not like sending too much mails at once in one single session. Example error messages are similar to "421 too many messages in this connection" Therefore, we make the limit adjustable (with the default value of 100 to be backwards compatible). From our experiences with the last 5 million emails sent, having a batch size of 10 works almost ever, and 50 seems to be the upper "real world" limit before hitting those rate limits by the remote MTAs.
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
Can we get this merged? :) |
|
Is this possibly a solution for https://github.com/orgs/postalserver/discussions/2917 ? |
Yeah, most likely yes. |
|
Since we're running the patched version of postal with the batch_queue_messages_limit set to 10 in our own infrastructure, we never had any of those messages mentioned in https://github.com/orgs/postalserver/discussions/2917 (we had many of those before, running with the default size of 100). |
|
Asi Iam not a developer :-) How can I apply this patch on my postal server in the easiest way? Many thanks at @schueffi for the commit |
|
Great fix I suppose. Thanks! Facing same issue as https://github.com/orgs/postalserver/discussions/2917 @schueffi @willpower232 Any idea how we can have this for those us using the Docker hosted version? |
|
The other question is, what is needed to merge this into the main branch? |
|
Unfortunately I'm just helping in support, I have no ability to merge things so its entirely down to the others to review and merge PRs |
|
So who would be responsible to do this? Because the last commit has been a while ago. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
please merge |
|
Would be amazing if this could be merged, experiencing similar issues sending to GMAIL |
|
@adamcooke Can you please merge this PR? |
|
@MatthieuBarthel people keep mentioning this issue and for some reason, it keeps getting ignored even though there was an update last month. Considering the solution was to just provide config option with a default instead of hardcoding it inside the library (which I also think isn't logical). I honestly don't understand why this hasn't been merged. It's affecting a lot of people. |
@charlesmudy you addressed this to the wrong person, I'm not in the postal organization :) |
Apologise ... @adamcooke |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
Hello, @adamcooke @melledouwsma @johankok @arthurzenika ... what do we need to do or how to beg you to please merge this commit? |
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
Adds batch_queued_messages_limit configuration option to control the batch size when de-queuing messages. Cherry-picked from postalserver/postal PR postalserver#3031 Co-authored-by: schueffi <schueffi@users.noreply.github.com> Co-authored-by: openhands <openhands@all-hands.dev>
|
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
10 participants
When sending messages to remote MTAs, the messages get dequeued in batches from the local queue. As the batch-key is the given remote MX server, those messages will be delivered to this remote MTA in one SMTP session. Although this is good for performance (to reuse the same SMTP session for many mails), many of the real-world MTAs do not like sending too much mails at once in one single session.
Example error messages are similar to "421 too many messages in this connection"
Therefore, we make the limit adjustable (with the default value of 100 to be backwards compatible). From our experiences with the last 5 million emails sent, having a batch size of 10 works almost ever, and 50 seems to be the upper "real world" limit before hitting those rate limits by the remote MTAs.