Maddy stopped receiving external e-mails; '451 4.4.5 High load, try again later'

[reivilibre]

Describe the bug

At some point in the last day or two, Maddy stopped accepting e-mails from other servers. A user reported this to me today, but said that they could still send e-mails between accounts on my server. I restarted Maddy and e-mails have started coming in again now.

Steps to reproduce

I don't know; it's been running smoothly for about 20 days continuously and I didn't see any reason why this started happening.

Once this issue had been alerted to me, I used https://dnslytics.com/email-test to test my server and it reported that it failed with this error: 451 4.4.5 High load, try again later

Log files

There isn't much interesting before it started happening

Mar 29 10:56:17 sallie.librepush.net maddy[194975]: smtp: accepted        {"msg_id":"a964e88e"}
Mar 29 11:05:34 sallie.librepush.net maddy[194975]: smtp: incoming message        {"msg_id":"","sender":"***@***","src_host":"***","src_ip":"***:32840"}
Mar 29 11:05:34 sallie.librepush.net maddy[194975]: smtp: RCPT ok        {"msg_id":"b7910aec","rcpt":"***@***"}
Mar 29 11:05:37 sallie.librepush.net maddy[194975]: check.spf: deferring action due to a DMARC policy        {"msg_id":"b7910aec"}
Mar 29 11:05:37 sallie.librepush.net maddy[194975]: smtp: accepted        {"msg_id":"b7910aec"}
Mar 29 11:10:17 sallie.librepush.net maddy[194975]: smtp: incoming message        {"msg_id":"","sender":"***@cwu.edu","src_host":"1334817421.cwu.edu","src_ip":"92.118.37.244:48554"}
Mar 29 11:10:17 sallie.librepush.net maddy[194975]: smtp: RCPT error        {"effective_rcpt":"***@outlook.com","msg_id":"ae94e9dd","rcpt":"***@outlook.com","reason":"reject directive used","smtp_code":550,"smtp_e>
Mar 29 11:10:17 sallie.librepush.net maddy[194975]: smtp: incoming message        {"msg_id":"","sender":"","src_host":"1334817421.cwu.edu","src_ip":"92.118.37.244:59978"}
Mar 29 11:10:17 sallie.librepush.net maddy[194975]: smtp: aborted        {"msg_id":"ae94e9dd"}
Mar 29 11:10:17 sallie.librepush.net maddy[194975]: smtp: MAIL FROM error (deferred)        {"check":"spf","msg_id":"9afc1094","rcpt":"***@outlook.com","reason":"Malformed address","smtp_code":550,"smtp_enchcode":"5.1.7","sm>
Mar 29 11:31:56 sallie.librepush.net maddy[194975]: submission/sasl: authentication failed        {"reason":"no auth. provider accepted creds, last err: dovecotsasl: authentication failed (code=)","src_ip":"***:16092","usern>
Mar 29 12:11:41 sallie.librepush.net maddy[194975]: smtp: incoming message        {"msg_id":"","sender":"***","src_host":"***","src_ip":"***>
(rest of log is 'smtp: incoming message' lines with e-mails that have not been delivered yet)

Configuration file

https://drop.q.tanukitsu.net/LqNrxmAP#maddy.conf

Environment information

• maddy version: maddy 0.4.3+5eec45e linux/amd64 go1.14.7
• 5eec45e

It's been running smoothly since I updated, so this is an infrequent bug.

Thank you.

[foxcpp]

Looks like rate limiting code is in need of a good stress testing. Alternatively, there may be something problematic in one of failure paths.
As a temporary workaround (while i try to figure out what might have caused this) you can comment out limits in your config (most likely the concurrency limit is the one to blame).

[foxcpp]

After looking through the code base, I noticed the concurrency semaphore is not being released correctly if MAIL FROM fails. In order to confirm whether it is the cause for your particular problem - are there many MAIL FROM error messages in your log?

[foxcpp]

Well, I see at least one such message in the log snippet you provided. I assume there is more.

[reivilibre]

After looking through the code base, I noticed the concurrency semaphore is not being released correctly if MAIL FROM fails. In order to confirm whether it is the cause for your particular problem - are there many MAIL FROM error messages in your log?

Seems like you may be spot on! I do get quite a lot of MAIL FROM error messages in the log, which probably add up over time.
(For what it's worth: The MAIL FROM errors occur a lot because of what look to be spammers, probably trying to see if the mail server is an open relay by the looks of it?)

[foxcpp]

I believe the referenced commit is the needed fix and it is included in 0.4.4 release. Closing.