Sending to Postal SMTP fails in MailKit 3.4.2

[haarhoff-frs]

Describe the bug
I'm trying to send emails to Postal (self-hosts SES alternative, more or less) via SMTP. This worked without a problem in MailKit 3.4.1 but stopped working in 3.4.2. On the server I can see the EHLO, but the connection is closed afterwards. The behaviour is the same with and without TLS enabled. I'd be happy to provide credentials for a test-organization in a private message/email.

Platform (please complete the following information):

• OS: Windows 11, Windows Server 2019
• .NET Runtime: CoreCLR
• .NET Framework: .NET 7.0
• MailKit Version: 3.4.2

Exception

Unhandled exception. MailKit.Net.Smtp.SmtpProtocolException: Unable to parse status code returned by the server.
   at MailKit.Net.Smtp.SmtpStream.ReadResponse(ByteArrayBuilder builder, Boolean& newLine, Boolean& more, Int32& code)
   at MailKit.Net.Smtp.SmtpStream.ReadResponseAsync(CancellationToken cancellationToken)
   at MailKit.Net.Smtp.SmtpStream.SendCommandAsync(String command, CancellationToken cancellationToken)
   at MailKit.Net.Smtp.SmtpClient.EhloAsync(CancellationToken cancellationToken)
   at MailKit.Net.Smtp.SmtpClient.ConnectAsync(String host, Int32 port, SecureSocketOptions options, CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in C:\Users\stipps\RiderProjects\MailTester\MailTester\Program.cs:line 15
   at Program.<Main>(String[] args)

To Reproduce
Steps to reproduce the behavior:

1. Setup Postal instance (https://docs.postalserver.io/install/installation)
2. Create .NET console application
3. Add minimal code to send email over postal smtp
4. See error

Expected behavior
MailKit connects to the server and sends the email.

Code Snippets
If applicable, add code snippets to help explain your problem.

using MailKit;
using MailKit.Net.Smtp;
using MailKit.Security;
using MimeKit;

var msg = new MimeMessage();
msg.From.Add(InternetAddress.Parse("hello@mydomain.com"));
msg.To.Add(InternetAddress.Parse("frs@yourdomain.com"));
msg.Subject = "Ihre Bestellung im Shop";

var bb = new BodyBuilder();
bb.TextBody = "Vielen Dank fuer Ihre Bestellung!";
msg.Body = bb.ToMessageBody();

var client = new SmtpClient();
await client.ConnectAsync("mailer.mydomain.com", 25, SecureSocketOptions.Auto);

await client.AuthenticateAsync("myorg/mydomain", "mypassword");
await client.SendAsync(msg);
await client.DisconnectAsync(true);

Protocol Logs

Connected to smtp://mailer.mydomain.com:25/?starttls=when-available
S: 220 mailer.haarhoff.eu ESMTP Postal/DPNZMRC: EHLO TICK

After this the exception is thrown.

Postal SMTP log:

postal-smtp-1  | [9] [2022-11-18T07:46:55.818] DEBUG -- : [YWMLLX]    Connection opened from ::ffff:176.199.38.70
postal-smtp-1  | [9] [2022-11-18T07:46:55.818] DEBUG -- : [YWMLLX]    Client identified as ::ffff:176.199.38.70
postal-smtp-1  | [9] [2022-11-18T07:46:55.839] DEBUG -- : [YWMLLX] <= EHLO TICK
postal-smtp-1  | [9] [2022-11-18T07:46:55.843] DEBUG -- : [YWMLLX] => 250-My capabilities are
postal-smtp-1  | [9] [2022-11-18T07:46:55.843] DEBUG -- : [YWMLLX] => 250-STARTTLS
postal-smtp-1  | [9] [2022-11-18T07:46:55.843] DEBUG -- : [YWMLLX] => 250 AUTH CRAM-MD5 PLAIN LOGIN
postal-smtp-1  | [9] [2022-11-18T07:46:55.856] DEBUG -- : [YWMLLX]    Connection closed

[jstedfast]

I suspect this has something to do with commit 39df387

[haarhoff-frs]

I stepped through the Connect method on SmtpClient, which let me connect without a problem. As soon as I removed the breakpoints, the initially described error occurs. So maybe this is some kind of timing issue? I'll try to find time to get a better grasp of the code in the next days.

[jstedfast]

Most likely your SMTP server is responding to the EHLO command with multiple TCP/IP packets (1 per line?) and that is somehow breaking the response parser in SmtpStream.cs

When you set breakpoints in that code, it delays SmtpStream.ReadAhead/Async() from reading data from the socket, allowing follow-up TCP/IP packets to arrive before it does a read/async, thus eliminating the issue.

[jstedfast]

I wrote a unit test to test that theory the other day (if you are interested in testing the scenario, maybe I should just commit it) but I haven't had time to investigate further because for some reason my Visual Studio for Mac's ability to run the unit tests is broken, so I'll need to switch to my Windows box to run the tests under the debugger.

[haarhoff-frs]

Thanks! I had a quick look at the unit test. The test currently fails, but with this message:

Capabilities
  Expected: None
  But was:  Size, EnhancedStatusCodes, Authentication, EightBitMime, StartTLS

If I have a look at the resource file, the capabilities found are exactly the ones that should be there, so I'm not sure why the Assert looks for None. Apart from that the test runs through without a problem.

[jstedfast]

The Assert in the test is just wrong.

[haarhoff-frs]

All right, no worries. I'll see what I can find out.
Thanks again for taking the time!

[jstedfast]

The above patch may resolve the bug. Let me know?

[haarhoff-frs]

Looks good, all my tests with our Postal server are now able to connect and send using that latest commit. Thank you!

[jstedfast]

Awesome, thanks for testing my changes! I'll try to make a new 3.4.3 release by early next week (after the holidays).