NOOP after CLOSE: Message counter not updated

[ThomasContrib]

Hi Jeffrey

Thanks for providing your great library. I'm not sure if what I write below is a bug. As it may help, I report it anyway.

Best regards,
Thomas

Description of potential bug

• The inbox contains one message.
• Two clients select the inbox.
• Client 1 deletes the only message in the inbox (sets delete flag).
• Client 1 closes the inbox (CLOSE command).
• Client 2 performs a NOOP command and receives an EXPUNGE response.

Now I would expect the message counter of client 2 (folder.Count) to be 0:

• Expected folder.Count: 0 (with MailKit 2.15.0, 3.0.0-preview1)
• Actual folder.Count: 1 (with MailKit 3.0.0, 3.5.0)

Log file of client 2

Connected to imap://testserver.com:143/?starttls=when-available
S: * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot (Debian) ready.
C: B00000000 STARTTLS
S: B00000000 OK Begin TLS negotiation now.
C: B00000001 CAPABILITY
S: * CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN
S: B00000001 OK Pre-login capabilities listed, post-login capabilities have more.
C: B00000002 AUTHENTICATE PLAIN AHdwMTIyMjA0NjEtOABhcWtIanNsdw==
S: B00000002 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY QUOTA] Logged in
C: B00000003 NAMESPACE
S: * NAMESPACE (("" ".")) NIL NIL
S: B00000003 OK Namespace completed (0.001 + 0.000 secs).
C: B00000004 LIST "" "INBOX" RETURN (SUBSCRIBED CHILDREN)
S: * LIST (\Subscribed \HasNoChildren) "." INBOX
S: B00000004 OK List completed (0.001 + 0.000 secs).
C: B00000005 SELECT INBOX (CONDSTORE)
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
S: * 1 EXISTS
S: * 0 RECENT
S: * OK [UNSEEN 1] First unseen.
S: * OK [UIDVALIDITY 1638865376] UIDs valid
S: * OK [UIDNEXT 10] Predicted next UID
S: * OK [HIGHESTMODSEQ 34] Highest
S: B00000005 OK [READ-WRITE] Select completed (0.001 + 0.000 secs).
C: B00000006 NOOP
S: * 1 EXPUNGE
S: B00000006 OK NOOP completed (0.001 + 0.000 secs).

Code to reproduce

using MailKit;
using MailKit.Net.Imap;
using MailKit.Search;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MimeKit;
using System.IO;

namespace ImapTests
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            FileInfo logFile1 = new("imapTest1.log");
            FileInfo logFile2 = new("imapTest2.log");

            string host = "testserver.com";
            string userName = "testuser";
            string password = "testpass";
            int port = 143;
            bool useTLS = false;

            var client1 = new ImapClient(
                new ProtocolLogger(logFile1.FullName, append: false));
            client1.Connect(host, port, useTLS);
            client1.Authenticate(userName, password);

            var client2 = new ImapClient(
                new ProtocolLogger(logFile2.FullName, append: false));
            client2.Connect(host, port, useTLS);
            client2.Authenticate(userName, password);

            // Performs LIST commands
            IMailFolder folder = client1.GetFolder("INBOX");
            IMailFolder folder2 = client2.GetFolder("INBOX");

            // Performs SELECT command
            folder.Open(MailKit.FolderAccess.ReadWrite);

            // Inbox should be empty before adding a message
            Assert.AreEqual(0, folder.Count);

            MimeMessage message = new(
                from: new[] { new MailboxAddress("A. Sender", "sender@test.com") }, 
                to: new[] { new MailboxAddress("B. Recipient", "recipient@test.com") },
                subject: "Test message", 
                body: new TextPart("Test text"));

            // Performs APPEND command
            folder.Append(message);

            // Performs SELECT command
            folder2.Open(MailKit.FolderAccess.ReadWrite);

            // The appended message should be visible for both clients
            Assert.AreEqual(1, folder.Count);
            Assert.AreEqual(1, folder2.Count);

            // Performs UID SEARCH command
            var uids = folder.Search(SearchQuery.All);

            // Performs STORE command (0 means message sequence number 1)
            folder.AddFlags(uids[0], MessageFlags.Deleted, silent: true);

            Assert.AreEqual(1, folder.Count);
            Assert.AreEqual(1, folder2.Count);

            // Performs CLOSE command
            folder.Close(expunge: true);

            // Performs NOOP command
            client2.NoOp();

            // Passes with: MilKit 2.0.15, 3.0.0-preview1
            // Fails with: MilKit 3.0.0, 3.5.0
            Assert.AreEqual(0, folder2.Count);
        }
    }
}

[jstedfast]

It's a bug, but it's a bug in the server for not sending an untagged * 0 EXISTS response.

[ThomasContrib]

Where would you expect the untagged * 0 EXISTS? As response to the NOOP command of client 2?

https://www.rfc-editor.org/rfc/rfc3501#section-7.4.1

The EXPUNGE response also decrements the number of messages in the mailbox; it is not necessary to send an EXISTS response with the new value.

[jstedfast]

Ah, right, I had the code correct at one point but changed it to mimic (exactly) the events that the server emitted.

So the question now is, should ImapFolder emit the CountChanged event as a result of an EXPUNGE notification?

And if so, after each EXPUNGE notification? Or only at the end, the way IMAP would normally do it?

The tricky part is not doing it on our own and then doing it again if the IMAP server responds with an EXISTS, because then we'd be duplicating events.

[jstedfast]

Turns out it wasn't that tricky...

Please try the above fix when it finishes building (it will upload a new nuget package to https://www.myget.org/feed/mimekit/package/nuget/MailKit). I suspect the build number will be 586.

[jstedfast]

Actually, my fix wasn't quite right but build 587 should work.

[ThomasContrib]

Build 586 and 587 both work correctly - thanks a lot.