MailKit.Net.Imap.ImapProtocolException : 'Syntax error in BODY. Unexpected token: ')''

[bleece]

Describe the bug
When I try to fetch messages "client.Inbox.Fetch(startIndex, -1, request, cancel.Token);" I got error

Platform (please complete the following information):
Microsoft.NETCore.App.Ref\7.0.20

Exception
à MailKit.Net.Imap.ImapUtils.ReadStringToken(ImapEngine engine, String format, CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapUtils.ParseContentType(ImapEngine engine, String format, CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapUtils.ParseBody(ImapEngine engine, String format, String path, CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapUtils.ParseBody(ImapEngine engine, String format, String path, CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapUtils.ParseMultipart(ImapEngine engine, String format, String path, CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapFolder.ParseSummaryItems(ImapEngine engine, MessageSummary message, FetchSummaryItemsCompletedCallback completed, CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapFolder.UntaggedFetchSummaryItemsHandler(ImapEngine engine, ImapCommand ic, Int32 index, Boolean doAsync)
à MailKit.Net.Imap.ImapEngine.ProcessUntaggedResponse(CancellationToken cancellationToken)
à MailKit.Net.Imap.ImapCommand.Step()
à MailKit.Net.Imap.ImapEngine.Iterate()
à MailKit.Net.Imap.ImapEngine.Run(ImapCommand ic)
à MailKit.Net.Imap.ImapFolder.Fetch(Int32 min, Int32 max, IFetchRequest request, CancellationToken cancellationToken)
à AlloVoisins.IdleClient.d__26.MoveNext() dans **\IdleClient.cs :ligne 95

To Reproduce
Steps to reproduce the behavior:
IList? fetched = null;
fetched = client.Inbox.Fetch(startIndex, -1, request, cancel.Token);

Expected behavior
Get all messages

client = new ImapClient(new ProtocolLogger(Console.OpenStandardError()));

     async Task ReconnectAsync()
     {
         if (!client.IsConnected)
             await client.ConnectAsync(_host, _port, sslOptions, cancel.Token);

         if (!client.IsAuthenticated)
         {
             await client.AuthenticateAsync(_gmailUsername, _gmailPassword, cancel.Token);
             await client.Inbox.OpenAsync(FolderAccess.ReadOnly, cancel.Token);
         }
     }


 IList<IMessageSummary>? fetched = null;
 fetched = client.Inbox.Fetch(startIndex, -1, request, cancel.Token);

[jstedfast]

I'm going to need the IMAP protocol log (which was dumped to the console - just copy & paste it here)

[bleece]

I'm going to need the IMAP protocol log (which was dumped to the console - just copy & paste it here)

Hello,
I just anonymized the confidential information

Connected to imaps://imap.gmail.com:993/
S: * OK Gimap ready for requests from 2a01:e0a:130:47d0:44b1:9cce:67e9:5b34 u9mb76462483wmq
C: B00000000 CAPABILITY
S: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN AUTH=OAUTHBEARER
S: B00000000 OK Thats all she wrote! u9mb76462483wmq
C: B00000001 AUTHENTICATE PLAIN ********
S: * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584
S: B00000001 OK ---.---87@gmail.com authenticated (Success)
C: B00000002 NAMESPACE
S: * NAMESPACE (("" "/")) NIL NIL
S: B00000002 OK Success
C: B00000003 LIST "" "INBOX" RETURN (SUBSCRIBED CHILDREN)
S: * LIST (\HasNoChildren \Subscribed) "/" "INBOX"
S: B00000003 OK Success
C: B00000004 LIST (SPECIAL-USE) "" "*" RETURN (SUBSCRIBED CHILDREN)
S: * LIST (\Drafts \HasNoChildren \Subscribed) "/" "[Gmail]/Brouillons"
S: * LIST (\HasNoChildren \Subscribed \Trash) "/" "[Gmail]/Corbeille"
S: * LIST (\HasNoChildren \Sent \Subscribed) "/" "[Gmail]/Messages envoy&AOk-s"
S: * LIST (\HasNoChildren \Junk \Subscribed) "/" "[Gmail]/Spam"
S: * LIST (\Flagged \HasNoChildren \Subscribed) "/" "[Gmail]/Suivis"
S: * LIST (\All \HasNoChildren \Subscribed) "/" "[Gmail]/Tous les messages"
S: B00000004 OK Success
C: B00000005 LIST "" "[Gmail]" RETURN (SUBSCRIBED CHILDREN)
S: * LIST (\HasChildren \NonExistent \Subscribed) "/" "[Gmail]"
S: B00000005 OK Success
C: B00000006 EXAMINE INBOX (CONDSTORE)
S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $Forwarded $Junk $NotJunk $NotPhishing $Phishing JunkRecorded NotJunk)
S: * OK [PERMANENTFLAGS ()] Flags permitted.
S: * OK [UIDVALIDITY 1] UIDs valid.
S: * 4615 EXISTS
S: * 0 RECENT
S: * OK [UIDNEXT 7821] Predicted next UID.
S: * OK [HIGHESTMODSEQ 814976]
S: B00000006 OK [READ-ONLY] INBOX selected. (Success)
C: B00000007 FETCH 1:* (UID FLAGS INTERNALDATE RFC822.SIZE ENVELOPE BODY)
S: * 52 FETCH (UID 117 RFC822.SIZE 66672 MODSEQ (19066) INTERNALDATE "04-Jan-2015 21:17:33 +0000" FLAGS ($NotJunk NotJunk \Seen) ENVELOPE ("Sun, 4 Jan 2015 22:15:26 +0100" "Re: MP 2/3" (("--- ---" NIL "---.---87" "gmail.com")) (("--- ---" NIL "---.---87" "gmail.com")) (("--- ---" NIL "---.---87" "gmail.com")) (("--- ---" NIL "---.---87" "gmail.com")) NIL NIL "<D4A31488-8A32-4DC3-AC6C-984054129154@gmail.com>" "<287586B1-4EFD-4D2F-8118-2E615F0D7374@gmail.com>") BODY (("TEXT" "PLAIN" ("CHARSET" "windows-1252") NIL NIL "QUOTED-PRINTABLE" 2117 63)(("TEXT" "HTML" ("CHARSET" "us-ascii") NIL NIL "7BIT" 147 0)("APPLICATION" "PDF" ("NAME" "IR-Justif-91-2013-14940541938365.pdf") NIL NIL "BASE64" 53522)("TEXT" "HTML" ("CHARSET" "windows-1252") NIL NIL "QUOTED-PRINTABLE" 9175 137) "MIXED") "ALTERNATIVE"))
S: * 53 FETCH (UID 118 RFC822.SIZE 10416306 MODSEQ (19150) INTERNALDATE "04-Jan-2015 21:18:59 +0000" FLAGS ($NotJunk NotJunk \Seen) ENVELOPE ("Sun, 4 Jan 2015 22:15:40 +0100" "Re: MP 3/3" (("--- ---" NIL "---.---87" "gmail.com")) (("--- ---" NIL "---.---87" "gmail.com")) (("--- ---" NIL "---.---87" "gmail.com")) (("--- ---" NIL "---.---87" "gmail.com")) NIL NIL "<D4A31488-8A32-4DC3-AC6C-984054129154@gmail.com>" "<35C4999E-DE9E-4B82-9D8C-982FEA73DAAF@gmail.com>") BODY (("TEXT" "PLAIN" ("CHARSE

[jstedfast]

I probably won't be able to get to this this week, but I should have time this weekend/next week to dig into this.

I suspect the issue is with: "ALTERNATIVE"))

MailKit likely expects more tokens after the "ALTERNATIVE" but before the ).

I have to verify the syntax, but I'm pretty sure that the syntax specifies that there must be a few more tokens there such as the Content-Type parameters and some of the other Content-* header values (Location, Language, etc).

[bleece]

I probably won't be able to get to this this week, but I should have time this weekend/next week to dig into this.

I suspect the issue is with: "ALTERNATIVE"))

MailKit likely expects more tokens after the "ALTERNATIVE" but before the ).

I have to verify the syntax, but I'm pretty sure that the syntax specifies that there must be a few more tokens there such as the Content-Type parameters and some of the other Content-* header values (Location, Language, etc).

Ok, thanks for your feedback. I look forward to hearing from you. Have a good evening.

[jstedfast]

What version of MailKit are you using? I am not able to reproduce this.

[bleece]

What version of MailKit are you using? I am not able to reproduce this.

I am using 4.8.0

[jstedfast]

Thanks. I'll try taking another look at this.

[jstedfast]

Does this happen every time?

What if you change your requested items to just MessageSummaryItems.Body? Does that still fail?

I'm wondering if there's a buffering issue because I still can't reproduce this.

[bleece]

Does this happen every time?
> Yes it happens every time
What if you change your requested items to just MessageSummaryItems.Body? Does that still fail?
> Same result
I'm wondering if there's a buffering issue because I still can't reproduce this.
> Too many messages to load?

[jstedfast]

If I feed your log into my unit test, it parses the BODY response just fine. No exceptions.

[bleece]

If I feed your log into my unit test, it parses the BODY response just fine. No exceptions.

my last log :
imap-th10-20h56.log

[jstedfast]

I can repro now - I'll look at debugging it this weekend.

[bleece]

I can repro now - I'll look at debugging it this weekend.

Great, thanks!

[jstedfast]

I found the issue:

* 51 FETCH (MODSEQ (18812) BODY (("TEXT" "PLAIN" ("CHARSET" "windows-1252") NIL NIL "QUOTED-PRINTABLE" 211 5)("MESSAGE" "DELIVERY-STATUS" NIL NIL NIL "7BIT" 344)("MESSAGE" "RFC822" NIL NIL NIL "7BIT" 2942 ("Sat, 3 Jan 2015 19:26:15 +0100" "Re: MPG" (("c p" NIL "---.---87" "gmail.com")) (("c p" NIL "---.---87" "gmail.com")) (("c p" NIL "---.---87" "gmail.com")) (("am sa[cft]" NIL "sm" "cft.ce.fr")) NIL NIL "<D3CAD328B4CFD24E83DD07E8923DFEE902FA8F1B@PRDGCE8P4523.sigce.ce.fr>" "<D4A-8A32-4DC3-AC6C-98429154@gmail.com>") ("ALTERNATIVE") 51) "REPORT"))

Specifically, this: ("ALTERNATIVE")

Now comes the hard part in trying to figure out how to interpret that because it's nonsensical. It's clearly meant to be an empty multipart/alternative, but the syntax should probably be something more like (NIL "ALTERNATIVE") where the NIL means it has no children.

I wonder if you could do inbox.GetMessage(50) (which maps to IMAP index 51) and see what that message structure looks like. You can anonymize pretty much everything other than the Content-Type headers of the message structure (and even remove the base64 attachment content and/or body text content). I am just trying to figure out what is causing the GMail IMAP server to generate such a thing.

[jstedfast]

Another thing to try, in the meantime, would be to use MessageSummaryItems.BodyStructure instead of MessageSummaryItems.Body.

They more-or-less give you the exact same info (BodyStructure gives you slightly more info), but sometimes IMAP servers use different implementations and it's possible that the BODYSTRUCTURE response will be syntactically valid while the BODY response is not.

Just something worth trying.

[jstedfast]

Looks like I already had a work-around for GMail's BODYSTRUCTURE response that did something similar.

[bleece]

Looks like I already had a work-around for GMail's BODYSTRUCTURE response that did something similar.

Thanks. I'll watch this weekend.

[jstedfast]

Release might get delayed a few days by a last-minute MimeKit feature that needs a bit of testing. Will update you in the comments when v4.9.0 goes out.

[jstedfast]

Released v4.9.0