Syntax error in BODYSTRUCTURE. Unexpected token: '('

[mantasaudickas]

Hello,

I am getting one more issue with BODYSTRUCTURE parsing.

See similar errors:

• Syntax error in BODYSTRUCTURE. Unexpected token: '(' #777
• Syntax error in BODYSTRUCTURE. Unexpected token: '(' #371

Here is the server log which I am receiving (its pretty big, so I think I too the one which is actually causing the issue (also tried to deanonymize it without breaking it)

C: Z00000053 UID FETCH 11889 (UID FLAGS ENVELOPE BODYSTRUCTURE BODY.PEEK[HEADER])
S: * 6285 FETCH (UID 11889 FLAGS (\Seen) ENVELOPE (\"Mon, 17 Nov 2014 13:29:22 +0100\" \"[undeliverable] Re: Adresy\" ((NIL NIL \"Mail Delivery System\" \"\")) ((NIL NIL \"Mail Delivery System\" \"\")) ((NIL NIL \"Mail Delivery System\" \"\")) ((NIL NIL \"e.username\" \"example.com\")) NIL NIL \"<D6173425-9D71-4D78-8C3C-1CEB03BCB4D0@example.com>\" \"<c0f0a134e5d7f943@one.example.com>\") BODYSTRUCTURE ((\"text\" \"plain\" (\"charset\" \"utf-8\") NIL NIL \"7bit\" 727 16 NIL NIL NIL NIL)(\"message\" \"delivery-status\" (\"name\" \"Delivery status\") NIL NIL \"7bit\" 416 NIL NIL NIL NIL)(\"message\" \"rfc822\" (\"name\" \"Message headers\") NIL NIL \"7bit\" 903 (\"Mon, 17 Nov 2014 13:29:21 +0100\" \"Re: Adresy\" ((\"username\" NIL \"e.username\" \"example.com\")) ((\"username\" NIL \"e.username\" \"example.com\")) ((\"username\" NIL \"e.username\" \"example.com\")) ((\"=?utf-8?Q?Justyna?=\" NIL \"salesde\" \"some-company.eu\")) ((NIL NIL \"saleseu\" \"some-company.eu\")(\"Bogdan\" NIL \"bogdan\" \"some-company.eu\")) NIL \"<004901d00260$35405970$9fc10c50$@some-company.eu>\" \"<D6173425-9D71-4D78-8C3C-1CEB03BCB4D0@example.com>\") (NIL \"alternative\" (\"boundary\" \"Apple-Mail=_352FCEEC-EB15-428F-9D8B-D3B4259DD646\") NIL NIL NIL) 17 NIL NIL NIL NIL) \"report\" (\"report-type\" \"delivery-status\" \"boundary\" \"_e0d7475d888f9882b71de053e5efb221_idea\") NIL NIL NIL) BODY[HEADER] {620}
S: Return-Path: <>
S: Received: from zero.example.com [11.11.11.111] (HELO zero.example.com)
S:  by one.example.com [79.96.20.237] with SMTP (IdeaSmtpServer v0.80)
S:  id d4236af797c2313f; Mon, 17 Nov 2014 13:29:22 +0100
S: From: Mail Delivery System
S: To: <user@example.com>
S: Subject: [undeliverable] Re: Adresy
S: Date: Mon, 17 Nov 2014 13:29:22 +0100
S: Message-ID: <c0f0a134e5d7f943@one.example.com>
S: In-Reply-To: <D6173425-9D71-4D78-8C3C-1CEB03BCB4D0@example.com>
S: MIME-Version: 1.0
S: Content-Type: multipart/report; report-type=delivery-status;
S: 	boundary=\"_e0d7475d888f9882b71de053e5efb221_idea\";
S: 
S: )
S: Z00000053 OK Completed

I am not expert in IMAP protocol, but seems to me that some brackets indeed are missing:

Any ideas, if this can be fixed, or maybe even message could be ignored somehow?
Maybe some rules can be loosened in some settings?

Using Dot Net Core 6.
Mailkit library: v3.2.0

[mantasaudickas]

Sorry, forgot to add exception stacktrace:

MailKit.Net.Imap.ImapProtocolException: Syntax error in BODYSTRUCTURE. Unexpected token: ')'
   at MailKit.Net.Imap.ImapEngine.ParseNumber(ImapToken token, Boolean nonZero, String format, Object[] args)
   at MailKit.Net.Imap.ImapUtils.ReadNumberAsync(ImapEngine engine, String format, Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapUtils.ParseBodyAsync(ImapEngine engine, String format, String path, Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapUtils.ParseBodyAsync(ImapEngine engine, String format, String path, Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapUtils.ParseMultipartAsync(ImapEngine engine, String format, String path, String subtype, Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapUtils.ParseBodyAsync(ImapEngine engine, String format, String path, Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapFolder.FetchSummaryItemsAsync(ImapEngine engine, MessageSummary message, Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapFolder.FetchSummaryItemsAsync(ImapEngine engine, ImapCommand ic, Int32 index, Boolean doAsync)
   at MailKit.Net.Imap.ImapEngine.ProcessUntaggedResponseAsync(Boolean doAsync, CancellationToken cancellationToken)
   at MailKit.Net.Imap.ImapCommand.StepAsync(Boolean doAsync)
   at MailKit.Net.Imap.ImapEngine.IterateAsync(Boolean doAsync)
   at MailKit.Net.Imap.ImapEngine.RunAsync(ImapCommand ic, Boolean doAsync)
   at MailKit.Net.Imap.ImapFolder.FetchAsync(IList`1 uids, IFetchRequest request, Boolean doAsync, CancellationToken cancellationToken)

[jstedfast]

What IMAP server is this? Can you paste the first line of the log ("* OK ... IMAP server ready")?

I might want to enable a QuirksMode for this server.

[mantasaudickas]

* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
N00000000 AUTHENTICATE PLAIN ********
N00000000 OK [CAPABILITY IMAP4rev1 SASL-IR LITERAL+ CHILDREN I18NLEVEL=1 ID IDLE SORT SORT=DISPLAY UIDPLUS UNSELECT QUOTA MOVE WITHIN LIST-EXTENDED LIST-STATUS SPECIAL-USE XLIST STARTTLS] Completed

[mantasaudickas]

However I do not have access to this server. Its a client which is using our software and has email integration enabled.

[jstedfast]

This comment is as much for myself as you or anyone else reading this.

BODYSTRUCTURE ((\"text\" \"plain\" (\"charset\" \"utf-8\") NIL NIL \"7bit\" 727 16 NIL NIL NIL NIL)(\"message\" \"delivery-status\" (\"name\" \"Delivery status\") NIL NIL \"7bit\" 416 NIL NIL NIL NIL)(\"message\" \"rfc822\" (\"name\" \"Message headers\") NIL NIL \"7bit\" 903 (\"Mon, 17 Nov 2014 13:29:21 +0100\" \"Re: Adresy\" ((\"username\" NIL \"e.username\" \"example.com\")) ((\"username\" NIL \"e.username\" \"example.com\")) ((\"username\" NIL \"e.username\" \"example.com\")) ((\"=?utf-8?Q?Justyna?=\" NIL \"salesde\" \"some-company.eu\")) ((NIL NIL \"saleseu\" \"some-company.eu\")(\"Bogdan\" NIL \"bogdan\" \"some-company.eu\")) NIL \"<004901d00260$35405970$9fc10c50$@some-company.eu>\" \"<D6173425-9D71-4D78-8C3C-1CEB03BCB4D0@example.com>\") (NIL \"alternative\" (\"boundary\" \"Apple-Mail=_352FCEEC-EB15-428F-9D8B-D3B4259DD646\") NIL NIL NIL) 17 NIL NIL NIL NIL) \"report\" (\"report-type\" \"delivery-status\" \"boundary\" \"_e0d7475d888f9882b71de053e5efb221_idea\") NIL NIL NIL)

Of that, this is what is causing the problem:

(NIL \"alternative\" (\"boundary\" \"Apple-Mail=_352FCEEC-EB15-428F-9D8B-D3B4259DD646\") NIL NIL NIL) 17 NIL NIL NIL NIL)

MailKit was trying to parse that as a body-type-1part even though it's supposed to be a body-type-mpart.

Syntax:

body            = "(" (body-type-1part / body-type-mpart) ")"

body-type-1part = (body-type-basic / body-type-msg / body-type-text)
                  [SP body-ext-1part]

body-type-basic = media-basic SP body-fields

body-type-mpart = 1*body SP media-subtype
                  [SP body-ext-mpart]

body-type-msg   = media-message SP body-fields SP envelope
                  SP body SP body-fld-lines

body-type-text  = media-text SP body-fields SP body-fld-lines
                    ; MESSAGE subtype MUST NOT be "rfc822"

media-basic     = ((DQUOTE ("APPLICATION" / "AUDIO" / "IMAGE" /
                  "MESSAGE" / "VIDEO") DQUOTE) / string) SP
                  media-subtype
                    ; Defined in [MIME-IMT]

media-message   = DQUOTE "MESSAGE" DQUOTE SP DQUOTE "[RFC822](https://datatracker.ietf.org/doc/html/rfc822)" DQUOTE
                    ; Defined in [MIME-IMT]

media-subtype   = string
                    ; Defined in [MIME-IMT]

media-text      = DQUOTE "TEXT" DQUOTE SP media-subtype
                    ; Defined in [MIME-IMT]

The issue here is that the way body-type-mpart is defined, it cannot have a NIL list of body tokens (children) and so we proceed to parse the NIL as if it is the first part of a media-basic.

What the IMAP server should probably do is this: (() "alternative" ("boundary", ...

Anyway, I've worked around the issue for now. I might have to re-think some of the parser code to handle this (but mostly other cases) cleaner, but for now, the work-around should work and I have added this to my unit tests to avoid breaking this scenario in case I do take a stab at re-working the BODYSTRUCTURE parser.

[jstedfast]

Hah! Turns out MailKit's own BodyPart.ToString() method uses NIL for an empty list of multipart children as well.

And taking a closer look at the specs, doing (() "alternative" ("boundary", ... is wrong because the () would be a broken body-type-1part or body-type-mpart child, so that's also a no-go.

I think that Dovecot is probably doing the only sane thing it can do in this case.

[jstedfast]

Okay, redid the BODYSTRUCTURE parser to make it a tad cleaner for this scenario.

[mantasaudickas]

That's an amazing quick fix!
When do you plan to upload it to nuget?

[jstedfast]

I just pushed v3.3.0. I'd been planning to make a release today so your bug report made it just in time :)