Syntax error in FLAGS. Unexpected token: '['

[AndrewZenith]

Getting this error logging onto one account on a server. Other accounts seem fine. Mailkit 1.4.2.1.

Account.Client = new ImapClient(new ProtocolLogger(Console.OpenStandardOutput()));
try
{
Account.Client.Connect(new Uri(account.address));

if (Account.Client.IsConnected)
{
    Account.Client.Authenticate(new NetworkCredential(account.UserName, account.Password));
    Account.Client.Inbox.Open(FolderAccess.ReadOnly);  /exception here
}

}

S: * OK Dovecot ready.
C: A00000000 CAPABILITY
S: * CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS STARTTLS AUTH=PLAIN
S: A00000000 OK Capability completed.
C: A00000001 STARTTLS
S: A00000001 OK Begin TLS negotiation now.
C: A00000002 CAPABILITY
S: * CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS AUTH=PLAIN
S: A00000002 OK Capability completed.
C: A00000003 AUTHENTICATE PLAIN ********************
S: A00000003 OK Logged in.
C: A00000004 CAPABILITY
S: * CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS
S: A00000004 OK Capability completed.
C: A00000005 NAMESPACE
S: * NAMESPACE (("" "/")) NIL NIL
S: A00000005 OK Namespace completed.
C: A00000006 LIST "" "INBOX"
S: * LIST (\NoInferiors \UnMarked) "/" "INBOX"
S: A00000006 OK List completed.
The thread 0x43e8 has exited with code 0 (0x0).
The thread 0x42a0 has exited with code 0 (0x0).
The thread 0x2230 has exited with code 0 (0x0).
Exception thrown: 'MailKit.Net.Imap.ImapProtocolException' in MailKit.dll
C: A00000007 EXAMINE INBOX
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk $Forwarded $MDNSent Junk $label4 $label1 $has_cal $label2 $label3 $label5 [all_2071] [asl_107] [dhl_150] [pds_2666])
S: * OK [PERMANENTFLAGS ()] Read-only mailbox.
S: * 971 EXISTS
S: * 0 RECENT
S: * OK [UIDVALIDITY 1266935373] UIDs valid
S: * OK [UIDNEXT 126629] Predicted next UID
S: A00000007 OK [READ-ONLY] Select completed.
The thread 0x5df4 has exited with code 0 (0x0).
The thread 0x52dc has exited with code 0 (0x0).
The thread 0x5b70 has exited with code 0 (0x0).

Exception Message:
Syntax error in FLAGS. Unexpected token: '['
Exception Stacktrace:
at MailKit.Net.Imap.ImapUtils.ParseFlagsList(ImapEngine engine, String name, HashSet`1 userFlags, CancellationToken cancellationToken)
at MailKit.Net.Imap.ImapEngine.ProcessUntaggedResponse(CancellationToken cancellationToken)
at MailKit.Net.Imap.ImapCommand.Step()
at MailKit.Net.Imap.ImapEngine.Iterate()
at MailKit.Net.Imap.ImapEngine.Wait(ImapCommand ic)
at MailKit.Net.Imap.ImapFolder.Open(FolderAccess access, CancellationToken cancellationToken)
at FCL.ForwardOffice.Hub.ModuleCore.EmailListViewModel.FillFolders(UserDataEmailAccountsXpo account) in E:\Documents\Visual Studio 2015\Projects\74B\ForwardOfficeHub\ForwardOfficeHub\Hub\ModuleCore\EmailListViewModel.cs:line 402

[jstedfast]

The problem is that [all_2071] [asl_107] [dhl_150] [pds_2666] are illegal flag names.

Here is the definition of the flag syntax:

atom            = 1*ATOM-CHAR

ATOM-CHAR       = <any CHAR except atom-specials>

atom-specials   = "(" / ")" / "{" / SP / CTL / list-wildcards /
                  quoted-specials / resp-specials

flag            = "\Answered" / "\Flagged" / "\Deleted" /
                  "\Seen" / "\Draft" / flag-keyword / flag-extension
                    ; Does not include "\Recent"

flag-extension  = "\" atom
                    ; Future expansion.  Client implementations
                    ; MUST accept flag-extension flags.  Server
                    ; implementations MUST NOT generate
                    ; flag-extension flags except as defined by
                    ; future standard or standards-track
                    ; revisions of this specification.

flag-fetch      = flag / "\Recent"

flag-keyword    = atom

flag-list       = "(" [flag *(SP flag)] ")"

flag-perm       = flag / "\*"

list-wildcards  = "%" / "*"

quoted-specials = DQUOTE / "\"

resp-specials   = "]"

So technically, MailKit should allow the "[" character, but the "]" is still illegal to use in flag names.

What client created those flag names? Or did you create them yourself?

[AndrewZenith]

I've got the user in question investigating. We think it's Thunderbird and favouriting searches, but hopefully he'll work it out so I can give a bit more information.

[AndrewZenith]

He's not found out yet, but it can only be Thunderbird that has put them there as he only uses that. I've asked him to carry on trying an thinking about it because it must be something he's done relatively recently in Thunderbird that put them there.

[jstedfast]

I've been trying to figure out how a client could have created those custom flags because I cannot get Dovecot to accept them:

A5 STORE 1 +FLAGS.SILENT ([custom-flag])
A5 NO [CANNOT] Invalid characters in keyword (0.000 + 0.000 secs).
A6 STORE 1 +FLAGS.SILENT ("[custom-flag]")
A6 BAD Error in IMAP command STORE: Flags list contains non-atoms (0.000 + 0.000 secs).
A7 STORE 1 FLAGS.SILENT (\Flagged [all_2071])
A7 NO [CANNOT] Invalid characters in keyword (0.000 + 0.000 secs).
A8 STORE 1 FLAGS.SILENT ("[all_2071]")
A8 BAD Error in IMAP command STORE: Flags list contains non-atoms (0.000 + 0.000 secs).

[jstedfast]

I can get it to accept "[", but not "]":

A9 STORE 1 FLAGS.SILENT ([all_2071)
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft [all_2071)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft [all_2071 \*)] Flags permitted.
A9 OK Store completed (0.008 + 0.000 secs).

[AndrewZenith]

We've got it.
In Thunderbird add a new tag (tags, manage, add) - it lets you put in for instance "[abcd]".
Then add this tag to an email.
At this point the flags list now has this tag in it.

[jstedfast]

Thanks, that seems to create the custom tag on dovecot for me, but only if the tag name I type in doesn't contain "]". Perhaps your installed version of dovecot is accepting "]"?

Anyway... I've fixed MailKit to allow "[" like the specification calls for, but it won't accept "]".

The version of dovecot that I have installed is 2.2.24 (2.2.25 seems to be the latest).

[AndrewZenith]

OK, I've discovered that our version of dovecot is on Linux RHEL 5.4, and.... wait for it.... it's version 1.0.7.9.
It lets us put a "]" in.
I'll say no more about that at this point!
Thanks for the fix!

[jstedfast]

I would recommend not using ]'s at least.

This documentation looks promising for how to delete custom created flag names: http://wiki2.dovecot.org/Tools/Doveadm/Flags, but it requires admin rights on the server.

Without that, I'm not sure how to remove flag names with "]"'s.

You could customize MailKit to allow "]" by modifying https://github.com/jstedfast/MailKit/blob/master/MailKit/Net/Imap/ImapUtils.cs#L1137

By setting specials = ImapStream.GMailLabelSpecials;" and/or by replacing AtomSpecials with a custom string that doesn't include []'s.

[jstedfast]

I've released MailKit 1.6.0 with this fix

[AndrewZenith]

Thanks!