-
-
Notifications
You must be signed in to change notification settings - Fork 828
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Search returns [BADCHARSET (US-ASCII)] when using on Outlook IMAP server #808
Comments
What that error is telling you is that Hotmail only supports US-ASCII strings, so you will not be able to search for that string. |
It seems that the US-ASCII is the offending charset; moreover i have no way to specify a charset when invoking folder.Search(). It seems that is automatically set as UTF-8 in BuildQueryExpression() and BuildQuery() methods of ImapFolderSearch.cs. I think that the user must be able to search for arbitrary text.. |
No, you are misreading the error. The BADCHARSET response-code provides a list of supported charsets, not the charset that is invalid. The charset that is invalid is the charset used in the SEARCH command (which was UTF-8). You CANNOT search for the string you are trying to search for because the server does not support any charset other than US-ASCII and the string you are trying to searching for is not US-ASCII. |
Ok you are right, but this requires that i beforehand change the text passed to the Search() method; is not there a way to force the charset parameter? |
What would you force it to? |
Avoiding |
That won't work. |
Funnily enough I encountered the same issue a few days prior to this. Seems like Exchange servers don't like UTF-8 at all. It is also worth noting that some servers who are able to interpret UTF-8 queries will still fail when calling imap.EnableUTF8(). I'm not sure what the purpose of that method is. So checking for failure on it isn't a good technique. My solution was to build a more complex search expression using And- and Or-Operators by checking for substrings surrounded by non Ascii characters. Microsoft ftw |
@wartab the UTF-8=ONLY and UTF-8=ACCEPT extensions (which is what
I'll double-check to make sure that MailKit conforms to that last bit (but I think it does). I don't think Exchange servers advertise the UTF8=ACCEPT or =ONLY extensions, so that's probably why EnableUTF8() failed for you? |
It was not an Exchange server. If I remember correctly it happened on an OVH mail server (the exchange server threw on both EnableUTF8() and when the search query was sent). The method EnableUTF8() threw an exception but the Search didn't fail and actually sent the correct result. If you want more details, I can provide them. |
EnableUTF8() should only throw if the server doesn't support the UTF8 extension (which is different from supporting UTF-8 in SEARCH - a server can support UTF-8 in SEARCH w/o supporting the |
Fair enough, thanks for the info :) |
Well, sorry, but I did not understand how one can search with IMAP in Outlook for e-mails with Subjects like The outlook server returns BADCHARSET, ok, but how can I search using such word? Do I convert the string into the specified encoding returned by the server? |
@JobaDiniz what is the full text of the response from the Exchange server? Does it only list US-ASCII as an available charset? What is the SEARCH command that Outlook uses? |
When you search for If so, you could modify your code to search for that. |
Searching for
S: A00000009 NO [BADCHARSET (US-ASCII)] The specified charset is not supported. Searching for Is that it? |
Try this: var inbox = (ImapFolder) client.Inbox;
var results = inbox.Search ("CHARSET US-ASCII SUBJECT \"Comunicação\""); if that doesn't work, try: var inbox = (ImapFolder) client.Inbox;
var results = inbox.Search ("CHARSET US-ASCII SUBJECT {13+}\r\nComunicação"); Edit: and if those fail, try without the If you can find a work-around by constructing the query string manually and let me know what the final solution is, I'll look into adding a workaround for Exchange SEARCH. A quick hack to see if any of these work would be: var inbox = (ImapFolder) client.Inbox;
inbox.Search ("CHARSET US-ASCII SUBJECT \"Comunicação\"");
inbox.Search ("CHARSET US-ASCII SUBJECT {13+}\r\nComunicação");
inbox.Search ("SUBJECT \"Comunicação\"");
inbox.Search ("SUBJECT {13+}\r\nComunicação"); Then just look at the resulting log to see which (if any) query worked successfully. If none of them work, then yes, it would seem that it is a limitation of the server that you are connecting to. |
First log - BAD Command Error
Second log - socketexception System.IO.IOException: Unable to read data from the transport connection
Third log - BAD Command Error. 11
Fourth log - socketexception System.IO.IOException: Unable to read data from the transport connection
|
Ok, so it turns out there was a bug in the This is what caused the IOExceptions. It may have also affected the other 2 commands. Basically, what was happening is that the query string was assumed to be ascii and so If you could grab the latest (2.3.1.8) nuget package from https://www.myget.org/feed/mimekit/package/nuget/MailKit and try again, that would be appreciated. Thanks! |
I updated and no exception was thrown but the search returned no results - and I assure you, there is a e-mail with
Another thing: |
I believe you that there is an e-mail with Comunicação in the subject. Yea, there's a ReleaseNotes.md file you can check. Are you saying that 1.8.1.1 was able to search for this string successfully? Can you get a log of the command that it sent? I have a hard time believing that 1.8.1.1 worked. |
Sorry, I just realized that you are probably saying that your app (not the little test app above) is using MailKit 1.8.1.1 and that you are planning to update it to use the latest MailKit (2.3.1.+). In that case, the ReleaseNotes.md is still a good place to look, but here's a little blurb that I've been including in the NuGet.org release notes that sums up the things to look out for: MailKit API Changes Since 2.0.x:
Note to users upgrading from MailKit 1.x: In order to authenticate using the XOAUTH2 SASL mechanism, you must now use the following approach: client.Authenticate (new SaslMechanismOAuth2 (username, auth_token)); As far as MimeKit goes, MimeKit has its own ReleaseNotes.md file. Pay closest attention to the notes for 2.0.0 (same for MailKit, really). |
This change will also *force* unicode strings into US-ASCII when if that is the only supported charset in an effort to work around the limitations of Exchange IMAP. See https://stackoverflow.com/questions/12691913/imap-search-charset-with-iso-8859-1 for an example of what Thunderbird does when forcing unicode into iso-8859-1. *May* fix #808
@JobaDiniz I've committed a potential fix based on https://stackoverflow.com/questions/12691913/imap-search-charset-with-iso-8859-1 If you could grab the MailKit v2.3.1.11 (or later) package from https://www.myget.org/feed/mimekit/package/nuget/MailKit and test it out, that would be fantastic. |
Essentially what it does is two-fold:
|
This change will also *force* unicode strings into US-ASCII when if that is the only supported charset in an effort to work around the limitations of Exchange IMAP. See https://stackoverflow.com/questions/12691913/imap-search-charset-with-iso-8859-1 for an example of what Thunderbird does when forcing unicode into iso-8859-1. *May* fix #808
@JobaDiniz did my above fix work for you? |
I didn’t have time to test it yet
On Mon, 23 Sep 2019 at 13:28 Jeffrey Stedfast ***@***.***> wrote:
@JobaDiniz <https://github.com/JobaDiniz> did my above fix work for you?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#808?email_source=notifications&email_token=AABILKRIBJVWPMESLGDNZ73QLDVELA5CNFSM4GSB27FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7LOMSQ#issuecomment-534177354>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AABILKTSIXO7G7ZSOMMWZZLQLDVELANCNFSM4GSB27FA>
.
--
*Atenciosamente,*
*Joberto Diniz*
jobertodiniz.com
|
It didn't work, using version
|
Sorry, I mean you should try by using |
Using
|
I'm going to have to throw in the towel and conclude that there's just no way to do this with Exchange :-\ |
Yeah... that's bummer... it isn't only an issue of C# developers: https://stackoverflow.com/a/55977511/1830639 |
Per, RFC6855:
and
Looking at the debug output from "SEP 19 2023" there was no attempt that showed up as:
The first attempt sent a charset which is wrong. The second attempt, looks like it used the wrong charset to generate the literal so it will not find a match. It should have used UTF-8 to generate bytes and omitted the charset from the search command. |
I found that this dose not work only when I use outlook email. title = title.encode('UTF-8') |
Office365 & Exchange only support US-ASCII as a charset for search. |
Thanks for the information. For completeness, I tested a patch with what I proposed above and it simply doesn't find a match due to as you stated only supporting US-ASCII in search and not advertising UTF-8=ACCEPT. |
Hi,
i'm trying to make this search on an hotmail IMAP account:
C: C00000014 UID SEARCH CHARSET UTF-8 NOT UID 18017 SUBJECT {59+}
C: Il tuo account OneDrive verrà eliminato in data 23/02/2019
S: C00000014 NO [BADCHARSET (US-ASCII)] The specified charset is not supported.
The text was updated successfully, but these errors were encountered: