NTLM authentication not working in some cases #582
Comments
|
MailKit 1.18 disabled NTLM authentication because I was getting too many bug reports about it not working and no way to diagnose the problems (because it worked just fine for me). This is why you are getting NotSupportedException. Just use LOGIN and it should work fine. |
|
Does this mean that NTLM won't be supported anymore? |
|
It's not in the list of SASL mechanisms that are tried by default, but starting with 2.0, you can use it manually: var ntlm = new SaslMechanismNtlm ("username", "password");
client.Authenticate (ntlm); |
|
Great! Thats seems to work! |
|
Just switched from .NET builtin SmtpClient (which was working fine) to MailKit and several customers are reporting this problem now. How can I make MailKit to also try using NTLM among other methods? |
if (client.AuthenticationMechanisms.Contains ("NTLM")) {
var ntlm = new SaslMechanismNtlm ("username", "password");
client.Authenticate (ntlm);
} else {
// use the default supported mechanisms
client.Authenticate ("username", "password");
} |
|
@jstedfast thank you sir |
|
Hello, I have tried using code mentioned above, var ntlm = new SaslMechanismNtlm ("username", "password"); but same error. i.e. username and password are windows login credentials. Sharing Protocol logs, Connected to smtp://pawan_dev.vxcountry.cinemax.com:25/?starttls=when-available Does Mailkit supports NTLM authentication ? Please suggest me some work-around for this issue. Thanks! |
|
@pawanbhabad I am also facing similar issue Case 1 : Successful Connection Connected to smtp://mail.celebritygroup.com:125/ Auth C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA= Case 2 : Failed Connection Connected to smtp://webmail.acilogistics.net:25/ Auth C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA= Case 3 : Failed Connection Connected to smtp://mail.ajlanbros.com:587/?starttls=always Auth C: AUTH NTLM TlRMTVNTUAABAAAABwIAAAAAAAAgAAAAAAAAACAAAAA= |
|
I don't suppose @sauravcaptech or @pawanbhabad can get me a log of some other SMTP client/library successfully authenticating via NTLM so that I can compare what MailKit is sending to what a successful authentication is sending? I'm stepping thru the debugger comparing @sauravcaptech's successful vs failed NTLM exchanges and I can't tell why one works and the others don't. In all cases, the exact same NTLM flags are used by the server (and MailKit). The only differences (other than usernames and passwords) are the domain names provided by the server (and used in the final authentication request to the server). That's it. |
|
@sauravcaptech As per your logs, it seems, the failure is definitely because of NTLM AUTH, i.e. case 2 & 3. |
|
No, all of @sauravcaptech's authentication examples use NTLM for authenticating (even the successful one). As you noted, however, in the case where the NTLM authentication was successful, the server also supported the That should be irrelevant, though, because these authentication mechanisms do not inter-relate. |
|
Hello @jstedfast Please have a look into the following logs, using "System.net.mail.SmtpClient" API, email is sent successfully. S: 220 PNV86SMTP01.PNE.VEN.CINEMAX.COM ESMTP Sendmail 8.14.4/8.14.4; Mon, 9 Nov 2020 14:30:43 +0530 S: 250 2.0.0 0A990hmD059588 Message accepted for delivery Logs using Mailkit; Connected to smtp://pawan_dev.vxcountry.cinemax.com:25/?starttls=when-available |
|
It looks like you connected to 2 different SMTP servers. In the case of System.Net.Mail, you used a Sendmail 8.14 server while in MailKit's case, you used a Microsoft Exchange ESMTP server. In the System.Net.Mail case, you also did not authenticate (well, the System.Net.Mail SmtpClient did not authenticate even if you gave it credentials to use). |
|
@jstedfast Also, in context to your second point, when I provide incorrect credentials to SMTPClient API OR no credentials, then it does not work. So I think SMTPClient authenticates with an appropriate credentials. |
|
What are you using to get the logs for System.Net.Mail's SmtpClient? Perhaps it stripped out the authentication commands? In any event, the Sendmail 8.14 server does not support NTLM, so System.Net.Mail probably used GSSAPI or CRAM-MD5 (I don't think System.Net.Mail supports DIGEST-MD5). |
|
Using WireShark to get logs. While capturing logs, the authentication commands are not captured. Shared logs, what all got generated from Wireshark. |
|
Unfortunately, the bit of the logs that I need are the authentication commands |
|
@pawanbhabad and @sauravcaptech if you guys can find a way to get me the logs of a successful NTLM authentication to a server where MailKit fails (via NTLM), please open a new bug. I'm going to close this one because the original issue is resolved (and was really a question rather than a bug). I'm definitely interested in fixing bugs in NTLM if I can, I just need info that unfortunately I don't have. Basically, what I want to do, is to compare what MailKit is sending with what a working implementation is sending in your cases, this way I can hopefully figure out what MailKit is doing wrong. |
|
Hello @jstedfast SMTP Relay server : "Pawan_Dev.vxcountry.cinemax.com". I am using same credentials in both the cases. Below are the logs shared for both APIs; Mailkit logs Connected to smtp://10.217.79.102:25/?starttls=when-available SMTPClient logs S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 23 Nov 2020 15:20:40 +0530 C: TlRMTVNTUAADAAAAGAAYAIoAAAB4AXgBogAAAAAAAABYAAAAGAAYAFgAAAAaABoAcAAAAAAAAAAaAgAABYKIogoA7kIAAAAP1t31DYB66IvmtXjlDBp3jXAAYQB3AGEAbgAuAGIAaABhAGIAYQBkAFcAMwAxADEANQA1ADcAMABRADgAMQBTAEcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABo9OFnsse8fOpnPtN+EODgEBAAAAAAAAS9aGGn7B1gENlOJzBTEX/AAAAAACAA4AVgBYAEkATgBEAEkAQQABABIAUABBAFcAQQBOAF8ARABFAFYABAAmAHYAeABpAG4AZABpAGEALgB2AGUAcgBpAHQAYQBzAC4AYwBvAG0AAwA6AFAAYQB3AGEAbgBfAEQAZQB2AC4AdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQAFACYAdgB4AGkAbgBkAGkAYQAuAHYAZQByAGkAdABhAHMALgBjAG8AbQAHAAgAS9aGGn7B1gEGAAQAAgAAAAgAMAAwAAAAAAAAAAEAAAAAIAAAQb6NkFRsPXqeapbOErEdQ9DC3EFPhyIR29iV0T05QSIKABAAAAAAAAAAAAAAAAAAAAAAAAkAKgBTAE0AVABQAFMAVgBDAC8AMQAwAC4AMgAxADcALgA3ADkALgAxADAAMgAAAAAAAAAAAA== S: 235 2.7.0 Authentication successful |
|
@pawanbhabad it looks like it will help, thanks. |
|
I sent you an email earlier detailing the differences that I found between MailKit and System.Net.Mail. I did a bit of reading of the NTLM docs and have spotted 2 potential problems in MailKit's code that I have "fixed", but I am not 100% confident in either of the fixes. What I need you to do is to test the NuGet package located at https://www.myget.org/feed/mimekit/package/nuget/MailKit/2.10.0.4 Once you've switched to using the above NuGet package, what I want you to do is to test the following cases for me:
To enable it, do the following: var ntlm = new SaslMechanismNtlm (username, password) {
NtlmFixes = NtlmFixes.NTLMv2IncludeZ24
};
// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
ntlm.OSVersion = Environment.OSVersion.Version;
To enable this fix, do the following: var ntlm = new SaslMechanismNtlm (username, password) {
NtlmFixes = NtlmFixes.NTLMv2UseTargetInfoTimestamp
};
// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
ntlm.OSVersion = Environment.OSVersion.Version;
var ntlm = new SaslMechanismNtlm (username, password) {
NtlmFixes = NtlmFixes.NTLMv2IncludeZ24 | NtlmFixes.NTLMv2UseTargetInfoTimestamp
};
// The following code enables the NTLMSSP_NEGOTIATE_VERSION debugging info and
// System.Net.Mail seems to include it, so I've made it possible to include as well.
if (Environment.OSVersion.Platform == PlatformID.Win32NT)
ntlm.OSVersion = Environment.OSVersion.Version;Let me know if any of the above solutions work. |
|
Official docs for NTLM can be found here: https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-nlmp/b38c36ed-2804-4868-a9ff-8dd3182128e4 Another useful resource can be found here: http://davenport.sourceforge.net/ntlm.html In fact, that second resource was the basis for the current NTLM implementation in MailKit (which was borrowed from Mono's NTLM code). |
|
Yes sure @jstedfast |
|
Hi @jstedfast I have tried all the above three cases but it does not seem to be working. Following are the test case logs, Case 1: 24 Byte array buffer case. S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 30 Nov 2020 10:45:06 +0530 Case 2: Timestamp included in server's TargetInfo; S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 30 Nov 2020 10:51:59 +0530 Case 3: Both fixes combined; S: 220 Pawan_Dev.vxcountry.cinemax.com Microsoft ESMTP MAIL Service, Version: 8.5.9600.16384 ready at Mon, 30 Nov 2020 10:19:18 +0530 |
|
Hi @jstedfast , I am also facing the same issue pointed by @pawanbhabad. Firstly, when I tried with "smtpClient.Authenticate(smtp.Login, smtp.Password);" to authenticate it gave me the error that " No compatible authentication mechanisms found." and the smtp server is using NTLM as Authentication mechanism. |
|
Hi @jstedfast , I am also facing the same issue on .NetCore 3.1 with Mailkit library. Can you please help with some workaround? |
|
@ukashanoor @kundankrishna the workaround right now is to use the I would replace MailKit's SaslMechanismNtlm with that other class, but unfortunately my goal is to make MailKit cross platform and that class won't work on Linux or Mac. |
|
@jstedfast Thank you for your response. Since our target runtime is Linux, this workaround is not applicable. Just wondering what could be the right solution for this and when that can be made available to all. |
|
Unfortunately reverse engineering NTLM is a difficult process :( |
jstedfast
changed the title
|
@kundankrishna @pawanbhabad @sauravcaptech For those of you who are having problems with MailKit's NTLM implementation:
|
|
Also, can anyone check this key in their registry?
|
|
I've completely rewritten the NTLM support in the https://github.com/jstedfast/MailKit/tree/ntlmv2 branch If any of you guys could test that for me, that would be amazing. |
|
@jstedfast While we have no such local SMTP server with NTLM support to play around with configurations, but still we can test it using the above branch with few SMTP servers we were having issues already. Will let you know how it goes by early next week. |
|
That would be awesome, thanks! |
|
@kundankrishna any update? |
|
Here is the code with which we tried private static SmtpClient CreateSMTPClient() MimeMessage mimeMessage = new MimeMessage(); Complete Exception: Error message: |
|
Any chance you could provide the protocol logs? |
|
This should work now. |
|
Hi @jstedfast , This is Nani from CISCO team. The above provided inputs and code haven't resolved NTML issue for Linux. @kundankrishna has already shared code snippet which we are used. Could you please assist on this, as it is a critical issue and impacting the deliverables. |
|
Which build is confirmed not to work? Have you guys tried the very latest CI builds? Can you get me an account on any of these mail servers so that I can test my code against them? |
|
@ccnani I've made more fixes since your comment. Any updates as to whether those fixes have helped? |
|
Hi @jstedfast !
Now I'm using version 2.13.0 of MailKit. I tried to use the version 3.0.0 but I got the same problems. Debugging the MailKit code I found out that the reason of problem "5.7.3 Authentication unsuccessful" in my case I tried to rollback MailKit to previous versions and found out that version 2.0.7 is the last version working in my case. In my case the domain from client credentials is different from the domain that smtp-server responded I tried to patch the version 2.13.0 of MailKit with this code and, lo and behold, that worked for me! Dear @jstedfast,
|
|
Hi @shav,
I'm looking at the NTLM logic right now and my first question to you (since you can debug this), is what code-path does it take in https://github.com/jstedfast/MailKit/blob/master/MailKit/Security/Ntlm/NtlmAuthenticateMessage.cs#L59 in your case? The code branch I'm looking at is this: if ((challenge.Flags & NtlmFlags.TargetTypeDomain) != 0) {
// The server is domain-joined, so the TargetName will be the domain.
Domain = challenge.TargetName;
} else {
// The server is not domain-joined, so the TargetName will be the machine name of the server.
Domain = challenge.TargetInfo?.DomainName;
// TODO: throw if TargetInfo is null?
}I think the approach that I will take is to pass the What we may find eventually is that we should always use the supplied domain name, if non-empty, but I would prefer to start off with narrowly fixing this in your particular case so that we don't we don't break this for anyone else if I'm wrong in my assumption. So to sum up what I want to know is does If you can get me that info for your case, I'll make the necessary changes (I'm already working on them locally) and I'll make the needed adjustment and push a fix. BTW, thanks for doing the research/debugging that you've already done, it is very exciting to feel like we've almost got NTLM working correctly in MailKit. It's been a long journey! |
|
I've committed what I think may be the correct fix and a new build will be uploaded to https://www.myget.org/feed/mimekit/package/nuget/MailKit in the next hour (or less). That said, I'd still be interested in knowing the answer to my question in my last comment just in case my fix breaks things for someone else (I made it always prefer the supplied domain name if non-empty). |
|
Hi, @jstedfast ! But unfortunately this fix doesn't work in case when the smtp-client is on a linux machine (I tested it on Ubuntu 20.04). In this case smtp-server still responds "535: 5.7.3 Authentication unsuccessful". To say the truth, I have some troubles with server-side certificate (smtp-client thinks that certificate is invalid) and I have to connect to the smtp-server without SSL/TLS (hack I have also debugged https://github.com/jstedfast/MailKit/blob/master/MailKit/Security/Ntlm/NtlmAuthenticateMessage.cs#L59 as you requested and found out that in my case code goes by path In my case |
|
@shav can you submit a new bug report for the case that is still failing? I'd prefer to have separate it from all the noise in this bug report. |
|
@jstedfast I'm sorry, that was wrong alarm about not working SmtpClient on Windows from internet. I wrote about my troubles to the administrator of my smtp-server and he informed me that a connection to smtp-server from internet was denied in server settings. He allowed connection to the smtp-server from my client machine via internet, and my SmtpClient established the connection successfully. |
|
@shav great news! Thanks for the update! |
Bumps [MailKit](https://github.com/jstedfast/MailKit) from 3.0.0 to 3.1.1. <details> <summary>Changelog</summary> *Sourced from [MailKit's changelog](https://github.com/jstedfast/MailKit/blob/master/ReleaseNotes.md).* > ### MailKit 3.1.1 (2022-01-30) > > * Reduced string allocations in Pop3Engine's capability parser. > * Updated GMail and Outlook.com SSL certificates. > * Modified SmtpClient to try and use the system hostname in EHLO/HELO commands. > (issue [#1314](jstedfast/MailKit#1314)) > > ### MailKit 3.1.0 (2022-01-14) > > * Fixed NTLM to always prefer the supplied domain over the TargetName or TargetInfo.DomainName. > (issue [#582](jstedfast/MailKit#582)) > * Updated GMail and Outlook.com SSL certificate info. > * Added a new SslCipherSuite property to each client that allows developers to get information > about the SSL/TLS cipher suite that was negotiated with the server. > (pull [#1312](jstedfast/MailKit#1312)) > * Reduced string allocations in SmtpClient's EHLO capability parsing logic. > * Default ProtocolLogger.RedactSecrets to true for added added security. > * Added work-around for parsing malformed GMail ENVELOPE responses that reverse the name and address components > of the Sender address. (pull [#1319](jstedfast/MailKit#1319)) > * Added net6.0 to the list of TargetFrameworks. </details> <details> <summary>Commits</summary> - [`7f3affd`](jstedfast/MailKit@7f3affd) Bumped version to 3.1.1 - [`f6f4a4e`](jstedfast/MailKit@f6f4a4e) Updated README.md - [`839ec61`](jstedfast/MailKit@839ec61) Updated README.md - [`8c50d12`](jstedfast/MailKit@8c50d12) Updated GettingStarted.md - [`a7db8cf`](jstedfast/MailKit@a7db8cf) Bump NUnit3TestAdapter from 4.2.0 to 4.2.1 ([#1323](jstedfast/MailKit#1323)) - [`a5637f9`](jstedfast/MailKit@a5637f9) minor update to previous unit test - [`163fa9b`](jstedfast/MailKit@163fa9b) Added unit tests for SmtpClient.OnNoRecipientsAccepted() - [`dfafce8`](jstedfast/MailKit@dfafce8) Added more SmtpClient unit tests - [`5c5e11d`](jstedfast/MailKit@5c5e11d) Reduce string allocations in Pop3Engine's capability parser - [`a818ada`](jstedfast/MailKit@a818ada) Updated GMail SSL certificates - Additional commits viewable in [compare view](jstedfast/MailKit@3.0.0...3.1.1) </details> <br /> Co-authored-by: Elanis <elanis@hotmail.com> Reviewed-on: https://gitea.dysnomia.studio/elanis/dysnomia-website/pulls/23 Co-authored-by: elanis <elanis@noreply.example.org> Co-committed-by: elanis <elanis@noreply.example.org>
Hello,
the issue occurs with version MailKit v1.18.1.1 (pulled via Nuget).
I've tried to get MailKit working with our Exchange Server. MailKit successfully connected to port 587 and established a secure channel using STARTTLS. After authenticating the Exchange server offered the mechanisms GSSAPI, NTLM and LOGIN.
I've removed GSSAPI and LOGIN from the authentication mechanisms MailKit may use:
Only NTLM is left in the list of authentication mechanisms.
Trying to authenticate using...
...throws the following exception
Here comes the log of the SMTP session:
I hope this helps you to track the issue down.
Thank you in advance!
The text was updated successfully, but these errors were encountered: