HttpClient ObjectDisposed after SDK upgrade from 34.0.95 -> 34.0.113

[haavamoa]

Android framework version

net8.0-android

Affected platform version

Android SDK 34.0.113

Description

Hello, and thank you for this amazing project.

We have a mobile application running MAUI in hospitals in Norway, where Android is one of the most used platforms. We needed to deliver a new version this week. Last Wednesday / Thursday we noticed that we started getting sporadic http client disposal messages. After some time we figured out that the only changes to the app was the Android SDK version delivered by .NET . 34.0.113 was released, and we noticed that pinning to 34.0.95 fixed the issues.

After a lot of times, I've actually not been able to give you a reproducible project , and due to internal policies in my company I am unable to share my project.

But I can give you a brief explanation of the architecture surrounding http client in the project:

1. We reuse our HttpClients as much as we can in between different pages.
2. We add our own DelegatingHandlers to make sure we add ticket, add logging etc.
3. We use Polly for timeouts.

The times I often see it is when I navigate between two pages that is using the same http client, but is using it against different request urls.

Here is the exception we keep seeing:

ObjectDisposed_Generic
ObjectDisposed_ObjectName_Name, Java.IO.InputStreamInvoker
System.ObjectDisposedException: ObjectDisposed_Generic
ObjectDisposed_ObjectName_Name, Java.IO.InputStreamInvoker
   at Java.Interop.JniPeerMembers.AssertSelf(IJavaPeerable )
   at Java.Interop.JniPeerMembers.JniInstanceMethods.InvokeVirtualVoidMethod(String , IJavaPeerable , JniArgumentValue* )
   at Java.IO.InputStream.Close()
   at Android.Runtime.InputStreamInvoker.Close()
   at System.IO.Stream.Dispose()
   at System.IO.BufferedStream.Dispose(Boolean )
   at System.IO.Stream.Close()
   at System.IO.Stream.Dispose()
   at System.Net.Http.StreamContent.Dispose(Boolean )
   at System.Net.Http.HttpContent.Dispose()
   at System.Net.Http.HttpResponseMessage.Dispose(Boolean )
   at Xamarin.Android.Net.AndroidHttpResponseMessage.Dispose(Boolean )
   at System.Net.Http.HttpResponseMessage.Dispose()
   at System.Net.Http.HttpClient.HandleFailure(Exception , Boolean , HttpResponseMessage , CancellationTokenSource , CancellationToken , CancellationTokenSource )
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage , HttpCompletionOption , CancellationTokenSource , Boolean , CancellationTokenSource , CancellationToken )

I understand that this a long shot as I do not have a reproducible project, but I hope that anyone in the team / community can reach out and see if they might have broken something, or have an idea on what we are potentially doing wrong.

Appreciate your time!

Steps to Reproduce

None unfortunately.

Did you find any workaround?

For now, we pin the SDK version by using the rollback feature of dotnet workload install, which saved our delivery, but we plan to upgrade every bits and pieces soon, so this will soon be an issue for us again.

Relevant log output

No response

[jpobst]

@simonrozsival Now that we are part of the dotnet GitHub organization, should I transfer HTTP issues like this to dotnet/runtime?

[simonrozsival]

@jpobst this might be an issue in Xamarin.Android.Net.AndroidHttpResponseMessage.Dispose(Boolean ) so I would keep it in the android repo for now, until we can say that this issue doesn't belong here.

[haavamoa]

I forgot to mention another work around I've had success with:

1. Catch the exception
2. Recreate the RequestMessage
3. Do the same HttpClient call

This might end up with throwing another dispose exception, or not. I've added a 5 second retry, which will end up with a success at some point during those 5 seconds.

If you are interested:
Here's my HttpClient helper class that gets run when an ObjectDisposedException occurs in a DelegateHandler:

internal static class SharedRetryHelper
{
    internal static Task<HttpResponseMessage> RetryDueToAndroidBug(this IHttpClient httpClient, HttpRequestMessage requestMessage, CancellationToken cancellationToken,ObjectDisposedException objectDisposedException)
    {
        var dateTimeNow = DateTime.Now;
        var now = new TimeOnly(dateTimeNow.Hour, dateTimeNow.Minute, dateTimeNow.Second);
        if (requestMessage.RequestUri == null) throw objectDisposedException;
        if (httpClient.DisposeInformation.LastTimeDisposed != null)
        {
            var nowTicks = DateTime.Now.Ticks;
            var lastTime = httpClient.DisposeInformation.LastTimeDisposed.Value;
            if (lastTime.Add(TimeSpan.FromSeconds(5)).Ticks > nowTicks) //If it's been more than 5seconds of disposal exceptions.
            {
                httpClient.DisposeInformation.LastTimeDisposed = null;
                Log(httpClient, requestMessage, "Got disposed exception on Android for more than 5 seconds. Will give up and throw exception.");
                throw objectDisposedException;
            }    
        }
        else
        {
            httpClient.DisposeInformation.LastTimeDisposed = now;
        }
        
        httpClient.DisposeInformation.TotalTimesObjectDisposed++;
        Log(httpClient, requestMessage, $"Got disposed exception on Android. Has been disposed {httpClient.DisposeInformation.TotalTimesObjectDisposed} times, last time was {httpClient.DisposeInformation.LastTimeDisposed.Value}:{httpClient.DisposeInformation.LastTimeDisposed.Value.Second}, will retry now at {now}:{now.Second}");
        if (requestMessage.Method == HttpMethod.Get)
        {
            return httpClient.Get(requestMessage.RequestUri.AbsoluteUri, cancellationToken);
        }
            
        if (requestMessage.Method == HttpMethod.Post)
        {
            return httpClient.Post(requestMessage.RequestUri.AbsoluteUri, requestMessage.Content, cancellationToken);
        }
            
        if (requestMessage.Method == HttpMethod.Put)
        {
            return httpClient.Put(requestMessage.RequestUri.AbsoluteUri, requestMessage.Content, cancellationToken);
        }
            
        if (requestMessage.Method == HttpMethod.Delete)
        {
            return httpClient.Delete(requestMessage.RequestUri.AbsoluteUri, cancellationToken);
        }

        throw objectDisposedException;
    }

    private static void Log(IHttpClient httpClient, HttpRequestMessage requestMessage, string message)
    {
#if __ANDROID__
        if (requestMessage.RequestUri != null)
        {
            Android.Util.Log.Debug("DME HttpClientAdapter",$"HttpClientName: {httpClient.Name} : {requestMessage.RequestUri.AbsoluteUri} : {message}");    
        }
#endif
    }

IHttpClient is just an abstraction on top of HttpClient in our project. The DispiseInformatin is just a simple class containing when it last got the bug and a counter of how many times it happened. This is convenient for debugging purposes.

[haavamoa]

@simonrozsival: Just shout out if you need me to test potential nightly builds or something in our project.

[simonrozsival]

@haavamoa can you please share more information about the settings you're using with the client? I'm especially interested in automatic decompression, built-in authentication (Basic?, NTLM?), proxies, ... Also are you able to tell what happened with the request just before it threw the exception (redirect, 4xx error, 5xx error)? Does this happen for a specific HTTP method or does it happen both with and without any body sent to the server?

[haavamoa]

Hi @simonrozsival.

We've seen this bug on both HttpClients that is using authentication, but also on HttpClients which his not. For the ones that do ; we are using built-in-authentication using Bearer Token from the Identity Model flow. We've not set any decompression or proxies to our HttpClients.

No requests have failed before we do the calls. One example is simply trying to reach a /status/ping endpoint which does nothing more than return 200 OK. This fail sporadicly. It happens for all kinds of HTTP methods, with or without bodies.

I've spent days trying to see the connection between the order of doing calls, or what happens to them before it fails, but I am unable to find a pattern at all. From my point of view it can happen to any call we do regardless of the situation, which I find too good to be true to be honest.

[simonrozsival]

Thanks for the details, @haavamoa. I remember seeing reports of ObjectDisposedException being thrown before but we've never been able to reproduce it. I will try to repro this again to see if I can narrow down what could cause this exception.

If you think of any additional details (are you making requests in parallel or just one at a time, you're making requests always when the app is in foreground or if the app is in background, ...) or if you're able to repro it reliably, please let me know.

[simonrozsival]

@haavamoa could you try building your app with <UseNativeHttpHandler>false</UseNativeHttpHandler>? That will internally change AndroidMessageHandler for SocketsHttpHandler and it might help you avoid this issue altogether.

[zachdean]

Hey @simonrozsival, I have started observing this exception randomly in our http requests (MAUI) after the tooling was upgraded to Android SDK 34.0.113. Looking through our logs I do not see any discernable patterns for when the failure happens.

[simonrozsival]

@zachdean do you have any stacktraces you can share? Are you able to reproduce the exception in your app?

[kmiterror]

Same happens for us after workload was updated to 34.0.113
34.0.95 works ok

As a workaround we will try this:

Make a file like this named workload.json:
{
"microsoft.net.sdk.android": "34.0.95/8.0.100"
}
Then dotnet workload update --from-rollback-file $(androidProjectFolder)/workload.json should install 34.0.95.

Stacktrace of the crash:

An error occured deserializing the response.
Refit.ApiException: An error occured deserializing the response.
 ---> System.ObjectDisposedException: ObjectDisposed_Generic
ObjectDisposed_ObjectName_Name, Java.IO.InputStreamInvoker
   at Java.Interop.JniPeerMembers.AssertSelf(IJavaPeerable )
   at Java.Interop.JniPeerMembers.JniInstanceMethods.InvokeVirtualVoidMethod(String , IJavaPeerable , JniArgumentValue* )
   at Java.IO.InputStream.Close()
   at Android.Runtime.InputStreamInvoker.Close()
   at System.IO.Stream.Dispose()
   at System.IO.BufferedStream.Dispose(Boolean )
   at System.IO.Stream.Close()
   at System.IO.Stream.Dispose()
   at System.IO.DelegatingStream.Dispose(Boolean )
   at System.IO.Stream.Close()
   at System.IO.StreamReader.Dispose(Boolean )
   at System.IO.StreamReader.Close()
   at Newtonsoft.Json.JsonTextReader.Close()
   at Newtonsoft.Json.JsonReader.Dispose(Boolean disposing)
   at Newtonsoft.Json.JsonReader.System.IDisposable.Dispose()
   at Refit.NewtonsoftJsonContentSerializer.<FromHttpContentAsync>d__4`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Refit.RequestBuilderImplementation.<DeserializeContentAsync>d__15`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Refit.RequestBuilderImplementation.<>c__DisplayClass14_0`2.<<BuildCancellableTaskFuncForMethod>b__0>d[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   Exception_EndOfInnerExceptionStack
   at Refit.RequestBuilderImplementation.<>c__DisplayClass14_0`2.<<BuildCancellableTaskFuncForMethod>b__0>d[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
--- End of stack trace from previous location ---
   at Polly.Timeout.AsyncTimeoutEngine.<ImplementationAsync>d__0`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Polly.Timeout.AsyncTimeoutEngine.<ImplementationAsync>d__0`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Polly.AsyncPolicy.<ExecuteAsync>d__21`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Polly.Retry.AsyncRetryEngine.<ImplementationAsync>d__0`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Polly.AsyncPolicy.<ExecuteAsync>d__21`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Polly.AsyncPolicy.<ExecuteAsync>d__21`1[[System.DateTime, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()

[simonrozsival]

@zachdean do you also use Polly?