First request with HttpClient on Android takes more than 45 seconds, after that less than second

[hardikphd]

Steps to Reproduce

1. Create Cross Platform-App (Xamarin-Forms) Projects
2. Add following code in constructor of MainPage of Shared Code:

HttpClientTest testHttpClient = new HttpClientTest();
Task.Run(() => testHttpClient.TestHttpClientAsync("Google","https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png")).Wait();
Task.Run(() => testHttpClient.TestHttpClientAsync("Microsoft","https://assets.onestore.ms/cdnfiles/external/uhf/long/9a49a7e9d8e881327e81b9eb43dabc01de70a9bb/images/microsoft-gray.png")).Wait();
Task.Run(() => testHttpClient.TestHttpClientAsync("Chrome","https://www.google.com/chrome/assets/common/images/chrome_logo_2x.png?mmfb=a5234ae3c4265f687c7fffae2760a907")).Wait();

3. Create HttpClientTest Class with following:

public class HttpClientTest
{
    private HttpClient _httpClient;
    public HttpClientTest()
    {
        HttpClientHandler hch = new HttpClientHandler
        {
            UseProxy = false,
        };
        _httpClient = new HttpClient(hch)
        {
            Timeout = new TimeSpan(0, 0, 0, 100)
        };
    }

    public async Task TestHttpClientAsync(string key, string url)
    {
        DateTime startTime = DateTime.Now;
        Console.WriteLine(string.Empty);
        Console.WriteLine($"TestHttpClientAsync : Start : {key} : {startTime}");
        var result = await _httpClient.GetByteArrayAsync(url).ConfigureAwait(false);
        DateTime endTime = DateTime.Now;
        Console.WriteLine(string.Empty);
        Console.WriteLine($"TestHttpClientAsync : End    : {key} : {endTime.Subtract(startTime).TotalMilliseconds}");
        Console.WriteLine(string.Empty);
        Console.WriteLine($"TestHttpClientAsync : Total  : {key} : {endTime} with {result?.Length}");
        Console.WriteLine(string.Empty);
    }
}

Expected Behavior

On Android, I have seen that HttpClient takes longer for first request (around 45 to 65 seconds) and then onward, It is always less than second. All HttpClient request should be of same time.

Actual Behavior

TestHttpClientAsync : Start : Google : 24/01/2018 19:39:30
TestHttpClientAsync : End : Google : 4610.646
TestHttpClientAsync : Total : Google : 24/01/2018 19:39:35 with 5969
TestHttpClientAsync : Start : Microsoft : 24/01/2018 19:39:35
TestHttpClientAsync : End : Microsoft : 681.655
TestHttpClientAsync : Total : Microsoft : 24/01/2018 19:39:36 with 4054
TestHttpClientAsync : Start : Chrome : 24/01/2018 19:39:36
TestHttpClientAsync : End : Chrome : 100.584
TestHttpClientAsync : Total : Chrome : 24/01/2018 19:39:36 with 5666

Version Information

Microsoft Visual Studio Community 2017
Version 15.5.3
VisualStudio.15.Release/15.5.3+27130.2020
Microsoft .NET Framework
Version 4.7.02556

Installed Version: Community

Visual Basic 2017 00369-60000-00001-AA603
Microsoft Visual Basic 2017

Visual C# 2017 00369-60000-00001-AA603
Microsoft Visual C# 2017

Visual F# 4.1 00369-60000-00001-AA603
Microsoft Visual F# 4.1

Application Insights Tools for Visual Studio Package 8.10.01106.1
Application Insights Tools for Visual Studio

ASP.NET and Web Tools 2017 15.0.31127.0
ASP.NET and Web Tools 2017

ASP.NET Core Razor Language Services 1.0
Provides languages services for ASP.NET Core Razor.

ASP.NET Web Frameworks and Tools 2012 4.0.20601.0
For additional information, visit https://www.asp.net/

ASP.NET Web Frameworks and Tools 2017 5.2.51007.0
For additional information, visit https://www.asp.net/

AWS Toolkit for Visual Studio 2017 1.14.0.0
AWS Toolkit for Visual Studio 2017.
Copyright 2011-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Release notes: https://aws-vs-toolkit.s3.amazonaws.com/versioninfo.html

This software includes third party software subject to the following copyrights:

• Logging from log4net, Apache License
  [http://logging.apache.org/log4net/license.html]
• Putty for PPK to PEM conversion, MIT license
  [http://www.chiark.greenend.org.uk/~sgtatham/putty/licence.html]
• NGit for AWS Elastic Beanstalk incremental push
  [https://github.com/mono/ngit/blob/master/NGit.license.txt]
• NSch dependency for NGit
  [https://github.com/mono/ngit/blob/master/NSch.license.txt]
• Sharpen dependency for NGit
  [https://github.com/mono/ngit/blob/master/Sharpen/AssemblyInfo.cs]
• ICSharpCode.SharpZipLib dependency for NGit
  [http://www.icsharpcode.net/opensource/sharpziplib/]
• Mono.Posix.dll and Mono.Security.dll dependencies for NGit
  [http://mono-project.com/FAQ:_Licensing#Licensing]
• MPFProj for Visual Studio Project Framework
  [http://mpfproj10.codeplex.com/license]
• JSON Checker for JSON validation
  [http://www.raboof.com/projects/jsonchecker/]
• Newtonsoft Json.NET
  [http://www.newtonsoft.com/json]
• YamlDotNet
  [http://aaubry.net/pages/yamldotnet.html]

Azure App Service Tools v3.0.0 15.0.31106.0
Azure App Service Tools v3.0.0

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Dotfuscator Community Edition 5.32.1.6167-6ce295ebd
PreEmptive Protection - Dotfuscator CE

JavaScript Language Service 2.0
JavaScript Language Service

JavaScript Project System 2.0
JavaScript Project System

JavaScript UWP Project System 2.0
JavaScript UWP Project System

Merq 1.1.17-rc (cba4571)
Command Bus, Event Stream and Async Manager for Visual Studio extensions.

Microsoft Continuous Delivery Tools for Visual Studio 0.3
Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE.

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger 1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual Studio Tools for Containers 1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Microsoft Visual Studio VC Package 1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio 4.8.4-pre (3fe64e3)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager 4.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

SQL Server Data Tools 15.1.61710.120
Microsoft SQL Server Data Tools

TypeScript Tools 15.5.11025.1
TypeScript Tools for Microsoft Visual Studio

Visual Studio Code Debug Adapter Host Package 1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for Universal Windows Apps 15.0.27130.2020
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

VisualStudio.Mac 1.0
Mac Extension for Visual Studio

Xamarin 4.8.0.757 (7f9ec2a)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 4.8.188 (c5813fa34)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK 8.1.3.0 (HEAD/ef47226b7)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK 11.6.1.2 (6857dfc)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Log File

adb.log

[hardikphd]

@JonDouglas Hi Jon, As per your request, I have raised issue here.
Somehow I could not find Area or Tags section here Hence I did not add them, I apologize for that in advance.

[grendello]

@hardikphd Can you try replacing DNS names in your tests with their corresponding IP addresses and see if the issue persist? DNS lookup taking a long time the first time is one of the possibilities (after that the looked up entry may be cached for some time and thus further lookups will be fast).

Also, assuming that all the requests should always take the same amount of time is not correct. We are dealing with the network here and there's a multitude of factors that can contribute to the request speed: DNS lookup, round-trip time between hosts, mobile device having to bring the network interface up on the first request and taking time to receive DHCP configuration, switch between mobile and wifi network just to name a handful of them.

[hardikphd]

@grendello It always takes 45-65 seconds for first request and then onwards, it takes less than second.
As I am using this in Mobile App and URL is on cloud, IP address of the cloud can change anytime, I dont have flexibility to use IP address.

Also, As you can see in my example, I am trying to download google image from its webpage and still it takes 45 seconds, such URL should be known to DNS, right?

[grendello]

@hardikphd IP can change, of course, but it shouldn't change for a duration of a dozen tests - this is the only way to eliminate the DNS cause. Regarding your question, no, there's no guarantee that the DNS entry for a server is in the DNS cache after some time, especially for round-robin DNS entries as is often the case with CDNs (like Google's), so while the entry may remain in cache 2 or 5 seconds after the initial request, it may be evicted after 30 or 60s and so your next test after that time will need to walk the DNS server chain again to get the new IP. Keep in mind, as well, that it doesn't matter what the entire URL is - what matters is just the host name and replacing it by any IP that works at the specific time will allow you to see if DNS server is a problem (which I suspect might be the case). You can also try to use a different HttpClientHandler (e.g. https://github.com/xamarin/xamarin-android/blob/master/src/Mono.Android/Xamarin.Android.Net/AndroidClientHandler.cs which uses native Android HTTP client to perform the request, unlike the BCL HttpClientHandler which uses the Mono BCL networking stack) with DNS to see if you can replicate the issue. If the problem occurs with the native HTTP client handler, then it's practically certain the issue is external to code - most likely DNS or other network conditions.

[hardikphd]

@grendello Here are my findings

1. Results with AndroidClientHandler are same as HttpClientHandler.
2. If I use same URL on chrome browser of same android phone, It comes up immediately.
3. If URL is HTTP, (not HTTPS), delay is around 1 seconds compare to 45-65 seconds with HTTPS
4. I tried ModernHttpClient plugin with NativeMessageHandler and request times out.
5. If I ran UWP version of same app on same network, I dont see delay with first request
6. If I ran same app on different network, I don't see delay issue. It is around 3-4 seconds.

I can see that there is some issue with our network and HttpClient on Android phone but same does not happen with UWP or chrome on same network.

If you think, It is just network, You can close the incident.

[grendello]

@hardikphd It does seem to be a case of bad mobile network - especially that you tried 3 different HTTP clients (including the native Java one) and all of them give similar problems. The fact that Chrome work doesn't prove anything - it's a privileged app, which may influence how it works. Item #3 is interesting, as it suggests an issue with establishing TLS connection (certificate validation?) - but again, doesn't appear to be specific to Mono/XA BCL since it behaves the same as the native HttpClientHandler. The fact that it works correctly on another network may also be due to transparent filtering on your test network for mobile devices. Mobile devices may be assigned IP from a different pool that's transparently proxied/monitored for security reasons.

I will indeed close the issue as I do not see any way to reproduce it and I don't believe the issue is within our networking stack, but do feel free to reopen it should you find any more data that'd help diagnose the problem.

[napoleonjones]

Was this issue resolved? I'm seeing something similar on my intitial request. Not 45-65 seconds, but 8-10 seconds and then subsequent calls are around 2 seconds.

[hardikphd]

@napoleonjones No, it is still around 8-10 seconds as of you.

[dang-gun]

I do the same.
It takes 8-12 seconds to try with a domain address,
It takes 1 to 2 seconds to call it by IP.