Fast deployment not working / very slow

[tipa]

Happens since VS 16.9 and the changes to the Fast deployment

Steps to Reproduce

1. Create standard Android app in VS
2. Build & deploy
3. Make C# code change
4. Redeploy

App1.zip

Expected Behavior

Fast build & deploy

Actual Behavior

Slow deploy.
The attached demo app takes a whopping 40 secs to recompile, apply the change and launch this barebone app in the simulator

Version Information

VS 16.9.0
Xamarin.Android 11.2.0.21

Log File

Build started...
1>------ Build started: Project: App1, Configuration: Debug Any CPU ------
1>  App1 -> C:\Users\***\Desktop\App1\bin\Debug\App1.dll
2>Starting deployment to Pixel_4_API_29 ...
2>Deploying to Pixel_4_API_29 ...
Build started.
Project "App1.csproj" (Install target(s)):
Found Java SDK version 1.8.0.
Found Java SDK version 1.8.0.
  PackageName: com.companyname.app1
Dex Fast Deployment Enabled: False
MonoAndroid Tools: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\
Android Platform API level: 30
TargetFrameworkVersion: v11.0
Android NDK: E:\android-sdk
dk-bundle\
Android SDK: E:\android-sdk\
Android SDK Build Tools: E:\android-sdk\build-tools\30.0.2\
Java SDK: C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\
Application Java class: android.app.Application
Touching "obj\Debug\110\stamp\_ResolveLibraryProjectImports.stamp".
Skipping target "_BuildLibraryImportsCache" because all output files are up-to-date with respect to the input files.
Skipping target "_GenerateLayoutBindings" because it has no outputs.
Touching "obj\Debug\110\stamp\_ConvertResourcesCases.stamp".
Touching "obj\Debug\110\R.cs.flag".
  Adding assembly reference for App1, recursively...
  Adding assembly reference for Java.Interop, recursively...
  Adding assembly reference for Mono.Android, recursively...
    Adding assembly reference for System.Net.Http, recursively...
    Adding assembly reference for System.Drawing.Common, recursively...
    Adding assembly reference for System.Runtime.Serialization, recursively...
      Adding assembly reference for System.ServiceModel.Internals, recursively...
  Adding assembly reference for mscorlib, recursively...
  Adding assembly reference for System.Core, recursively...
  Adding assembly reference for System, recursively...
    Adding assembly reference for Mono.Security, recursively...
  Adding assembly reference for System.Numerics, recursively...
  Adding assembly reference for System.Numerics.Vectors, recursively...
  Adding assembly reference for System.Runtime, recursively...
    Adding assembly reference for System.ComponentModel.Composition, recursively...
  Adding assembly reference for System.Xml, recursively...
  Adding assembly reference for Xamarin.AndroidX.Activity, recursively...
  Adding assembly reference for Xamarin.AndroidX.Annotation, recursively...
  Adding assembly reference for Xamarin.AndroidX.AppCompat.AppCompatResources, recursively...
  Adding assembly reference for Xamarin.AndroidX.AppCompat, recursively...
  Adding assembly reference for Xamarin.AndroidX.Arch.Core.Common, recursively...
  Adding assembly reference for Xamarin.AndroidX.Arch.Core.Runtime, recursively...
  Adding assembly reference for Xamarin.AndroidX.AsyncLayoutInflater, recursively...
  Adding assembly reference for Xamarin.AndroidX.Browser, recursively...
  Adding assembly reference for Xamarin.AndroidX.CardView, recursively...
  Adding assembly reference for Xamarin.AndroidX.Collection, recursively...
  Adding assembly reference for Xamarin.AndroidX.CoordinatorLayout, recursively...
  Adding assembly reference for Xamarin.AndroidX.Core, recursively...
  Adding assembly reference for Xamarin.AndroidX.CursorAdapter, recursively...
  Adding assembly reference for Xamarin.AndroidX.CustomView, recursively...
  Adding assembly reference for Xamarin.AndroidX.DocumentFile, recursively...
  Adding assembly reference for Xamarin.AndroidX.DrawerLayout, recursively...
  Adding assembly reference for Xamarin.AndroidX.Fragment, recursively...
  Adding assembly reference for Xamarin.AndroidX.Interpolator, recursively...
  Adding assembly reference for Xamarin.AndroidX.Legacy.Support.Core.UI, recursively...
  Adding assembly reference for Xamarin.AndroidX.Legacy.Support.Core.Utils, recursively...
  Adding assembly reference for Xamarin.AndroidX.Lifecycle.Common, recursively...
  Adding assembly reference for Xamarin.AndroidX.Lifecycle.LiveData.Core, recursively...
  Adding assembly reference for Xamarin.AndroidX.Lifecycle.Runtime, recursively...
  Adding assembly reference for Xamarin.AndroidX.Lifecycle.ViewModel, recursively...
  Adding assembly reference for Xamarin.AndroidX.Lifecycle.ViewModelSavedState, recursively...
  Adding assembly reference for Xamarin.AndroidX.Loader, recursively...
  Adding assembly reference for Xamarin.AndroidX.LocalBroadcastManager, recursively...
  Adding assembly reference for Xamarin.AndroidX.MultiDex, recursively...
  Adding assembly reference for Xamarin.AndroidX.Print, recursively...
  Adding assembly reference for Xamarin.AndroidX.RecyclerView, recursively...
  Adding assembly reference for Xamarin.AndroidX.SavedState, recursively...
  Adding assembly reference for Xamarin.AndroidX.SlidingPaneLayout, recursively...
  Adding assembly reference for Xamarin.AndroidX.SwipeRefreshLayout, recursively...
  Adding assembly reference for Xamarin.AndroidX.Transition, recursively...
  Adding assembly reference for Xamarin.AndroidX.VectorDrawable.Animated, recursively...
  Adding assembly reference for Xamarin.AndroidX.VectorDrawable, recursively...
  Adding assembly reference for Xamarin.AndroidX.VersionedParcelable, recursively...
  Adding assembly reference for Xamarin.AndroidX.ViewPager, recursively...
  Adding assembly reference for Xamarin.Essentials, recursively...
  Adding assembly reference for Xamarin.Google.Android.Material, recursively...
  Adding assembly reference for Xamarin.Google.Guava.ListenableFuture, recursively...
Skipping target "_CopyConfigFiles" because it has no inputs.
Skipping target "_ConvertPdbFiles" because it has no outputs.
Skipping target "_CopyConfigFiles" because it has no inputs.
Skipping target "_ConvertPdbFiles" because it has no outputs.
Touching "obj\Debug\110\stamp\_GenerateJavaStubs.stamp".
C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -cp "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\manifestmerger.jar" com.xamarin.manifestmerger.Main obj\Debug\110\android\manifestmerger.rsp 
Skipping target "_ConvertCustomView" because all output files are up-to-date with respect to the input files.
Copying file from "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\ReferenceAssemblies\Microsoft\Framework\MonoAndroid\v11.0\mono.android.jar" to "C:\Users\***\Desktop\App1\obj\Debug\110\android\bin\mono.android.jar".
Touching "obj\Debug\110\android\bin\mono.android.jar".
Touching "obj\Debug\110\static.flag".
Touching "obj\Debug\110\stamp\_GeneratePackageManagerJava.stamp".
Skipping target "_GenerateAndroidAssetsDir" because it has no outputs.
Touching "obj\Debug\110\android\bin\packaged_resources".
Skipping target "_CompileJava" because all output files are up-to-date with respect to the input files.
Deleting file "obj\Debug\110\android\bin\classes.dex".
C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -Xmx1G -classpath "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\r8.jar" com.android.tools.r8.D8 --debug --min-api 23 --output obj\Debug\110\android\bin\ --lib E:\android-sdk\platforms\android-30\android.jar obj\Debug\110\android\bin\classes.zip "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\ReferenceAssemblies\Microsoft\Framework\MonoAndroid\v11.0\mono.android.jar" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\\java_runtime.jar" C:\Users\***\.nuget\packages\xamarin.google.guava.listenablefuture\1.0.0.2\buildTransitive\monoandroid50\guava-listenablefuture.jar C:\Users\***\.nuget\packages\xamarin.androidx.annotation\1.1.0.7\buildTransitive\monoandroid90\..\..\jar\androidx.annotation.annotation.jar C:\Users\***\.nuget\packages\xamarin.androidx.collection\1.1.0.5\buildTransitive\monoandroid90\..\..\jar\androidx.collection.collection.jar C:\Users\***\.nuget\packages\xamarin.androidx.lifecycle.common\2.2.0.3\buildTransitive\monoandroid90\..\..\jar\androidx.lifecycle.lifecycle-common.jar C:\Users\***\.nuget\packages\xamarin.androidx.arch.core.common\2.1.0.6\buildTransitive\monoandroid90\..\..\jar\androidx.arch.core.core-common.jar obj\Debug\110\lp\41\jl\classes.jar obj\Debug\110\lp\42\jl\classes.jar obj\Debug\110\lp\43\jl\classes.jar obj\Debug\110\lp\44\jl\classes.jar obj\Debug\110\lp\45\jl\classes.jar obj\Debug\110\lp\46\jl\classes.jar obj\Debug\110\lp\47\jl\classes.jar obj\Debug\110\lp\48\jl\classes.jar obj\Debug\110\lp\49\jl\classes.jar obj\Debug\110\lp\50\jl\classes.jar obj\Debug\110\lp\51\jl\classes.jar obj\Debug\110\lp\52\jl\classes.jar obj\Debug\110\lp\53\jl\classes.jar obj\Debug\110\lp\54\jl\classes.jar obj\Debug\110\lp\55\jl\classes.jar obj\Debug\110\lp\56\jl\classes.jar obj\Debug\110\lp\57\jl\classes.jar obj\Debug\110\lp\58\jl\classes.jar obj\Debug\110\lp\59\jl\classes.jar obj\Debug\110\lp\60\jl\classes.jar obj\Debug\110\lp\61\jl\classes.jar obj\Debug\110\lp\62\jl\classes.jar obj\Debug\110\lp\63\jl\classes.jar obj\Debug\110\lp\64\jl\classes.jar obj\Debug\110\lp\65\jl\classes.jar obj\Debug\110\lp\66\jl\classes.jar obj\Debug\110\lp\67\jl\classes.jar obj\Debug\110\lp\68\jl\classes.jar obj\Debug\110\lp\69\jl\classes.jar obj\Debug\110\lp\70\jl\classes.jar obj\Debug\110\lp\71\jl\classes.jar obj\Debug\110\lp\72\jl\classes.jar obj\Debug\110\lp\73\jl\classes.jar obj\Debug\110\lp\74\jl\classes.jar 
Warning in obj\Debug\110\lp\74\jl\classes.jar:androidx/browser/browseractions/BrowserServiceFileProvider.class:
  Type `androidx.concurrent.futures.ResolvableFuture` was not found, it is required for default or static interface methods desugaring of `Landroidx/browser/browseractions/BrowserServiceFileProvider;saveBitmap(Landroid/content/Context;Landroid/graphics/Bitmap;Ljava/lang/String;I)Landroidx/concurrent/futures/ResolvableFuture;`
Warning in obj\Debug\110\lp\74\jl\classes.jar:androidx/browser/trusted/ConnectionHolder.class:
  Type `androidx.concurrent.futures.CallbackToFutureAdapter` was not found, it is required for default or static interface methods desugaring of `Landroidx/browser/trusted/ConnectionHolder;getServiceWrapper()Lcom/google/common/util/concurrent/ListenableFuture;`
Warning in synthesized for lambda desugaring:
  Type `androidx.concurrent.futures.CallbackToFutureAdapter$Resolver` was not found, it is required for default or static interface methods desugaring of `androidx.browser.trusted.-$$Lambda$ConnectionHolder$Rp3Hxy1AUk9cWatAvfzHyT3B2KI`
Touching "obj\Debug\110\stamp\_CompileToDalvik.stamp".
Deleting file "bin\Debug\com.companyname.app1.apk".
Copying file from "C:\Users\***\Desktop\App1\obj\Debug\110\android\bin\com.companyname.app1.apk" to "C:\Users\***\Desktop\App1\bin\Debug\com.companyname.app1.apk".
Creating "obj\Debug\110\android_debug_keystore.flag" because "AlwaysCreate" was specified.
Deleting file "C:\Users\***\Desktop\App1\bin\Debug\com.companyname.app1-Signed.apk".
E:\android-sdk\build-tools\30.0.2\zipalign.exe -p 4 "C:\Users\***\Desktop\App1\obj\Debug\110\android\bin\com.companyname.app1.apk" "bin\Debug\\com.companyname.app1-Signed.apk" 
C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -jar "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\apksigner.jar" sign --ks "C:\Users\***\AppData\Local\Xamarin\Mono for Android\debug.keystore" --ks-pass pass:android --ks-key-alias androiddebugkey --key-pass pass:android --min-sdk-version 23 --max-sdk-version 30  C:\Users\***\Desktop\App1\bin\Debug\com.companyname.app1-Signed.apk 
Signed android package 'bin\Debug\com.companyname.app1-Signed.apk'
Using cached value from RegisterTaskObject
Found device: emulator-5554
Creating "obj\Debug\110\upload.flag" because "AlwaysCreate" was specified.
Done building project "App1.csproj".
Build succeeded.
2>Deployment was successful to Pixel_4_API_29.
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
========== Deploy: 1 succeeded, 0 failed, 0 skipped ==========

Also, 2 questions:

• It seems like the Shared Runtime and Xamarin.Android API-XX Support packages are no longer installed onto the device. I've seen in this PR that this is expected behavior for the Shared Runtime. Is it also for the Xamarin.Android API-XX Support packages?
• I've seen that the Visual Studio UI doesn't offer a toggle for the "Use shared runtime" settings which toggles the AndroidUseSharedRuntime property in the csproj. Is this property now obsolete and can be removed? The default template still includes it

[dellis1972]

@tipa

I suspect you are hitting the same problem as #5532 (comment).

We found that on certain devices (mostly Samsung) the broadcast which gets the external directory was taking 30+ seconds to complete.
This broadcast is no longer required since the removal of the SharedRuntime.
There are some patched files attached to that issue which include the changes we made to remove the broadcast.
You can give those a try if you are willing. If you do remember to backup the existing files.

If it is still a problem we will need a full diagnostic deployment log in order to figure out where the slowdown is.
The following command should produce a msbuild.binlog file which the required data.

msbuild <foo.csproj> /restore /t:Install /bl /p=_FastDeploymentDiagnosticLogging=true

Please attach the file to this issue (you might have to zip it first).

You are correct the AndroidUseSharedRuntime is now redundant.

[dellis1972]

The Xamarin.Android API-XX Support package are also redundant. So that is expected behaviour as well

[dellis1972]

The fixed files we included in main by #5696

[tipa]

Thanks for your quick replay.
I do not experience any hangs and the problem occurs on various devices, I tested on the default simulator mainly.
I replaced the files anyways, but this caused VS to not start correctly any more. Errors like these seem to cause the problem:

A MEF Component threw an exception at runtime: Microsoft.VisualStudio.Composition.CompositionFailedException: An exception was thrown while initializing part "Xamarin.VisualStudio.Android.AndroidVirtualDeviceProvider". ---> System.IO.FileLoadException: Could not load file or assembly 'Mono.AndroidTools, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0cb85e4f39dfce87' or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515) ---> System.IO.FileLoadException: Could not load file or assembly 'file:///C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2019\ENTERPRISE\COMMON7\IDE\EXTENSIONS\XAMARIN.VISUALSTUDIO\Mono.AndroidTools.dll' or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515) ---> System.NotSupportedException: An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.
   --- End of inner exception stack trace ---
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, Boolean suppressSecurityChecks, StackCrawlMark& stackMark)
   at System.Reflection.Assembly.LoadFrom(String assemblyFile)
   at Microsoft.VisualStudio.Platform.VsAppDomainManager.FindAssembly(String name, IEnumerable`1 probingPaths, Boolean logTelemetry)
   at Microsoft.VisualStudio.Platform.VsAppDomainManager.InnerResolveHandler(String name)
   at Microsoft.VisualStudio.Platform.VsAppDomainManager.ResolveHandler(Object sender, ResolveEventArgs args)
   at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)
   --- End of inner exception stack trace ---
   at Xamarin.VisualStudio.Android.AndroidVirtualDeviceProvider..ctor(IAndroidProperties androidProperties, IAndroidVirtualDeviceProviderConfiguration config, IAndroidToolsService androidService, ITimerFactory timerFactory, IProcessManager processManager, JoinableTaskContext taskContext, ISolutionState solutionState, IEventStream stream, IAccelerationManager accelerationManager, IAndroidDeviceManagerTool deviceManagerTool, IBuildService buildService)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.RuntimePartLifecycleTracker.CreateValue()
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.Create()
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.MoveNext(PartLifecycleState nextState)
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.MoveToState(PartLifecycleState requiredState)
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.GetValueReadyToExpose()
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.<>c__DisplayClass15_0.<GetExportedValueHelper>b__0()
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.GetValueForImportElement(RuntimePartLifecycleTracker importingPartTracker, RuntimeImport import, RuntimeExport export, Func`3 lazyFactory)
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.GetValueForImportSite(RuntimePartLifecycleTracker importingPartTracker, RuntimeImport import)
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.RuntimePartLifecycleTracker.<CreateValue>b__11_0(RuntimeImport import)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.RuntimePartLifecycleTracker.CreateValue()
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.Create()
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.MoveNext(PartLifecycleState nextState)
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.MoveToState(PartLifecycleState requiredState)
   at Microsoft.VisualStudio.Composition.ExportProvider.PartLifecycleTracker.GetValueReadyToExpose()
   at Microsoft.VisualStudio.Composition.RuntimeExportProviderFactory.RuntimeExportProvider.<>c__DisplayClass15_0.<GetExportedValueHelper>b__0()

Here's the binlog:
msbuild.zip

[dellis1972]

@tipa what version of VS 2019 d16-9 are you running?
Did you replace the files in all the locations mentioned? It looks like there is one missing.

I'll take a look at the build log