[Xamarin.Android.Build.Tasks] Fast Deployment v2.0

Fixes #5009
Fixes #5147
Fixes #4996

The Fast Deployment system used for debugging Xamarin.Android
apps has been completely re-written. This is mostly due to
changes in the android OS which means we can no longer use
the external storage directory to store assemblies.

Fast Deployment works by not including files which change often,
like assemblies, in the actual apk. This means the .apk will
mostly not need to be re-installed during a debugging/development
session. Instead the assemblies are "Fast Deployed" to a special
directory where a debug version of our runtime knows where to find
them.

Historically this was on the external storage directory such as

    `/storage/emulated/0/Android/data/com.some.package`
    `/mnt/shell/emulated/0/Android/data/com.some.package`
    `/storage/sdcard/Android/data/com.some.package`

With the advent of Android 11, these directories are no longer accessable.
So instead we need to deploy the assemblies into the app internal `files`
directory. This is usually located in `/data/data/com.some.package`. This
is not a global writable folder, so we need to use the `run-as` tool to
run all the commands to copy the files into that diectory.

The `run-as` tool does not always work on older devices. So from this point on
Fast Deployment will only be available on API 21 devices or newer. If a certain
device does not support the `run-as` tool, then you can always fall back to
debugging without Fast Deployment. While this is slower, it should still work
on most devices.

The [Enhanced Fast Deployment](~/android/deploy-test/building-apps/build-properties.md#AndroidFastDeploymentType) mode is still available with this new system.
This will deploy both assemblies, native libraries, typemaps and dexes to the `files`
directory. Support for Fast Deploying resources and assets via that system was
removed in commit [f0d565f](f0d565f). This was becuase it required the use of
depreicated API's to work.

The Shared Runtime has also be removed in this new system. Before we used to deploy
the BCL and API specific assemblies via seperate .apks. This new system removes
the need for that. All the BCL and API specific assemblies will be deployed to the
`files` directory like all the other assemblies.

The new system is on par with the existing system when it comes to speed. More
improvements are planned in future releases which should make it much quicker.

Using the `samples\HelloWorld` project these are the performance differences
using `HelloWorld.csproj /restore /t:Install /v:n`

Old From Clean Time Elapsed 00:00:11.42
Old C# Change Time Elapsed 00:00:02.58

New From Clean Time Elapsed 00:00:11.78
New C# Change Time Elapsed 00:00:02.43

Author: dellis1972

.external

@@ -1,2 +1,2 @@
-xamarin/monodroid:master@1ac5333ec5e90b4e1fb9b324714811effdede37f
+xamarin/monodroid:master@767f647151936303c294d154d0d0a4da8b601464
 mono/mono:2020-02@be2226b5a1c57df065efc4c1cf008d581e5cec7d

.vscode/settings.json

@@ -4,6 +4,7 @@
         "bin/TestDebug/MSBuildDeviceIntegration/MSBuildDeviceIntegration.dll",
         "bin/TestDebug/net472/Xamarin.Android.Build.Tests.dll",
         "bin/TestDebug/Xamarin.Android.Build.Tests.Commercial.dll",
+        "bin/TestRelease/MSBuildDeviceIntegration/MSBuildDeviceIntegration.dll",
     ],
     "cmake.configureOnOpen": false
 }

Documentation/guides/building-apps/build-process.md

@@ -36,38 +36,35 @@ In broad terms, there are two types of Android application packages
 Not coincidentally, these match the MSBuild `Configuration` which
 produces the package.
 
-## Shared Runtime
-
-The *shared runtime* is a pair of additional Android packages which
-provide the Base Class Library (`mscorlib.dll`, etc.) and the
-Android binding library (`Mono.Android.dll`, etc.). Debug builds
-rely upon the shared runtime in lieu of including the Base Class Library and
-Binding assemblies within the Android application package, allowing the
-Debug package to be smaller.
-
-The shared runtime may be disabled in Debug builds by setting the
-[`$(AndroidUseSharedRuntime)`](~/android/deploy-test/building-apps/build-properties.md#androidusesharedruntime)
-property to `False`.
-
 <a name="Fast_Deployment"></a>
 
 ## Fast Deployment
 
-*Fast deployment* works in concert with the shared runtime to further
-shrink the Android application package size. This is done by not
-bundling the app's assemblies within the package. Instead, they are
-copied onto the target via `adb push`. This process speeds up the
-build/deploy/debug cycle because if *only* assemblies are changed,
-the package is not reinstalled. Instead, only the updated assemblies are
-re-synchronized to the target device.
+*Fast deployment* works by further shrinking Android application
+package size. This is done by not bundling the app's assemblies
+within the package. Instead, the are deployed directly to the
+application internal `files` directory. This is usually located
+in `/data/data/com.some.package`. This is not a global writable
+folder, so we need to use the `run-as` tool to run all the
+commands to copy the files into that diectory.
 
-Fast deployment is known to fail on devices which block `adb` from
-synchronizing to the directory
-`/data/data/@PACKAGE_NAME@/files/.__override__`.
+This process speeds up the build/deploy/debug cycle because
+if *only* assemblies are changed, the package is not reinstalled.
+Instead, only the updated assemblies are re-synchronized to the
+target device.
+
+Fast deployment is known to fail on devices which block `run-as`. This
+is usually devices of API 20 and lower.
 
 Fast deployment is enabled by default, and may be disabled in Debug builds
 by setting the `$(EmbedAssembliesIntoApk)` property to `True`.
 
+The [Enhanced Fast Deployment](~/android/deploy-test/building-apps/build-properties.md#AndroidFastDeploymentType) mode can
+be used in conjunction with this feature to speed up deployments even further.
+This will deploy both assemblies, native libraries, typemaps and dexes to the `files`
+directory. But you should only really need to enable this if you are changing
+native libraries, bindings or Java code.
+
 ## MSBuild Projects
 
 The Xamarin.Android build process is based on MSBuild, which is also

Documentation/guides/building-apps/build-properties.md

@@ -418,6 +418,8 @@ compiler.
 
 Added in Xamarin.Android 10.2.
 
+<a name="AndroidFastDeploymentType"></a>
+
 ## AndroidFastDeploymentType
 
 A `:` (colon)-separated list
@@ -433,12 +435,16 @@ faster.) Valid values include:
 
 - `Assemblies`: Deploy application assemblies.
 
-- `Dexes`: Deploy `.dex` files, Android Resources, and Android
-  Assets. **This value can *only* be used on devices running
+- `Dexes`: Deploy `.dex` files, native libraries and typemaps.
+  **This value can *only* be used on devices running
   Android 4.4 or later (API-19).**
 
 The default value is `Assemblies`.
 
+Support for Fast Deploying resources and assets via that system was
+removed in commit [f0d565fe](https://github.com/xamarin/xamarin-android/commit/f0d565fe4833f16df31378c77bbb492ffd2904b9). This was becuase it required the use of
+deprecated API's to work.
+
 **Experimental**. Added in Xamarin.Android 6.1.
 
 ## AndroidGenerateJniMarshalMethods
@@ -1003,18 +1009,6 @@ than `aapt`.
 
 Added in Xamarin.Android 8.1.
 
-## AndroidUseSharedRuntime
-
-A boolean property that
-determines whether the *shared runtime packages* are required in
-order to run the Application on the target device. Relying on the
-shared runtime packages allows the Application package to be
-smaller, speeding up the package creation and deployment process,
-resulting in a faster build/deploy/debug turnaround cycle.
-
-This property should be `True` for Debug builds, and `False` for
-Release projects.
-
 ## AndroidVersionCodePattern
 
 A string property which allows

Documentation/guides/messages/README.md

@@ -69,6 +69,15 @@ ms.date: 01/24/2020
 + [XA0122](xa0122.md): Assembly '{assembly}' is using a deprecated attribute '[assembly: Java.Interop.DoNotPackageAttribute]'. Use a newer version of this NuGet package or notify the library author.
 + XA0123: Removing {issue} from {propertyName}. Lint {version} does not support this check.
 + [XA0124](xa0124.md): Interpreter is not supported by the x86 ABI
++ [XA0125](xa0125.md): `{Project}` is using a deprecated debug information level.
+Set the debugging information to Portable in the Visual Studio project property pages or edit the project file in a text editor and set the 'DebugType' MSBuild property to 'portable' to use the newer, cross-platform debug information level.
+If this file comes from a NuGet package, update to a newer version of the NuGet package or notify the library author.
++ [XA0126](xa0126.md): Error installing FastDev Tools. This device does not support Fast Deployment. Please rebuild your app using `EmbedAssembliesIntoApk = True`.
++ [XA0127](xa0127.md): There was an issue deploying {destination} using {FastDevTool}. We encountered the following error {output}. Please rebuild your app using `EmbedAssembliesIntoApk = True`.
++ [XA0128](xa0128.md): Stdio Redirection is enabled. Please disable it to use Fast Deployment.
++ [XA0129](xa0129.md): Error deploying `{File}`. Please disable fast deployment in the Visual Studio project property pages or edit the project file in a text editor and set the 'EmbedAssembliesIntoApk' MSBuild property to 'true'.
++ [XA0130](xa0130.md): Sorry. Fast deployment is only supported on devices running Android 5.0 (API level 21) or higher.
+Please disable fast deployment in the Visual Studio project property pages or edit the project file in a text editor and set the 'EmbedAssembliesIntoApk' MSBuild property to 'true'.
 
 ## XA1xxx: Project related
 
@@ -233,6 +242,7 @@ and `NNN` is a 3 digit number indicating the type of the unhandled `Exception`.
 * `DES` - `Desugar`
 * `DJL` - `DetermineJavaLibrariesToCompile`
 * `DX8` - `D8`
+* `FD`  - `FastDeploy`
 * `FLB` - `FindLayoutsToBind`
 * `FLT` - `FilterAssemblies`
 * `GAD` - `GetAndroidDefineConstants`

Documentation/guides/messages/xa0119.md

@@ -31,12 +31,10 @@ Remove the following options from `Debug` configurations:
 
 * `<AndroidLinkMode>None</AndroidLinkMode>`
 * `<EmbedAssembliesIntoApk>False</EmbedAssembliesIntoApk>`
-* `<AndroidUseSharedRuntime>True</AndroidUseSharedRuntime>`
 
 *DO* use the following options for `Release` configurations:
 
 * `<EmbedAssembliesIntoApk>True</EmbedAssembliesIntoApk>`
-* `<AndroidUseSharedRuntime>False</AndroidUseSharedRuntime>`
 
 Consider submitting a [bug][bug] if you are getting one of these
 warnings under normal circumstances.

Documentation/guides/messages/xa0126.md

@@ -0,0 +1,20 @@
+---
+title: Xamarin.Android error XA0126
+description: XA0126 error code
+ms.date: 08/07/2020
+---
+# Xamarin.Android error XA0126
+
+## Issue
+
+This issue happens when you are trying to use fast deployment on a device which
+does not support it. Fast deployment requires features which are not available
+on devices running API 20 or lower. The Fast Deployment system makes use of the
+[`run-as`](https://android.googlesource.com/platform/system/core.git/+/refs/heads/master/run-as/run-as.cpp#42) feature of the Android OS. This feature was either not available or had
+limited capabilities in API 20 and earlier.
+
+## Solution
+
+Disable Fast Deployment by setting  `EmbedAssembliesIntoApk = True` in your .csproj.
+Or turn off `Fast Deployment` in the IDE. You will still be able to debug on the device,
+all the required files will be packaged inside the .apk.

Documentation/guides/messages/xa0127.md

@@ -0,0 +1,19 @@
+---
+title: Xamarin.Android error XA0127
+description: XA0127 error code
+ms.date: 08/07/2020
+---
+# Xamarin.Android error XA0127
+
+## Issue
+
+If you encounter this issue something unexpected happened with the Fast Deployment
+native tooling. This is most likely to be a native crash.
+
+## Solution
+
+To work around the issue disable Fast Deployment by setting  `EmbedAssembliesIntoApk = True`
+in your .csproj. Or turn off `Fast Deployment` in the IDE. You will still be able to debug
+on the device, all the required files will be packaged inside the .apk.
+
+In addition raise an issue at [https://github.com/xamarin/xamarin-android/issues/new/choose](https://github.com/xamarin/xamarin-android/issues/new/choose). Please provide the deployment log and as much detail as possible.

Documentation/guides/messages/xa0128.md

@@ -0,0 +1,26 @@
+---
+title: Xamarin.Android error XA0128
+description: XA0128 error code
+ms.date: 08/07/2020
+---
+# Xamarin.Android error XA0128
+
+## Issue
+
+The Fast Deployment system relys on reading the output from the various system applications
+on the device. This is done by reading `System.out`, which by default is where all output
+will be redirected. If `log.redirect-stdio` this will cause all of the Fast Deployment tooling
+to fail.
+
+## Solution
+
+Disable the redirection of stdio in order to use Fast Deployment. This can be done using
+the following commands.
+
+```
+adb shell stop
+adb shell setprop log.redirect-stdio false
+adb shell start
+```
+
+Depending on the device you might need to run `adb root` before running the above commands.

Documentation/guides/messages/xa0129.md

@@ -0,0 +1,19 @@
+title: Xamarin.Android error XA0129
+description: XA0129 error code
+ms.date: 08/07/2020
+---
+# Xamarin.Android error XA0129
+
+## Issue
+
+This issue happens when you are trying to use fast deployment on a device which
+does not support it. In this case both the normal and backup types of fast
+deployment failed. The Fast Deployment system makes use of the
+[`run-as`](https://android.googlesource.com/platform/system/core.git/+/refs/heads/master/run-as/run-as.cpp#42) feature of the Android OS. This feature was either not available or had
+limited capabilities in API 20 and earlier
+
+## Solution
+
+Disable Fast Deployment by setting  `EmbedAssembliesIntoApk = True` in your .csproj.
+Or turn off `Fast Deployment` in the IDE. You will still be able to debug on the device,
+all the required files will be packaged inside the .apk.