[Xamarin.Android.Build.Tasks] Fast Deployment v2.0

[dellis1972]

Fixes #5009
Fixes #5147
Fixes #4996

The Fast Deployment system used for debugging Xamarin.Android
apps has been completely re-written. This is mostly due to
changes in the android OS which means we can no longer use
the external storage directory to store assemblies.

Fast Deployment works by not including files which change often,
like assemblies, in the actual apk. This means the .apk will
mostly not need to be re-installed during a debugging/development
session. Instead the assemblies are "Fast Deployed" to a special
directory where a debug version of our runtime knows where to find
them.

Historically this was on the external storage directory such as

`/storage/emulated/0/Android/data/com.some.package`
`/mnt/shell/emulated/0/Android/data/com.some.package`
`/storage/sdcard/Android/data/com.some.package`

With the advent of Android 11, these directories are no longer accessable.
So instead we need to deploy the assemblies into the app internal files
directory. This is usually located in /data/data/com.some.package. This
is not a global writable folder, so we need to use the run-as tool to
run all the commands to copy the files into that diectory.

The run-as tool does not always work on older devices. So from this point on
Fast Deployment will only be available on API 21 devices or newer. If a certain
device does not support the run-as tool, then you can always fall back to
debugging without Fast Deployment. While this is slower, it should still work
on most devices.

The Enhanced Fast Deployment mode is still available with this new system.
This will deploy both assemblies, native libraries, typemaps and dexes to the files
directory. Support for Fast Deploying resources and assets via that system was
removed in commit f0d565fe. This was becuase it required the use of
depreicated API's to work.

The Shared Runtime has also be removed in this new system. Before we used to deploy
the BCL and API specific assemblies via seperate .apks. This new system removes
the need for that. All the BCL and API specific assemblies will be deployed to the
files directory like all the other assemblies.

The new system is on par with the existing system when it comes to speed. More
improvements are planned in future releases which should make it much quicker.

Using the samples\HelloWorld project these are the performance differences
using HelloWorld.csproj /restore /t:Install /v:n

Old From Clean Time Elapsed 00:00:11.42
Old C# Change Time Elapsed 00:00:02.58

New From Clean Time Elapsed 00:00:11.78
New C# Change Time Elapsed 00:00:02.43

[brendanzagaeski]

Draft release notes

No hurry on adding a draft release notes while this pull request is still in draft and under active revision. I'll leave a first little comment now just to give a starting place for when the pull request is ready to get into the documentation phase. Thanks!

Some (probably all obvious) items and questions that might be relevant to cover in the eventual release note:

• By chance, for the initial release, will users have the option of switching back to the previous fast deployment approach with a deprecation warning?

• Relatedly, will users control fast deployment the same way as in the previous implementation? For example, will the $(AndroidFastDeploymentType) property still be used, and will the Assemblies and Dexes options be the same?

• From glancing at the corresponding pieces in monodroid, I think there might be some Android OS version requirements that the release note can mention.

• And then of course, a little comparison of the exciting user-facing timing performance and other feature benefits of the new implementation will be good to include!

[jonpryor]

Squash-and-merge commit message:

Fixes: https://github.com/xamarin/xamarin-android/issues/4996
Fixes: https://github.com/xamarin/xamarin-android/issues/5009
Fixes: https://github.com/xamarin/xamarin-android/issues/5147

Changes: https://github.com/xamarin/monodroid/compare/1ac5333ec5e90b4e1fb9b324714811effdede37f...767f647151936303c294d154d0d0a4da8b601464

  * xamarin/monodroid@767f64715: [msbuild] Fast Deployment v2.0 (#1090)
  * xamarin/monodroid@0f04ba56d: Merge pull request #1115 from xamarin/remove-xreitem
  * xamarin/monodroid@d75341fc3: Remove provisionator file completely
  * xamarin/monodroid@b62e8c693: Replace XreItem with supported Xcode and JavaJDK syntax

The Fast Deployment system used for debugging Xamarin.Android apps has
been completely re-written.  This is mostly due to changes in Android
which means we can no longer use the external storage directory to
store assemblies.

Fast Deployment works by not including files which change often,
like assemblies, in the actual apk.  This means the `.apk` will mostly
not need to be re-installed during a debugging/development session.
Instead the assemblies are "Fast Deployed" to a special directory where
a debug version of our runtime knows where to find them.

Historically this was on the external storage directory such as

	/storage/emulated/0/Android/data/com.some.package
	/mnt/shell/emulated/0/Android/data/com.some.package
	/storage/sdcard/Android/data/com.some.package

With Android 11, these directories are no longer accessible.  Instead,
we need to deploy the assemblies into the app's internal `files`
directory.  This is usually located in `/data/data/@PACKAGE_NAME@`.
This is not a global writable folder, so we need to use the `run-as`
tool to run all the commands to copy the files into that directory.

The `run-as` tool does not always work on older devices.  From this
point on Fast Deployment v2 will only be available on API-21+ devices.
If a certain device does not support the `run-as` tool, then you can
always fall back to debugging without Fast Deployment.  While this is
slower, it should still work on most devices.

[`$(AndroidFastDeploymentType)`][0] is still supported.  This will
deploy both assemblies, native libraries, typemaps, and `.dex` files to
the `files` directory.  Support for Fast Deploying Android resources
and assets was removed in commit f0d565fe, as it required the use of
deprecated API's to work.

The Shared Runtime has also be removed in this new system.  Previously,
we used to deploy the BCL and API specific assemblies via separate
`.apk` files.  This new system removes the need for that.  All the BCL
and API specific assemblies will be deployed to the `files` directory
like all the other assemblies.

The new system is on par with the existing system when it comes to
speed.  More improvements are planned in future releases which should
make it much quicker.

Using the `samples\HelloWorld` project these are the performance
differences using `HelloWorld.csproj /restore /t:Install /v:n`:

  * Deploy "from Clean"
    * v1: 00:00:11.42
    * v2: 00:00:11.78 [3% longer]
  * Incrementally deploy C#-based change
    * v1: 00:00:02.58
    * v2: 00:00:02.43 [6% faster]

[0]: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-properties#androidfastdeploymenttype

[MagicAndre1981]

I always disabled FastDeployment because I use ADB over Wifi most time and had issues in the past. Does this v2 fix it? Does it work fine with the buildin wireless debugging of Android 11? @dellis1972

[dellis1972]

@MagicAndre1981 I'm not sure I can test it and see though :)

[dellis1972]

@MagicAndre1981 the new fast dev system doesn't change how the debugger attaches, which seems to be the actual problem with #3018. So the new system should still be able to deploy over wifi, the question is if the debugger can attach. Its possible there might have been fix in the adb port forwarding over wifi that might make it work in the latest sdk versions.

[MagicAndre1981]

ok, when will this change be part of VS2019? 16.9? @dellis1972

[dellis1972]

@MagicAndre1981 it will be in 16.9 unless something really bad happens (like it breaks everyone :P).

I just tested out Debugging Unit tests on a Wifi connected device (Android 11) with this new system and it all seems to work without any issues :) Those tests don't run inside VS2019 though so it might still be an issue from VS. But they do use the mono SoftDebugger which is what VS uses so it might work.

I seem to remember you use custom ROMs, so it will depend on what changes you made to the ROM images.

[MagicAndre1981]

I only used custom ROM in past for an older phone, now the Pixel3a supports it officially with Android11. So I'll try it with 16.9 @dellis1972. Hopefully until 16.9, my blocker issue gets also fixed because XF5 will require VS2019 and I have to ditch my VS2017 VM