[Xamarin.Android.Build.Tasks] Make all assemblies RID-specific (#8478)

Fixes: #8168

Context: #4337
Context: #8155

Context: 55e5c34
Context: 6836818
Context: 929e701
Context: c927026
Context: 2f19238

Issue #8155 noted a *fundamental* mismatch in
expectations between the Classic Xamarin.Android packaging worldview
and the .NET worldview: In Classic Xamarin.Android, all assemblies
are presumed to be architecture agnostic ("AnyCPU"), while in .NET:

 1. `System.Private.CoreLib.dll` was *always* an architecture-specific
    assembly (see #4337), and

 2. The .NET Trimmer is extensible and can apply ABI-specific changes
    to IL which *effectively* results in an architecture-specific
    assembly (#8155).  Meanwhile, there is no
    way of knowing that this is happening, and the trimmer doesn't
    mark the resulting assembly as architecture-specific.

We long tried to "paper over" this difference, by trying to find --
and preserve the "nature" of -- architecture-agnostic assemblies
(55e5c34, …).  Unfortunately, all attempts at trying to preserve the
concept of architecture-agnostic assemblies have failed; we're
fighting against .NET tooling in attempting to do so.

In commit 6836818 this came to a head: a long worked-on feature
LLVM Marshal Methods (8bc7a3e) had to be disabled because of
hangs within MAUI+Blazor Hybrid+.NET Android apps, and we suspect
that treating an assembly as architecture-agnostic when it was
"actually" architecture-specific is a plausible culprit.

Bite the bullet: there is no longer such a thing as an architecture-
agnostic assembly.  Treat *all* assemblies as if they were
architecture-specific.

Additionally, alter assembly packaging so that instead of using
`assemblies/assemblies*.blob` files (c927026), we instead store the
assemblies within `lib/ABI` of the `.apk`/`.aab`.

The Runtime config blob `rc.bin` is stored as `lib/ABI/libarc.bin.so`.

When `$(AndroidUseAssemblyStore)`=true, assemblies will be stored
within `lib/ABI/libassemblies.ABI.blob.so`, e.g.
`lib/arm64-v8a/libassemblies.arm64-v8a.blob.so`.

When `$(AndroidUseAssemblyStore)`=false and Fast Deployment is *not*
used, then assemblies are stored individually within `lib/ABI` as
compressed assembly data, with the following "name mangling"
convention:

  * Regular assemblies: `lib_` + Assembly File Name + `.so`
  * Satellite assemblies:
    `lib-` + culture + `-` + Assembly File Name + `.so`

For example, consider this selected `unzip -l` output:

	% unzip -l bin/Release/net9.0-android/*-Signed.apk | grep lib/arm64-v8a
	   723560  01-01-1981 01:01   lib/arm64-v8a/libSystem.IO.Compression.Native.so
	    70843  01-01-1981 01:01   lib/arm64-v8a/lib_Java.Interop.dll.so
	   157256  01-01-1981 01:01   lib/arm64-v8a/libaot-Java.Interop.dll.so
	     1512  01-01-1981 01:01   lib/arm64-v8a/libarc.bin.so

  * `libSystem.IO.Compression.Native.so` is a native shared library
    from .NET
  * `lib_Java.Interop.dll.so` is compressed assembly data for
    `Java.Interop.dll`
  * `libaot-Java.Interop.dll.so` contains Profiled AOT output for
    `Java.Interop.dll`
  * `libarc.bin.so` is the `rc.bin` file used by .NET runtime startup

Additionally, note that Android limits the characters that can be
used in native library filenames to the regex set `[-._A-Za-z0-9]`.

TODO: No error checking is done to ensure that "Assembly File Name"
stays within the limits of `[-.A-Za-z0-9]`, e.g. if you set
`$(AssemblyName)=Emoji😅` *and `$(AndroidUseAssemblyStore)`=false,
then we'll try to add `lib/arm64-v8a/lib_Emoji😅.dll.so`, which will
fail at runtime.  This works when `$(AndroidUseAssemblyStore)`=true,
which is the default.

Pros:

  * We're no longer fighting against .NET tooling features such as
    ILLink Substitutions.

  * While `.aab` files will get larger, we expect that the actual
    `.apk` files sent to Android devices from the Google Play
    Store will be *smaller*, as the Google Play Store would always
    preserve/transmit *all* `assemblies/assemblies*.blob` files,
    while now it will be able to remove `lib/ABI/*` for unsupported
    ABIs.
    
Cons:

  * `.apk` files containing more than one ABI ***will get larger***,
    as there will no longer be "de-duping" of architecture-agnostic
    assembly data.  We don't consider this a significant concern, as
    we believe `.aab` is the predominant packaging format.

~~ All assemblies are architecture-specific ~~

Assembly pre-processing changes so that every assembly ends up in
every target architecture batch, regardless of whether its MVID
differs from its brethren or not.  This is done very early in the
build process on our side, where we make sure that each assembly
either has the `%(Abi)` metadata or is given one, and is placed in
the corresponding batch.  Further processing of those batches is
"parallel", in that no code attempts to de-duplicate the batches.


~~ Impact on Fast Deployment, `$(IntermediateOutputPath)` ~~

The changes also required us to place all the assemblies in new
locations on disk within `$(IntermediateOutputPath)` when building
the application.  (Related: 2f19238.)  Assemblies are now placed in
subdirectories named after either the target architecture/ABI or the
.NET `$(RuntimeIdentifier)`, e.g.
`obj/Release/netX.Y-android/android-arm64`.  This, in turn, affects
e.g. Fast Deployment as now the synchronized content is in the
`…/.__override__/ABI` directory on device, instead of just in
`…/.__override__`.


~~ File Formats ~~

The assembly store format (c927026) is updated to use the following
structures:

	struct AssemblyStoreHeader {
	    uint32_t magic;
	    uint32_t version;
	    uint32_t entry_count;               // Number of assemblies in the store
	    uint32_2 index_entry_count;
	    uint32_t index_size;
	};
	struct AssemblyStoreIndexEntry {
	    intptr_t name_hash;	                // xxhash of assembly filename
	    uint32_t descriptor_index;          // index into `descriptors` array
	};
	struct AssemblyStoreEntryDescriptor {
	    uint32_t mapping_index;             // index into an internal runtime array
	    uint32_t data_offset;               // index into `data` for assembly `.dll`
	    uint32_t data_size;                 // size of assembly, in bytes
	    uint32_t debug_data_offset;         // index into `data` for assembly `.pdb`; 0 if not present
	    uint32_t debug_data_size;           // size of `.pdb`, in bytes; 0 if not present
	    uint32_t config_data_offset;        // index into `data` for assembly `.config`; 0 if not present
	    uint32_t config_data_size;          // size of `.config`, in bytes; 0 if not present
	};
	struct AssemblyStoreAssemblyInfo {
	    uint32_t length;                    // bytes
	    uint8_t  name[length];
	};

`libassemblies.ABI.blob.so` has the following format, and is *not* a
valid ELF file:

	AssemblyStoreHeader                 header {…};
	AssemblyStoreIndexEntry             index [header.index_entry_count];
	AssemblyStoreAssemblyDescriptor     descriptors [header.entry_count];
	AssemblyStoreAssemblyInfo           names [header.entry_count];
	uint8_t data[];

Author: grendello

Directory.Build.props

@@ -48,7 +48,7 @@
     <NuGetApiPackageVersion>5.4.0</NuGetApiPackageVersion>
     <LZ4PackageVersion>1.1.11</LZ4PackageVersion>
     <MonoOptionsVersion>6.12.0.148</MonoOptionsVersion>
-    <SystemCollectionsImmutableVersion>6.0.0</SystemCollectionsImmutableVersion>
+    <SystemCollectionsImmutableVersion>8.0.0</SystemCollectionsImmutableVersion>
     <SystemRuntimeCompilerServicesUnsafeVersion>6.0.0</SystemRuntimeCompilerServicesUnsafeVersion>
     <ELFSharpVersion>2.13.1</ELFSharpVersion>
     <HumanizerVersion>2.14.1</HumanizerVersion>

Xamarin.Android.sln

@@ -109,7 +109,7 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "decompress-assemblies", "to
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "tmt", "tools\tmt\tmt.csproj", "{1A273ED2-AE84-48E9-9C23-E978C2D0CB34}"
 EndProject
-Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "assembly-store-reader", "tools\assembly-store-reader\assembly-store-reader.csproj", "{DA50FC92-7FE7-48B5-BDB6-CDA57B37BB51}"
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "assembly-store-reader", "tools\assembly-store-reader-mk2\assembly-store-reader.csproj", "{DA50FC92-7FE7-48B5-BDB6-CDA57B37BB51}"
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Java.Interop.Tools.JavaTypeSystem", "external\Java.Interop\src\Java.Interop.Tools.JavaTypeSystem\Java.Interop.Tools.JavaTypeSystem.csproj", "{4EFCED6E-9A6B-453A-94E4-CE4B736EC684}"
 EndProject

build-tools/scripts/TestApks.targets

@@ -111,7 +111,14 @@
         Timeout="60000"
     />
     <Xamarin.Android.Tools.BootstrapTasks.Adb
-        Arguments="$(_AdbTarget) logcat -G 4M"
+        Arguments="$(_AdbTarget) logcat -G 256M"
+        IgnoreExitCode="True"
+        ToolExe="$(AdbToolExe)"
+        ToolPath="$(AdbToolPath)"
+        Timeout="60000"
+    />
+    <Xamarin.Android.Tools.BootstrapTasks.Adb
+        Arguments="$(_AdbTarget) logcat -c"
         IgnoreExitCode="True"
         ToolExe="$(AdbToolExe)"
         ToolPath="$(AdbToolPath)"

src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.AssemblyResolution.targets

@@ -120,11 +120,14 @@ _ResolveAssemblies MSBuild target.
       <_ResolvedSymbolFiles   Include="@(ResolvedFileToPublish)" Condition=" '%(ResolvedFileToPublish.Extension)' == '.pdb' " />
       <_ResolvedJavaLibraries Include="@(ResolvedFileToPublish)" Condition=" '%(ResolvedFileToPublish.Extension)' == '.jar' " />
     </ItemGroup>
+
+    <!-- All assemblies must be per-RID, thus no `->Distinct()` on `InputAssemblies` or `ResolvedSymbols` items -->
     <ProcessAssemblies
         RuntimeIdentifiers="@(_RIDs)"
-        InputAssemblies="@(_ResolvedAssemblyFiles->Distinct())"
+        DesignTimeBuild="$(DesignTimeBuild)"
+        InputAssemblies="@(_ResolvedAssemblyFiles)"
         InputJavaLibraries="@(_ResolvedJavaLibraries->Distinct())"
-        ResolvedSymbols="@(_ResolvedSymbolFiles->Distinct())"
+        ResolvedSymbols="@(_ResolvedSymbolFiles)"
         AndroidIncludeDebugSymbols="$(AndroidIncludeDebugSymbols)"
         PublishTrimmed="$(PublishTrimmed)">
       <Output TaskParameter="OutputAssemblies" ItemName="_ProcessedAssemblies" />

src/Xamarin.Android.Build.Tasks/Tasks/BuildApk.cs

@@ -4,6 +4,8 @@
 using Microsoft.Build.Framework;
 using Microsoft.Android.Build.Tasks;
 
+using Xamarin.Android.Tools;
+
 namespace Xamarin.Android.Tasks
 {
 	public class GenerateCompressedAssembliesNativeSourceFiles : AndroidTask
@@ -41,38 +43,54 @@ void GenerateCompressedAssemblySources ()
 				return;
 			}
 
-			var assemblies = new SortedDictionary<string, CompressedAssemblyInfo> (StringComparer.Ordinal);
-			foreach (ITaskItem assembly in ResolvedAssemblies) {
-				if (bool.TryParse (assembly.GetMetadata ("AndroidSkipAddToPackage"), out bool value) && value) {
-					continue;
-				}
+			Dictionary<AndroidTargetArch, Dictionary<string, ITaskItem>> perArchAssemblies = MonoAndroidHelper.GetPerArchAssemblies (
+				ResolvedAssemblies,
+				SupportedAbis,
+				validate: true,
+				shouldSkip: (ITaskItem asm) => bool.TryParse (asm.GetMetadata ("AndroidSkipAddToPackage"), out bool value) && value
+			);
+			var archAssemblies = new Dictionary<AndroidTargetArch, Dictionary<string, CompressedAssemblyInfo>> ();
+			var counters = new Dictionary<AndroidTargetArch, uint> ();
+
+			foreach (var kvpPerArch in perArchAssemblies) {
+				AndroidTargetArch arch = kvpPerArch.Key;
+				Dictionary<string, ITaskItem> resolvedArchAssemblies = kvpPerArch.Value;
+
+				foreach (var kvp in resolvedArchAssemblies) {
+					ITaskItem assembly = kvp.Value;
+
+					if (!archAssemblies.TryGetValue (arch, out Dictionary<string, CompressedAssemblyInfo> assemblies)) {
+						assemblies = new Dictionary<string, CompressedAssemblyInfo> (StringComparer.OrdinalIgnoreCase);
+						archAssemblies.Add (arch, assemblies);
+					}
 
-				var assemblyKey = CompressedAssemblyInfo.GetDictionaryKey (assembly);
-				if (assemblies.ContainsKey (assemblyKey)) {
-					Log.LogDebugMessage ($"Skipping duplicate assembly: {assembly.ItemSpec}");
-					continue;
-				}
+					var assemblyKey = CompressedAssemblyInfo.GetDictionaryKey (assembly);
+					if (assemblies.ContainsKey (assemblyKey)) {
+						Log.LogDebugMessage ($"Skipping duplicate assembly: {assembly.ItemSpec} (arch {MonoAndroidHelper.GetAssemblyAbi(assembly)})");
+						continue;
+					}
 
-				var fi = new FileInfo (assembly.ItemSpec);
-				if (!fi.Exists) {
-					Log.LogError ($"Assembly {assembly.ItemSpec} does not exist");
-					continue;
-				}
+					var fi = new FileInfo (assembly.ItemSpec);
+					if (!fi.Exists) {
+						Log.LogError ($"Assembly {assembly.ItemSpec} does not exist");
+						continue;
+					}
 
-				assemblies.Add (assemblyKey, new CompressedAssemblyInfo (checked((uint)fi.Length)));
-			}
 
-			uint index = 0;
-			foreach (var kvp in assemblies) {
-				kvp.Value.DescriptorIndex = index++;
+					if (!counters.TryGetValue (arch, out uint counter)) {
+						counter = 0;
+					}
+					assemblies.Add (assemblyKey, new CompressedAssemblyInfo (checked((uint)fi.Length), counter++, arch, Path.GetFileNameWithoutExtension (assembly.ItemSpec)));
+					counters[arch] = counter;
+				}
 			}
 
 			string key = CompressedAssemblyInfo.GetKey (ProjectFullPath);
 			Log.LogDebugMessage ($"Storing compression assemblies info with key '{key}'");
-			BuildEngine4.RegisterTaskObjectAssemblyLocal (key, assemblies, RegisteredTaskObjectLifetime.Build);
-			Generate (assemblies);
+			BuildEngine4.RegisterTaskObjectAssemblyLocal (key, archAssemblies, RegisteredTaskObjectLifetime.Build);
+			Generate (archAssemblies);
 
-			void Generate (IDictionary<string, CompressedAssemblyInfo> dict)
+			void Generate (Dictionary<AndroidTargetArch, Dictionary<string, CompressedAssemblyInfo>> dict)
 			{
 				var composer = new CompressedAssembliesNativeAssemblyGenerator (Log, dict);
 				LLVMIR.LlvmIrModule compressedAssemblies = composer.Construct ();