[Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183)

We have been getting on and off reports of the following build errors
in .NET for Android for quite a while now:

	Resources/values/styles.xml(2): error APT2260: resource attr/colorPrimary (aka Microsoft.Maui:attr/colorPrimary) not found.

It was always very confusing as to why this was happening.  No one
ever seemed to be able to figure out why these files were deleted.

Well it turns out they were not deleted, just not included. 
While building dotnet/maui I came across this issue and managed to
capture a `.binlog`.

It turns out that the `_CleanMonoAndroidIntermediateDir` target was
running, and it was deleting certain files.  This target was running
because `_CleanIntermediateIfNeeded` was running.  That was running
because the `build.props` file had changed, which was because the
NuGet `project.assets.json` file had a new timestamp!

On an incremental build!

So it looks like NuGet sometimes touches the `project.assets.json`
file even if it does not change.  We can't blame them for that as we
do similar things.

The next confusing thing and the principal cause is that the
`libraryprojectimports.cache` was not present.  However, the
`_ResolveLibraryProjectImports.stamp` file *was* present.  This means
that the `_ResolveLibraryProjectImports` target does NOT run, so we
end up NOT including ANY of the resource `res.zip` files when calling
`aapt2`.  The `_ResolveLibraryProjectImports` target did not include
`libraryprojectimports.cache` in its `Outputs`, so if the file did
not exist, but the stamp file *did*, then the target would not run.

It is confusing because in the `_CleanMonoAndroidIntermediateDir` we
delete the `libraryprojectimports.cache` and the entire
`$(IntermediateOutputPath)stamp` directory.

After much searching I did something while maui was opened in VSCode:
I deleted its `artifacts` directory to clean up and start a new build
and ..... it magically re-appeared!!!

Then I thought... Design Time Builds!!!!

So the problem we have is this: in our current system the
Design Time Builds and the main builds all use the same stamp files!!!
What was happening is a design time build was running before the main
build, and it was creating a design time cache file into
`$(_AndroidIntermediateDesignTimeBuildDirectory)`
(`$(IntermediateOutputPath)designtime`), but place the stamp files
INTO THE SAME LOCATION as the main build would.  As a result the main
build would skip the `_ResolveLibraryProjectImports` target
COMPLETELY even if the output cache file was not present.  Once that
happens it will NEVER recover until you delete the bin/obj directories.

This only happened occasionally because it would depend on if you
have the project open in a IDE and when the IDE decided to run a
Design Time Build.

Fix this by giving design-time builds their own stamp directory.
It might cause some targets to run during a design time build but I
think that is preferred to a failing main build.

To work around the NuGet causing a build when it touches a the
`project.assets.json` file we now use `<GetFileHash/>`.  This means
we only refresh the library projects if the contents of the
`project.assets.json` file change, rather than the timestamp.

Finally, there was a bug where we would automatically fallback to the
NuGet lock file even if we found the `project.assets.json` file.

Author: dellis1972

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/DesignerTests.cs

@@ -90,7 +90,8 @@ public CustomTextView(Context context, IAttributeSet attributes) : base(context,
 			using (var libb = CreateDllBuilder (Path.Combine (path, lib.ProjectName), false, false))
 			using (var appb = CreateApkBuilder (Path.Combine (path, proj.ProjectName), false, false)) {
 				// Save the library project, but don't build it yet
-				libb.Save (lib);
+				libb.Save (lib, saveProject: true);
+				FileAssert.Exists (Path.Combine(Root, path, lib.ProjectName, lib.ProjectFilePath));
 				appb.BuildLogFile = "build1.log";
 				appb.Target = target;
 				Assert.IsTrue (appb.Build (proj, parameters: DesignerParameters), $"build should have succeeded for target `{target}`");
@@ -104,10 +105,10 @@ public CustomTextView(Context context, IAttributeSet attributes) : base(context,
 				Assert.IsNotNull (doc.Element ("LinearLayout").Element ("unnamedproject.CustomTextView"),
 					"unnamedproject.CustomTextView should have not been replaced with a $(Hash).CustomTextView");
 				// Build the library project now
-				Assert.IsTrue (libb.Build (lib, doNotCleanupOnUpdate: true), "library build should have succeeded.");
+				Assert.IsTrue (libb.Build (lib, doNotCleanupOnUpdate: true, saveProject: true), "library build should have succeeded.");
 				appb.Target = "Build";
 				appb.BuildLogFile = "build2.log";
-				Assert.IsTrue (appb.Build (proj, doNotCleanupOnUpdate: true), "app build should have succeeded.");
+				Assert.IsTrue (appb.Build (proj, doNotCleanupOnUpdate: true, saveProject: false), "app build should have succeeded.");
 				Assert.IsTrue (appb.Output.AreTargetsAllBuilt ("_UpdateAndroidResgen"), "_UpdateAndroidResgen should have run completely.");
 				Assert.IsTrue (appb.Output.AreTargetsAllBuilt ("_Foo"), "_Foo should have run completely");
 				doc = XDocument.Load (customViewPath);
@@ -117,7 +118,16 @@ public CustomTextView(Context context, IAttributeSet attributes) : base(context,
 					"unnamedproject.CustomTextView should have been replaced with a $(Hash).CustomTextView");
 				appb.Target = target;
 				appb.BuildLogFile = "build3.log";
-				Assert.IsTrue (appb.Build (proj, parameters: DesignerParameters, doNotCleanupOnUpdate: true), $"build should have succeeded for target `{target}`");
+				Assert.IsTrue (appb.DesignTimeBuild (proj, parameters: DesignerParameters, doNotCleanupOnUpdate: true), $"build should have succeeded for target `{target}`");
+				Assert.IsFalse (appb.Output.AreTargetsAllSkipped ("_UpdateAndroidResgen"), "_UpdateAndroidResgen should not have been skipped.");
+				Assert.IsTrue (appb.Output.AreTargetsAllBuilt ("_Foo"), "_Foo should have run completely");
+				doc = XDocument.Load (customViewPath);
+				Assert.IsNull (doc.Element ("LinearLayout").Element ("UnnamedProject.CustomTextView"),
+					"UnnamedProject.CustomTextView should have been replaced with a $(Hash).CustomTextView");
+				Assert.IsNull (doc.Element ("LinearLayout").Element ("unnamedproject.CustomTextView"),
+					"unnamedproject.CustomTextView should have been replaced with a $(Hash).CustomTextView");
+				appb.BuildLogFile = "build4.log";
+				Assert.IsTrue (appb.DesignTimeBuild (proj, parameters: DesignerParameters, doNotCleanupOnUpdate: true), $"build should have succeeded for target `{target}`");
 				Assert.IsTrue (appb.Output.AreTargetsAllSkipped ("_UpdateAndroidResgen"), "_UpdateAndroidResgen should have been skipped.");
 				Assert.IsTrue (appb.Output.AreTargetsAllBuilt ("_Foo"), "_Foo should have run completely");
 				doc = XDocument.Load (customViewPath);

src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs

@@ -105,6 +105,11 @@ public void CheckNothingIsDeletedByIncrementalClean ([Values (true, false)] bool
 					File.SetLastWriteTimeUtc (file, DateTime.UtcNow);
 				}
 				Assert.IsTrue (b.Build (proj, doNotCleanupOnUpdate: true, saveProject: false), "Second should have succeeded");
+				b.Output.AssertTargetIsNotSkipped ("_CleanMonoAndroidIntermediateDir");
+				var stampFiles = Path.Combine (intermediate, "stamp", "_ResolveLibraryProjectImports.stamp");
+				FileAssert.Exists (stampFiles, $"{stampFiles} should exists!");
+				var libraryProjectImports = Path.Combine (intermediate, "libraryprojectimports.cache");
+				FileAssert.Exists (libraryProjectImports, $"{libraryProjectImports} should exists!");
 
 				//No changes
 				Assert.IsTrue (b.Build (proj, doNotCleanupOnUpdate: true, saveProject: false), "Third should have succeeded");
@@ -827,6 +832,7 @@ public void ResolveLibraryProjectImports ()
 
 				b.BuildLogFile = "build2.log";
 				Assert.IsTrue (b.Build (proj), "second build should have succeeded.");
+				b.Output.AssertTargetIsNotSkipped ("_ResolveLibraryProjectImports");
 				FileAssert.Exists (cacheFile);
 				var actual = ReadCache (cacheFile);
 				CollectionAssert.AreEqual (actual.Jars.Select (j => j.ItemSpec).OrderBy (j => j),
@@ -867,6 +873,22 @@ public void ResolveLibraryProjectImports ()
 				foreach (var targetName in targets) {
 					Assert.IsTrue (b.Output.IsTargetSkipped (targetName), $"`{targetName}` should be skipped!");
 				}
+
+				var filesToTouch = new [] {
+ 					Path.Combine (intermediate, "build.props"),
+ 					Path.Combine (intermediate, $"{proj.ProjectName}.pdb"),
+ 				};
+				foreach (var file in filesToTouch) {
+					FileAssert.Exists (file);
+					File.SetLastWriteTimeUtc (file, DateTime.UtcNow);
+				}
+
+				b.BuildLogFile = "build5.log";
+				Assert.IsTrue (b.Build (proj), "fifth build should have succeeded.");
+				b.Output.AssertTargetIsNotSkipped ("_CleanMonoAndroidIntermediateDir");
+				b.Output.AssertTargetIsNotSkipped ("_ResolveLibraryProjectImports");
+				FileAssert.Exists (cacheFile);
+				FileAssert.Exists (stamp);
 			}
 		}
 
@@ -1155,13 +1177,16 @@ public void DesignTimeBuild ()
 			var proj = new XamarinAndroidApplicationProject ();
 			using (var b = CreateApkBuilder (Path.Combine ("temp", $"{nameof (IncrementalBuildTest)}{TestName}"))) {
 				Assert.IsTrue (b.DesignTimeBuild (proj), "first dtb should have succeeded.");
+				var target = "_ResolveLibraryProjectImports";
+				Assert.IsFalse (b.Output.IsTargetSkipped (target), $"`{target}` should not have been skipped.");
 				// DesignTimeBuild=true lowercased
 				var parameters = new [] { "DesignTimeBuild=true" };
 				Assert.IsTrue (b.RunTarget (proj, "Compile", doNotCleanupOnUpdate: true, parameters: parameters), "second dtb should have succeeded.");
-				var target = "_ResolveLibraryProjectImports";
 				Assert.IsTrue (b.Output.IsTargetSkipped (target), $"`{target}` should have been skipped.");
 				Assert.IsTrue (b.RunTarget (proj, "UpdateGeneratedFiles", doNotCleanupOnUpdate: true, parameters: parameters), "UpdateGeneratedFiles should have succeeded.");
 				Assert.IsTrue (b.Output.IsTargetSkipped (target), $"`{target}` should have been skipped.");
+				Assert.IsTrue (b.Build (proj, doNotCleanupOnUpdate: true, saveProject: false), "full build should have succeeded.");
+				Assert.IsFalse (b.Output.IsTargetSkipped (target), $"`{target}` should not have been skipped.");
 			}
 		}
 

src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets

@@ -910,13 +910,18 @@ because xbuild doesn't support framework reference assemblies.
 		<AndroidLinkResources Condition="'$(AndroidLinkResources)' == '' And '$(AndroidIncludeDebugSymbols)' != 'True'">False</AndroidLinkResources>
 		<_AndroidBuildPropertiesCacheExists Condition=" Exists('$(_AndroidBuildPropertiesCache)') ">True</_AndroidBuildPropertiesCacheExists>
 		<_NuGetAssetsFile      Condition=" Exists('$(ProjectAssetsFile)') ">$(ProjectAssetsFile)</_NuGetAssetsFile>
-		<_NuGetAssetsFile      Condition=" Exists('$(ProjectLockFile)') ">$(ProjectLockFile)</_NuGetAssetsFile>
+		<_NuGetAssetsFile      Condition=" '$(_NuGetAssetsFile)' == '' and Exists('$(ProjectLockFile)') ">$(ProjectLockFile)</_NuGetAssetsFile>
 		<_NuGetAssetsFile      Condition=" '$(_NuGetAssetsFile)' == '' and Exists('packages.config') ">packages.config</_NuGetAssetsFile>
-		<_NuGetAssetsTimestamp Condition=" '$(_NuGetAssetsFile)' != '' ">$([System.IO.File]::GetLastWriteTime('$(_NuGetAssetsFile)').Ticks)</_NuGetAssetsTimestamp>
 		<_TypeMapKind Condition=" '$(AndroidIncludeDebugSymbols)' != 'True' ">mvid</_TypeMapKind>
 		<_TypeMapKind Condition=" '$(AndroidIncludeDebugSymbols)' == 'True' And '$(_InstantRunEnabled)' == 'True' ">strings-files</_TypeMapKind>
 		<_TypeMapKind Condition=" '$(AndroidIncludeDebugSymbols)' == 'True' And '$(_InstantRunEnabled)' != 'True' ">strings-asm</_TypeMapKind>
 	</PropertyGroup>
+	<GetFileHash
+				Condition=" Exists('$(_NuGetAssetsFile)') "
+				Files="$(_NuGetAssetsFile)"
+		>
+			<Output TaskParameter="Items" ItemName="_NuGetAssetsFileHash" />
+	</GetFileHash>
 	<ItemGroup>
 		<!-- List of items we want to trigger a build if changed -->
 		<_PropertyCacheItems Include="AotAssemblies=$(AotAssemblies)" />
@@ -945,7 +950,7 @@ because xbuild doesn't support framework reference assemblies.
 		<_PropertyCacheItems Include="OS=$(OS)" />
 		<_PropertyCacheItems Include="AndroidIncludeDebugSymbols=$(AndroidIncludeDebugSymbols)" />
 		<_PropertyCacheItems Include="AndroidPackageNamingPolicy=$(AndroidPackageNamingPolicy)" />
-		<_PropertyCacheItems Include="_NuGetAssetsTimestamp=$(_NuGetAssetsTimestamp)" />
+		<_PropertyCacheItems Include="_NuGetAssetsFileHash=%(_NuGetAssetsFileHash.FileHash)" />
 		<_PropertyCacheItems Include="TypeMapKind=$(_TypeMapKind)" />
 		<_PropertyCacheItems Include="AndroidManifestPlaceholders=$(AndroidManifestPlaceholders)" />
 		<_PropertyCacheItems Include="ProjectFullPath=$(MSBuildProjectFullPath)" />

src/Xamarin.Android.Build.Tasks/Xamarin.Android.DesignTime.targets

@@ -13,45 +13,50 @@ This file is used by all project types, including binding projects.
 
   <PropertyGroup>
     <_AndroidIntermediateDesignTimeBuildDirectory>$(IntermediateOutputPath)designtime\</_AndroidIntermediateDesignTimeBuildDirectory>
+    <_AndroidIntermediateDesignTimeStampDirectory>$(_AndroidIntermediateDesignTimeBuildDirectory)stamp\</_AndroidIntermediateDesignTimeStampDirectory>
     <_AndroidDesignTimeBuildPropertiesCache>$(_AndroidIntermediateDesignTimeBuildDirectory)build.props</_AndroidDesignTimeBuildPropertiesCache>
     <_AndroidLibraryImportsDesignTimeCache>$(_AndroidIntermediateDesignTimeBuildDirectory)libraryimports.cache</_AndroidLibraryImportsDesignTimeCache>
     <_AndroidLibraryProjectImportsDesignTimeCache>$(_AndroidIntermediateDesignTimeBuildDirectory)libraryprojectimports.cache</_AndroidLibraryProjectImportsDesignTimeCache>
     <_AndroidManagedResourceDesignerFile>$(_AndroidIntermediateDesignTimeBuildDirectory)$(_AndroidResourceDesigner)</_AndroidManagedResourceDesignerFile>
   </PropertyGroup>
 
-  <Target Name="_CreateStampDirectory">
-    <MakeDir
-        Condition=" '$(_AndroidStampDirectory)' != '' And !Exists('$(_AndroidStampDirectory)') "
-        Directories="$(_AndroidStampDirectory)"
-    />
-  </Target>
-
   <Target Name="_SetupMSBuildAllProjects">
     <ItemGroup>
       <_AndroidMSBuildAllProjects Include="$(MSBuildAllProjects)" Exclude="$(MSBuildProjectFullPath).user" />
     </ItemGroup>
   </Target>
 
-  <Target Name="_SetupDesignTimeBuildForBuild" DependsOnTargets="_CreateStampDirectory">
+  <Target Name="_SetupDesignTimeBuildForBuild">
     <PropertyGroup>
       <DesignTimeBuild Condition=" '$(DesignTimeBuild)' == '' ">false</DesignTimeBuild>
     </PropertyGroup>
+    <MakeDir
+        Condition=" '$(_AndroidStampDirectory)' != '' And !Exists('$(_AndroidStampDirectory)') "
+        Directories="$(_AndroidStampDirectory)"
+    />
   </Target>
 
-  <Target Name="_SetupDesignTimeBuildForCompile" DependsOnTargets="_CreateStampDirectory">
+  <Target Name="_SetupDesignTimeBuildForCompile">
     <PropertyGroup>
       <DesignTimeBuild Condition=" '$(DesignTimeBuild)' == '' ">true</DesignTimeBuild>
       <ManagedDesignTimeBuild Condition=" '$(AndroidGenerateResourceDesigner)' == 'True' And '$(AndroidUseManagedDesignTimeResourceGenerator)' == 'true' And '$(DesignTimeBuild)' == 'true' And '$(BuildingInsideVisualStudio)' == 'true' ">true</ManagedDesignTimeBuild>
       <ManagedDesignTimeBuild Condition=" '$(ManagedDesignTimeBuild)' == '' And '$(DesignTimeBuild)' == 'True' And '$(AndroidUseDesignerAssembly)' == 'True' " >True</ManagedDesignTimeBuild>
       <ManagedDesignTimeBuild Condition=" '$(ManagedDesignTimeBuild)' == '' ">False</ManagedDesignTimeBuild>
+      <_AndroidStampDirectory Condition=" '$(DesignTimeBuild)' == 'true' ">$(_AndroidIntermediateDesignTimeStampDirectory)</_AndroidStampDirectory>
       <_AndroidLibraryImportsCache Condition=" '$(DesignTimeBuild)' == 'true' And !Exists ('$(_AndroidLibraryImportsCache)') ">$(_AndroidLibraryImportsDesignTimeCache)</_AndroidLibraryImportsCache>
       <_AndroidLibraryProjectImportsCache Condition=" '$(DesignTimeBuild)' == 'true' And !Exists ('$(_AndroidLibraryProjectImportsCache)') ">$(_AndroidLibraryProjectImportsDesignTimeCache)</_AndroidLibraryProjectImportsCache>
       <_AndroidBuildPropertiesCache Condition=" '$(DesignTimeBuild)' == 'true' ">$(_AndroidDesignTimeBuildPropertiesCache)</_AndroidBuildPropertiesCache>
+      <_GeneratorStampFile Condition=" '$(DesignTimeBuild)' == 'true' And !Exists('$(_GeneratorStampFile)') ">$(_AndroidIntermediateDesignTimeStampDirectory)generator.stamp</_GeneratorStampFile>
+      <_AndroidResgenFlagFile Condition=" '$(DesignTimeBuild)' == 'true' And !Exists('$(_AndroidResgenFlagFile)') ">$(_AndroidIntermediateDesignTimeBuildDirectory)R.cs.flag</_AndroidResgenFlagFile>
     </PropertyGroup>
     <MakeDir
         Condition=" '$(_AndroidIsBindingProject)' != 'true' And !Exists ('$(_AndroidIntermediateDesignTimeBuildDirectory)') "
         Directories="$(_AndroidIntermediateDesignTimeBuildDirectory)"
     />
+    <MakeDir
+        Condition=" '$(_AndroidStampDirectory)' != '' And !Exists('$(_AndroidStampDirectory)') "
+        Directories="$(_AndroidStampDirectory)"
+    />
   </Target>
 
 </Project>