Use analyzers from targeting pack for NetCoreAppCurrent (dotnet#87726)

* Use analyzers from targeting pack for NetCoreAppCurrent

Fixes that analyzer failures didn't show-up in dotnet#74897

Add analyzers to the frameworklist that OOB projects in src/libraries
use, and only auto ProjectReference the analyzers in generators.targets
when not using the analyzers from the targeting pack.

Also move the generator projects related code into a separate file.

Continuation of dotnet#75093

* Fix project build

* Add missing reference to Regex tests

* Add missing generators for netfx build

* Fix paht in test project

* Fix typo

* Disable runtime marshalling for SharedTypes.csproj

* Disable runtime marshalling for NativeExports.csproj

Author: ViktorHofer

Directory.Build.targets

@@ -12,6 +12,7 @@
   <Import Project="$(RepositoryEngineeringDir)liveBuilds.targets" />
   <Import Project="$(RepositoryEngineeringDir)generators.targets" />
   <Import Project="$(RepositoryEngineeringDir)python.targets" />
+  <Import Project="$(RepositoryEngineeringDir)generatorProjects.targets" Condition="'$(IsGeneratorProject)' == 'true'" />
   <Import Project="$(RepositoryEngineeringDir)resolveContract.targets" Condition="'$(IsSourceProject)' == 'true'" />
   <Import Project="$(RepositoryEngineeringDir)packaging.targets" Condition="'$(IsPackable)' == 'true' and '$(MSBuildProjectExtension)' != '.pkgproj'" />
 
@@ -115,26 +116,6 @@
     </PropertyGroup>
   </Target>
 
-  <Target Name="GetAnalyzerPackFiles"
-          DependsOnTargets="$(GenerateNuspecDependsOn)"
-          Returns="@(_AnalyzerPackFile)">
-    <PropertyGroup>
-      <_analyzerPath>analyzers/dotnet</_analyzerPath>
-      <_analyzerPath Condition="'$(AnalyzerRoslynVersion)' != ''">$(_analyzerPath)/roslyn$(AnalyzerRoslynVersion)</_analyzerPath>
-      <_analyzerPath Condition="'$(AnalyzerLanguage)' != ''">$(_analyzerPath)/$(AnalyzerLanguage)</_analyzerPath>
-    </PropertyGroup>
-
-    <!-- Filter on netstandard2.0 so that generator projects can multi-target for the purpose of enabling nullable reference type compiler checks. -->
-    <ItemGroup>
-      <_AnalyzerPackFile Include="@(_BuildOutputInPackage->WithMetadataValue('TargetFramework', 'netstandard2.0'))" IsSymbol="false" />
-      <_AnalyzerPackFile Include="@(_TargetPathsToSymbols->WithMetadataValue('TargetFramework', 'netstandard2.0'))" IsSymbol="true" />
-      <_AnalyzerPackFile PackagePath="$(_analyzerPath)/%(TargetPath)" />
-    </ItemGroup>
-
-    <Error Text="Analyzers must target netstandard2.0 since they run in the compiler which targets netstandard2.0. $(MSBuildProjectFullPath) targets '$([MSBuild]::ValueOrDefault('$(TargetFrameworks)', '$(TargetFramework)'))' instead."
-           Condition="'@(_AnalyzerPackFile)' == ''" />
-  </Target>
-
   <!-- Allows building against source assemblies when the 'SkipUseReferenceAssembly' attribute is present on ProjectReference items. -->
   <Target Name="HandleReferenceAssemblyAttributeForProjectReferences"
           AfterTargets="ResolveProjectReferences"
@@ -197,6 +178,16 @@
                                                       Exclude="@(_targetingPackIncludedReferenceWithProjectName)" />
       <Reference Remove="@(_targetingPackExcludedReferenceWithProjectName->Metadata('OriginalIdentity'))" />
     </ItemGroup>
+
+    <ItemGroup>
+      <_targetingPackAnalyzerReferenceWithProjectName Include="@(Analyzer->WithMetadataValue('ExternallyResolved', 'true')->Metadata('Filename'))"
+                                                      OriginalIdentity="%(Identity)" />
+      <_targetingPackIncludedAnalyzerReferenceWithProjectName Include="@(_targetingPackAnalyzerReferenceWithProjectName)"
+                                                              Exclude="@(_targetingPackReferenceExclusion)" />
+      <_targetingPackExcludedAnalyzerReferenceWithProjectName Include="@(_targetingPackAnalyzerReferenceWithProjectName)"
+                                                              Exclude="@(_targetingPackIncludedAnalyzerReferenceWithProjectName)" />
+      <Analyzer Remove="@(_targetingPackExcludedAnalyzerReferenceWithProjectName->Metadata('OriginalIdentity'))" />
+    </ItemGroup>
   </Target>
 
   <!--

docs/coding-guidelines/project-guidelines.md

@@ -185,7 +185,7 @@ All test outputs should be under
 
 ## gen
 In the gen directory any source generator related to the assembly should exist. This does not mean the source generator is only used for that assembly only that it is conceptually apart of that assembly. For example, the assembly may provide attributes or low-level types the source generator uses.
-To consume a source generator, simply add a `<ProjectReference Include="..." ReferenceOutputAssembly="false" OutputItemType="Analyzer" />` item to the project, usually next to the `Reference` and `ProjectReference` items.
+To consume a source generator that isn't provided via a targeting pack, simply add a `<ProjectReference Include="..." ReferenceOutputAssembly="false" OutputItemType="Analyzer" />` item to the project, usually next to the `Reference` and `ProjectReference` items.
 
 A source generator must target `netstandard2.0` as such assemblies are loaded into the compiler's process which might run on either .NET Framework or modern .NET depending on the tooling being used (CLI vs Visual Studio). While that's true, a source project can still multi-target and include `$(NetCoreAppToolCurrent)` (which is the latest non live-built .NETCoreApp tfm that is supported by the SDK) to benefit from the ehancanced nullable reference type warnings emitted by the compiler. For an example see [System.Text.Json's roslyn4.4 source generator](/src/libraries/System.Text.Json/gen/System.Text.Json.SourceGeneration.Roslyn4.4.csproj). While the repository's infrastructure makes sure that only the source generator's `netstandard2.0` build output is included in packages, to consume such a multi-targeting source generator via a `ProjectReference` (as described above), you need to add the `SetTargetFramework="TargetFramework=netstandard2.0"` metadata to the ProjectReference item to guarantee that the netstandard2.0 asset is chosen.
 

eng/generatorProjects.targets

@@ -0,0 +1,34 @@
+<Project>
+
+  <PropertyGroup>
+    <GeneratorProjectBaseTargetPath>analyzers/dotnet</GeneratorProjectBaseTargetPath>
+    <GeneratorProjectBaseTargetPath Condition="'$(AnalyzerLanguage)' != ''">$(GeneratorProjectBaseTargetPath)/$(AnalyzerLanguage)</GeneratorProjectBaseTargetPath>
+  </PropertyGroup>
+
+  <ItemDefinitionGroup>
+    <TargetPathWithTargetPlatformMoniker>
+      <GeneratorProjectBaseTargetPath>$(GeneratorProjectBaseTargetPath)</GeneratorProjectBaseTargetPath>
+    </TargetPathWithTargetPlatformMoniker>
+  </ItemDefinitionGroup>
+
+  <Target Name="GetAnalyzerPackFiles"
+          DependsOnTargets="$(GenerateNuspecDependsOn)"
+          Returns="@(_AnalyzerPackFile)">
+    <PropertyGroup>
+      <_analyzerPath>$(GeneratorProjectBaseTargetPath)</_analyzerPath>
+      <_analyzerPath Condition="'$(AnalyzerRoslynVersion)' != ''">$(_analyzerPath)/roslyn$(AnalyzerRoslynVersion)</_analyzerPath>
+      <_analyzerPath Condition="'$(AnalyzerLanguage)' != ''">$(_analyzerPath)/$(AnalyzerLanguage)</_analyzerPath>
+    </PropertyGroup>
+
+    <!-- Filter on netstandard2.0 so that generator projects can multi-target for the purpose of enabling nullable reference type compiler checks. -->
+    <ItemGroup>
+      <_AnalyzerPackFile Include="@(_BuildOutputInPackage->WithMetadataValue('TargetFramework', 'netstandard2.0'))" IsSymbol="false" />
+      <_AnalyzerPackFile Include="@(_TargetPathsToSymbols->WithMetadataValue('TargetFramework', 'netstandard2.0'))" IsSymbol="true" />
+      <_AnalyzerPackFile PackagePath="$(_analyzerPath)/%(TargetPath)" />
+    </ItemGroup>
+
+    <Error Text="Analyzers must target netstandard2.0 since they run in the compiler which targets netstandard2.0. $(MSBuildProjectFullPath) targets '$([MSBuild]::ValueOrDefault('$(TargetFrameworks)', '$(TargetFramework)'))' instead."
+           Condition="'@(_AnalyzerPackFile)' == ''" />
+  </Target>
+
+</Project>

eng/generators.targets

@@ -1,4 +1,5 @@
 <Project>
+
   <PropertyGroup>
     <EnableLibraryImportGenerator Condition="'$(EnableLibraryImportGenerator)' == '' and
                                              '$(MSBuildProjectName)' == 'System.Private.CoreLib'">true</EnableLibraryImportGenerator>
@@ -13,7 +14,7 @@
     <!-- If the current project is not System.Private.CoreLib, we enable the LibraryImportGenerator source generator
          when the project is a C# source project that:
          - doesn't target the latest TFM or
-         - references everything from the targeting pack (OOB) or
+         - doesn't reference the targeting pack (i.e. when inbox) and
          - references System.Private.CoreLib, or
          - references System.Runtime.InteropServices -->
     <EnabledGenerators Include="LibraryImportGenerator"
@@ -22,28 +23,26 @@
                                   '$(MSBuildProjectExtension)' == '.csproj' and
                                   (
                                     '$(TargetFrameworkMoniker)' != '$(NetCoreAppCurrentTargetFrameworkMoniker)' or
-                                    '$(DisableImplicitFrameworkReferences)' != 'true' or
-                                    (
-                                      '@(Reference)' != '' and
-                                      @(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.InteropServices'))
-                                    ) or
                                     (
-                                      '@(ProjectReference)' != '' and
-                                      @(ProjectReference->AnyHaveMetadataValue('Identity', '$(CoreLibProject)'))
+                                      '$(DisableImplicitFrameworkReferences)' == 'true' and
+                                      (
+                                        '@(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.InteropServices'))' == 'true' or
+                                        '@(ProjectReference->AnyHaveMetadataValue('Identity', '$(CoreLibProject)'))' == 'true'
+                                      )
                                     )
                                   )" />
     <!-- We enable the ComInterfaceGenerator source generator
          when the project is a C# source project that:
-         - References everything from the targeting pack (OOB) or
+         - Doesn't reference the targeting pack (i.e. when inbox) and
          - references System.Runtime.InteropServices -->
     <EnabledGenerators Include="ComInterfaceGenerator"
                        Condition="'$(IsSourceProject)' == 'true' and
                                   '$(MSBuildProjectExtension)' == '.csproj' and
                                   (
-                                    '$(DisableImplicitFrameworkReferences)' != 'true' or
+                                    '$(TargetFrameworkMoniker)' != '$(NetCoreAppCurrentTargetFrameworkMoniker)' or
                                     (
-                                      '@(Reference)' != '' and
-                                      @(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.InteropServices'))
+                                      '$(DisableImplicitFrameworkReferences)' == 'true' and
+                                      '@(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.InteropServices'))' == 'true'
                                     )
                                   )" />
   </ItemGroup>
@@ -61,18 +60,14 @@
     <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices\gen\Microsoft.Interop.SourceGeneration\Microsoft.Interop.SourceGeneration.csproj"
                       ReferenceOutputAssembly="false"
                       OutputItemType="Analyzer" />
-  </ItemGroup>
-  <ItemGroup Condition="'@(EnabledGenerators)' != '' and
-                        @(EnabledGenerators->AnyHaveMetadataValue('Identity', 'LibraryImportGenerator'))">
     <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices\gen\LibraryImportGenerator\LibraryImportGenerator.csproj"
                       ReferenceOutputAssembly="false"
-                      OutputItemType="Analyzer" />
-  </ItemGroup>
-  <ItemGroup Condition="'@(EnabledGenerators)' != '' and
-                        @(EnabledGenerators->AnyHaveMetadataValue('Identity', 'ComInterfaceGenerator'))">
+                      OutputItemType="Analyzer"
+                      Condition="@(EnabledGenerators->AnyHaveMetadataValue('Identity', 'LibraryImportGenerator'))" />
     <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices\gen\ComInterfaceGenerator\ComInterfaceGenerator.csproj"
                       ReferenceOutputAssembly="false"
-                      OutputItemType="Analyzer" />
+                      OutputItemType="Analyzer"
+                      Condition="@(EnabledGenerators->AnyHaveMetadataValue('Identity', 'ComInterfaceGenerator'))" />
   </ItemGroup>
 
   <Target Name="ConfigureGenerators"
@@ -87,10 +82,8 @@
     <PropertyGroup>
         <LibraryImportGenerator_UseMarshalType>true</LibraryImportGenerator_UseMarshalType>
     </PropertyGroup>
-
-    <ItemGroup Condition="'$(NetCoreAppCurrentTargetFrameworkMoniker)' == '$(TargetFrameworkMoniker)' and '$(IncludeLibraryImportGeneratorSources)' != 'false'">
-    </ItemGroup>
   </Target>
 
   <Import Project="$(LibrariesProjectRoot)System.Runtime.InteropServices\gen\LibraryImportGenerator\Microsoft.Interop.LibraryImportGenerator.props" />
+
 </Project>

eng/targetingpacks.targets

@@ -129,12 +129,14 @@
   <Target Name="UpdateLocalTargetingAndRuntimePack"
           Condition="'$(UseLocalTargetingRuntimePack)' == 'true'"
           AfterTargets="ResolveFrameworkReferences">
+    <Error Text="The shared framework must be built before the local targeting pack can be consumed."
+           Condition="!Exists('$(MicrosoftNetCoreAppRefPackDir)data\FrameworkList.xml')" />
+    
     <ItemGroup>
       <ResolvedTargetingPack Path="$(MicrosoftNetCoreAppRefPackDir.TrimEnd('/\'))"
                              NuGetPackageVersion="$(ProductVersion)"
                              PackageDirectory="$(MicrosoftNetCoreAppRefPackDir.TrimEnd('/\'))"
-                             Condition="'%(ResolvedTargetingPack.RuntimeFrameworkName)' == '$(LocalFrameworkOverrideName)' and
-                                        Exists('$(MicrosoftNetCoreAppRefPackDir)data\FrameworkList.xml')" />
+                             Condition="'%(ResolvedTargetingPack.RuntimeFrameworkName)' == '$(LocalFrameworkOverrideName)'" />
       <ResolvedRuntimePack PackageDirectory="$(MicrosoftNetCoreAppRuntimePackDir)"
                            Condition="'$(MicrosoftNetCoreAppRuntimePackDir)' != '' and
                                       '%(ResolvedRuntimePack.FrameworkName)' == '$(LocalFrameworkOverrideName)'" />

src/libraries/Directory.Build.targets

@@ -90,7 +90,7 @@
     </BinPlaceTargetFrameworks>
 
     <!-- Source generator projects might multi-target. Make sure that only the netstandard2.0 compiled assets get binplaced. -->
-    <BinPlaceDir Include="$(MicrosoftNetCoreAppRefPackDir)analyzers\dotnet\$(AnalyzerLanguage)"
+    <BinPlaceDir Include="$(MicrosoftNetCoreAppRefPackDir)$(GeneratorProjectBaseTargetPath)"
                  Condition="'$(IsNETCoreAppAnalyzer)' == 'true' and
                             '$(TargetFramework)' == 'netstandard2.0'" />
 

src/libraries/Microsoft.NET.WebAssembly.Webcil/src/Microsoft.NET.WebAssembly.Webcil.csproj

@@ -6,6 +6,8 @@
     <Serviceable>true</Serviceable>
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <CLSCompliant>false</CLSCompliant>
+    <!-- This library can't use the live targeting / runtime pack as it builds before the shared framework (as part of a task). -->
+    <UseLocalTargetingRuntimePack>false</UseLocalTargetingRuntimePack>
   </PropertyGroup>
 
   <ItemGroup>

src/libraries/System.Data.Odbc/src/System.Data.Odbc.csproj

@@ -138,13 +138,6 @@ System.Data.Odbc.OdbcTransaction</PackageDescription>
              Link="Common\System\Runtime\InteropServices\HandleRefMarshaller.cs" />
   </ItemGroup>
 
-  <ItemGroup Condition="$([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net8.0'))">
-    <ProjectReference Include="..\..\System.Text.RegularExpressions\gen\System.Text.RegularExpressions.Generator.csproj"
-                  SetTargetFramework="TargetFramework=netstandard2.0"
-                  OutputItemType="Analyzer"
-                  ReferenceOutputAssembly="false" />
-  </ItemGroup>
-
   <ItemGroup Condition="'$(TargetPlatformIdentifier)' == 'linux' or '$(TargetPlatformIdentifier)' == 'freebsd' or '$(TargetPlatformIdentifier)' == 'illumos' or '$(TargetPlatformIdentifier)' == 'solaris'">
     <Compile Include="$(CommonPath)Interop\Linux\Interop.Libraries.cs"
              Link="Common\Interop\Linux\Interop.Libraries.cs" />

src/libraries/System.Data.OleDb/src/System.Data.OleDb.csproj

@@ -153,10 +153,4 @@ System.Data.OleDb.OleDbTransaction</PackageDescription>
     <ProjectReference Include="$(LibrariesProjectRoot)System.Diagnostics.PerformanceCounter\src\System.Diagnostics.PerformanceCounter.csproj" />
   </ItemGroup>
 
-  <ItemGroup Condition="$([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net8.0'))">
-    <ProjectReference Include="..\..\System.Text.RegularExpressions\gen\System.Text.RegularExpressions.Generator.csproj"
-                  SetTargetFramework="TargetFramework=netstandard2.0"
-                  OutputItemType="Analyzer"
-                  ReferenceOutputAssembly="false" />
-  </ItemGroup>
 </Project>

src/libraries/System.Memory.Data/tests/System.Memory.Data.Tests.csproj

@@ -12,7 +12,7 @@
     <ProjectReference Include="..\src\System.Memory.Data.csproj"/>
   </ItemGroup>
 
-  <ItemGroup>
+  <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
     <ProjectReference Include="..\..\System.Text.Json\gen\System.Text.Json.SourceGeneration.Roslyn4.0.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
   </ItemGroup>
 

src/libraries/System.Net.Http.Json/tests/FunctionalTests/System.Net.Http.Json.Functional.Tests.csproj

@@ -1,16 +1,20 @@
 <Project Sdk="Microsoft.NET.Sdk">
+
   <PropertyGroup>
     <TargetFrameworks>$(NetCoreAppCurrent);$(NetFrameworkCurrent)</TargetFrameworks>
   </PropertyGroup>
+
   <ItemGroup>
     <Compile Include="HttpClientJsonExtensionsTests.cs" />
     <Compile Include="HttpContentJsonExtensionsTests.cs" />
     <Compile Include="JsonContentTests.cs" />
     <Compile Include="TestClasses.cs" />
   </ItemGroup>
+
   <ItemGroup Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)'">
     <Compile Include="JsonContentTests.netcoreapp.cs" />
   </ItemGroup>
+
   <ItemGroup>
     <Compile Include="$(CommonTestPath)System\Net\Capability.Security.cs" Link="Common\System\Net\Capability.Security.cs" />
     <Compile Include="$(CommonTestPath)System\Net\Configuration.Certificates.cs" Link="Common\System\Net\Configuration.Certificates.cs" />
@@ -23,11 +27,14 @@
     <Compile Include="$(CommonTestPath)System\Security\Cryptography\PlatformSupport.cs" Link="CommonTest\System\Security\Cryptography\PlatformSupport.cs" />
     <Compile Include="$(CommonTestPath)System\Threading\Tasks\TaskTimeoutExtensions.cs" Link="Common\System\Threading\Tasks\TaskTimeoutExtensions.cs" />
   </ItemGroup>
+
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
     <Reference Include="System.Net.Http" />
     <ProjectReference Include="..\..\src\System.Net.Http.Json.csproj" />
   </ItemGroup>
-  <ItemGroup>
+
+  <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
     <ProjectReference Include="$(LibrariesProjectRoot)System.Text.Json\gen\System.Text.Json.SourceGeneration.Roslyn4.0.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
   </ItemGroup>
+
 </Project>

src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.Legacy.UnitTests/System.Runtime.InteropServices.JavaScript.Legacy.Tests.csproj

@@ -32,7 +32,5 @@
     <WasmExtraFilesToDeploy Include="timers.mjs" />
     <None Include="timers.mjs" />
     <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices.JavaScript\src\System.Runtime.InteropServices.JavaScript.csproj" SkipUseReferenceAssembly="true"/>
-    <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices.JavaScript\gen\JSImportGenerator\JSImportGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
-    <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices\gen\Microsoft.Interop.SourceGeneration\Microsoft.Interop.SourceGeneration.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
   </ItemGroup>
 </Project>

src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System.Runtime.InteropServices.JavaScript.Tests.csproj

@@ -22,9 +22,4 @@
     <WasmExtraFilesToDeploy Include="System\Runtime\InteropServices\JavaScript\JavaScriptTestHelper.mjs" />
     <WasmExtraFilesToDeploy Include="System\Runtime\InteropServices\JavaScript\SecondRuntimeTest.js" />
   </ItemGroup>
-  <ItemGroup>
-    <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices.JavaScript\gen\JSImportGenerator\JSImportGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
-    <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices\gen\Microsoft.Interop.SourceGeneration\Microsoft.Interop.SourceGeneration.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
-    <ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices.JavaScript\src\System.Runtime.InteropServices.JavaScript.csproj" />
-  </ItemGroup>
 </Project>