Move all of the ref-pack generators to use the "LatestVS" Roslyn build so they don't pull in prebuilts in source-build

[jkoritzinsky]

Related to #81468

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Related to #81468

Author:	jkoritzinsky
Assignees:	-
Labels:	source-build, area-Infrastructure, dev-innerloop
Milestone:	-

[pavelsavara]

We have references like below in the code base.

<ProjectReference 
    Include="$(LibrariesProjectRoot)System.Text.Json\gen\System.Text.Json.SourceGeneration.Roslyn4.0.csproj" 
    OutputItemType="Analyzer" 
    ReferenceOutputAssembly="false" />

Is it somehow relevant for this ?

[jkoritzinsky]

No, those shouldn't matter for this case as far as I know. Someone from source-build can confirm as they know where the prebuilt detection tooling is and how to use it.

[ViktorHofer]

Given that we had this explicit remark section here, are we changing plans now? I understand the morivtion behind this change but will this effectively force our nighlty build consumers and/or our repo contributors to use an int Preview build of VS (which they likely not have access to)?

cc @stephentoub for Regex

[jkoritzinsky]

No, the LatestVS version tracks the most recent public preview that we need to use APIs from.

I removed the remark as just using the LatestVS version implies the exact same plan (it's the whole reason we introduced it in our infrastructure).

[tmds]

This breaks a /p:DotNetBuildFromSource=true build with:

CSC : error CS1705: Assembly 'System.Text.RegularExpressions.Generator' with identity 'System.Text.RegularExpressions.Generator, Version=42.42.42.42, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' uses 'Microsoft.CodeAnalysis, Version=4.6.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' which has a higher version than referenced assembly 'Microsoft.CodeAnalysis' with identity 'Microsoft.CodeAnalysis, Version=4.4.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' [/home/tmds/repos/runtime/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/System.Text.RegularExpressions.Tests.csproj::TargetFramework=net8.0]

It has to do with this condition:

runtime/eng/Versions.props

Line 71 in 725de9e

<MicrosoftCodeAnalysisVersion_LatestVS Condition="'$(DotNetBuildFromSource)' == 'true'">$(MicrosoftCodeAnalysisVersion)</MicrosoftCodeAnalysisVersion_LatestVS>

@ViktorHofer @jkoritzinsky should the line be removed?

[jkoritzinsky]

@tmds that line was based on when we used to build Roslyn before runtime in source-build. We kept hitting issues where we’d only have issues with the newest Roslyn reported when changes propagated to dotnet/installer. If there’s a better value to use there for source-build to ensure coherency and accurate testing, we can change it. That line should do whatever is needed to get source-build working correctly.

[tmds]

Before this change, the projects were using MicrosoftCodeAnalysisVersion_4_4 on source-build, and removing that line will make them use that version again.

Removing that line fixes my build. I don't know if that means it is the proper fix.

[tmds]

Some other options could be to

Change the line to:

 <MicrosoftCodeAnalysisVersion_LatestVS Condition="'$(DotNetBuildFromSource)' == 'true'">$(MicrosoftCodeAnalysisVersion_4_4)</MicrosoftCodeAnalysisVersion_LatestVS>

Or change the generator projects so they are happy to use the 4.6 version when "'$(DotNetBuildFromSource)' == 'true'".

@ViktorHofer do you know what the proper change is?

[jkoritzinsky]

We need to be able to adopt new Roslyn versions to use new APIs that they provide. Being able to use 4.6 would be much better than 4.4. Personally, I think source-build shouldn't have moved Roslyn to build after Runtime and it should build before as we started to take a dependency on that experience.

[tmds]

Since your didn't intend to change the version used by source build, we should pin it back to 4.4.

It would be interesting to have the discussion with the source build maintainers about the build order. I don't know the rationale for the build order, though I'd like to.

[jkoritzinsky]

I did intend to change source-build, but not in a way that would break it. If just removing this condition would still work for source-build, then we should do that. Also, we should update our source-build leg to accurately represent source-build so we don't have this issue again in the future.

[tmds]

I'm looking into this once again, and now I noticed the error happens when compiling tests.
Tests aren't built as part of source-build, but I'm building them locally while setting /p:DotNetBuildFromSource=true.

I'll make a PR with a change for the tests, and maybe we can include/change a CI job so it sets DotNetBuildFromSource.