NuGet Static graph restore invoked on a slnf file restores excluded projects

[ViktorHofer]

NuGet Product Used

dotnet.exe

Product Version

8.0.100

Worked before?

no

Impact

I'm unable to use this version

Repro Steps & Context

Repro:
dotnet restore ttt.slnf -> only the projects listed in the slnf file are restored
dotnet restore ttt.slnf /p:RestoreUseStaticGraphEvaluation=true -> all projects listed in the sln file are restored

restorediff.zip

This impacts changes like dotnet/msbuild#9555 as when building from source, we must not restore more projects than necessary to not bring in prebuilts.

cc @jeffkl

Verbose Logs

No response

[nkolev92]

Team Triage: @jeffkl what would be the cost of fixing this?

[jeffkl]

Unfortunately, MSBuild does not expose the path to the solution filter during the build (dotnet/msbuild#6162). The way it currently works for regular restore is that MSBuild generates an in-memory .metaproj that contains a <ProjectReference /> to just the projects to build. Regular restore then walks those projects.

For static graph-based restore, MSBuild only passes along the path to the solution file and the path to the in-memory .metaproj:

For regular restore, since its running in the same process it has access to the in-memory .metaproj. Since static graph-based restore runs out-of-proc, it would need to either write the .metaproj to disk or pass along the path to the solution filter instead. It could also use reflection to get the path to the solution filter:

object buildRequest = BuildEngine.GetType().GetField("_requestEntry", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(BuildEngine);
object buildRequestConfiguration = buildRequest.GetType().GetProperty("RequestConfiguration", BindingFlags.Public | BindingFlags.Instance).GetValue(buildRequest);
string solutionFilterFileFullPath = buildRequestConfiguration.GetType().GetProperty("ProjectFullPath", BindingFlags.Public | BindingFlags.Instance).GetValue(buildRequestConfiguration) as string;

So we need to either:

1. Update MSBuild to indicate the path to the solution filter being restored
2. Use reflection to get the path to the solution filter being restored

[nkolev92]

We use reflection in nuget.exe, but the bar is arguably lower there: https://github.com/NuGet/NuGet.Client/blob/d7607e2f08721498801ace7fe47933d4e6dfeffd/src/NuGet.Clients/NuGet.CommandLine/Common/Solution.cs#L120.

[zivkan]

Can static graph restore pass the list of ProjectReferences from the metaproj to the restore task?

It's not perfect, because:

• sln based restore will pass the full list of projects, and then waste CPU time filtering the list to what ends up being the full list of projects
• slnf based restore will have MSBuild evaluate the full solution, but then NuGet will ignore all the projects not part of the filter

but at least there will be fewer projects that need to resolve the package graph, hopefully fewer packages to download, etc.

[jeffkl]

Can static graph restore pass the list of ProjectReferences from the metaproj to the restore task?

Yes, its technically possible to pass all of the necessary information along. I'm hesitant to implement it that way because we already have issues streaming global properties to the other process. I'd feel a lot more comfortable just passing the full path to the SLNF to the process instead.

[zivkan]

I agree getting the slnf path is the ideal solution.

I'm just a bit worried about using reflection to get a non-public property. If MSBuild makes a change that breaks static restore, it might take us a while to notice. After all, our dev branch will be inserted into the 8.0.3xx SDK, but our build scripts are still testing against a pre-release 8.0.1xx SDK build. Therefore, our own CI pipeline wouldn't even detect the break.

Sending the long list of projects feels like a lower risk temporary solution, while we wait for MSBuild to make the request, and slnf path, a public API.