Many repositories inside the VMR use Arcade's filtering mechanism to filter out (TFMs and) projects via the ExcludeFrom... properties. The problem with that is that it requires to evaluate the to be filtered out project. This leads to several issues:

1. Poor performance as at least one evaluation is kick-off per to be filtered out project.
2. Doesn't support static graph builds and only supports static graph restore with hacks.
3. More verbose logs and binlogs as the excluded projects are part of the build graph.
4. Hacks. Due to the evaluation of excluded projects, referenced msbuild sdks are restored and imported. This is the reason why the VMR has this big hack: https://github.com/dotnet/dotnet/tree/main/eng/tools/EmptySdk & https://github.com/dotnet/dotnet/blob/a3488dad389cab8b740939b9d2e9f49ff7dd4b43/repo-projects/Directory.Build.props#L177. If the excluded project wouldn't be evaluated, that msbuild sdk resolver would never see a request to download and import that specific msbuild sdk.

Alternatives:

1. Solution filter files (slnf). Some repositories use solution filter files to only build a subset of their repository. There are a couple of downsides with that approach:
   • Static. Solution filter files specify the projects that shouldn't filtered out statically. Meaning, you need one file per filtering rule. For the VMR build this means that repositories need to define slnf files for the source-only build, when not building source-only but excluding tests, when not building source-build without excluding tests, etc. This becomes unmaintainable relatively quickly.
   • There's currently a defect in static graph restore that causes filtered-out projects to still be restored (and therefore, evaluated).
2. Microsoft.Build.Traversal. This is an msbuild sdk and is heavily used by big repositories like runtime which want to have full control over their build graph based on different inputs. There's really only one downside to this approach: You can't open a project that uses that msbuild sdk directly via tools like Visual Studio or Visual Studio Code. Tools like slngen can on-the-fly generate a solution file from a Traversal project, but that requires to be inside a command prompt. You can of course check the solution file that is generated via slngen in but then you need to make sure that the traversal project (i.e. Build.proj) stays in sync with the generated sln file.
3. Combination of solution file and Microsoft.Build.Traversal. See my POC that enables that in dotnet/xdt: dotnet/xdt@93902be. MSBuild generates a .metaproj from a given solution file which can be mutated by adding a Directory.Solution.props/targets file with custom msbuild logic. This would be the best option for small to medium sized repositories which want to continue using a solution file but with a more dynamic filtering mechanism than what slnfs offer. Unfortunately there are couple of downsides when using that filtering mechanism today:
   • Static graph restore still restores the projects: Microsoft.Build.Traversal dynamically skipped projects in solution files still get restored with static graph restore microsoft/MSBuildSdks#521
   • By default, evaluations still happen for excluded projects and a target is invoked: [Traversal] Filtering projects solution files without kicking off evaluations microsoft/MSBuildSdks#522

I've been dealing with this topic for years now as we needed a solid way to filter projects in XXXL repositories like dotnet/runtime. It's clear to me that neither the ExcludeFrom... Arcade infrastructure, nor slnfs satisfy our needs. We need a highly performant filtering mechanism (=> no evaluations) with the flexibility to change the set of projects based on different inputs (i.e. DotNetBuildSourceOnly vs DotNetBuild vs DotNetBuildTests).

Repositories like runtime which already use option 2 don't need to change anything. They already declare the set of projects to be built based on various inputs. Ones that use slnfs or Arcade's infra would highly benefit from option 3. Therefore I propose that we iterate on option 3 to make it work without kicking-off evaluations (might need a small product change in msbuild's metaproj generation) and then convert the existing repositories to depend on that.

cc @mmitche @jeffkl @ericstj @MichaelSimons