Preserve intermediates when building the VMR with "--clean-while-building" mode (CI) that are inputs to compliance tools

[ViktorHofer]

The VMR deletes the entire "artifacts" inner repo folder when building with --clean-while-building which is used in CI to reduce disk space. That means that compliance tools like CG don't have access to the intermediates that it looks at.

CG looks at a number of different intermediate files, i.e. nuget.config, projects.assets.json, ...
https://github.com/search?q=repo%3Amicrosoft%2Fcomponent-detection%20%2FIList%3Cstring%3E%20SearchPatterns%2F&type=code

We should define a list of intermediates that we want to preserve in all builds modes (source-only and non-source-only) and update this target: https://github.com/dotnet/sdk/blob/5c5a92beffab32631feb44515188c298b53598d2/src/SourceBuild/content/repo-projects/Directory.Build.targets#L654-L663

[ericstj]

Another option if we need to reclaim machine space would be to run those compliance tools in the middle of the build before you clean things up. Have we ever considered breaking the VMR down this way?

[mmitche]

Another option if we need to reclaim machine space would be to run those compliance tools in the middle of the build before you clean things up. Have we ever considered breaking the VMR down this way?

I'd rather not go that direction if possible. The trend in 1ES's tooling is to integrate it more tightly with AzDO rather than as standalone tools. This way teams get changes, updates, new tooling, etc. for free. The tools can still be run standalone, usually, but there is a lot of logic integrated into the templates. This would mean that either we need to drop out of the build to inject 1ESPT tasks after every repo build, which would essentially kill any parallelism, or we would need to keep pace with the changes in 1ES tooling. I would prefer to avoid both options.

[mmitche]

I don't know if it is feasible, but it would be better if we could eventually get rid of all cleaning while building. On VMs and docker containers this should be entirely workable. It's about the disk space available in our base VMs. On Mac, this is potentially tougher, and maybe we need to be selective there. Overall, cleaning while building feels a bit like a hack

[MichaelSimons]

I don't know if it is feasible, but it would be better if we could eventually get rid of all cleaning while building

Just a word of caution. Even if we are able to stop using clean while building, we should preserve the option. This was originally added because of machine size limitations our distro partners were hitting in their build environments.