Skip to content

Conversation

@dfederm
Copy link
Member

@dfederm dfederm commented Feb 7, 2025

When using the MSBuildCacheGetResultsForUnqueriedDependencies feature, today the unqueried dependencies' outputs are materialized on disk, assuming they're cache hits. This is problematic because the caller did not explicitly ask for those nodes and thus may be assuming that those outputs are not changing in any way.

This change avoids materializing outputs for unqueried dependencies.

There is special-casing for outer builds of multitargeting projects. Outer builds depend on the inner builds, so if only the outer build were materialized, almost nothing would end up materializing. So we do still materialize dependencies if the project is an outer build and the dependency is an inner build.

Note that there was some debate around this (I was the hold-out), so capturing the gist of the discussion here to help anyone looking back on this for historical context.

  • What if the dependency's outputs are out-of-date or worse missing entirely? This scenario is quite unlikely and would suggest a bug in the caller if this was a real problem. The caller may have determined the current state of the dependency is "good enough" and does not demand exact correctness. Comparing with other systems, in Visual Studio if there is a project which is marked to not build or not in the sln at all, VS will happily avoid it and build projects which depend on it. In all likelihood, those projects will fail, but in the exceedingly strange case that they don't (eg the don't actually depend on their dependency and perhaps set ReferenceOutputAssembly=false or something), then VS would also leave the disk in a state where the dependency is left unbuilt. So at worst, this matches VS's behavior so there's some consistency within the ecosystem.
  • What's problematic about materializing the files? Well in the case of projects A and B both depending on D, and D is the unqueried dependency, and A and B have no relationship, then there is a race condition where the processing of A might cause D to materialize, and concurrent processing of B might see inconsistent state for D and make irrational decisions based on that.

@dfederm
Copy link
Member Author

dfederm commented Feb 7, 2025

Ugh, this breaks the scenario of multitargeting if the outer build is queried and the inner builds haven't been. It ends up being a cache hit without actually placing any files.

May need to special-case the node for the outer build of a multitargetting project...

@dfederm dfederm force-pushed the avoid-materializing-recursively branch from 5d031cc to 6ee2d38 Compare February 10, 2025 22:48
@dfederm dfederm enabled auto-merge (squash) February 10, 2025 23:08
@dfederm dfederm merged commit 3f311b6 into main Feb 10, 2025
6 checks passed
@dfederm dfederm deleted the avoid-materializing-recursively branch February 10, 2025 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants