enable out of process execution of inline tasks

documentation/wiki/MSBuild-Environment-Variables.md

@@ -1,33 +1,36 @@
 # MSBuild environment variables list
 
-This document describes the environment variables that are respected in MSBuild, its purpose and usage. 
+This document describes the environment variables that are respected in MSBuild, its purpose and usage.
 
 Some of the env variables listed here are unsupported, meaning there is no guarantee that variable or a specific combination of multiple variables will be respected in upcoming release, so please use at your own risk.
 
-* `MSBuildDebugEngine=1` & `MSBUILDDEBUGPATH=<DIRECTORY>`
-  * Set this to cause any MSBuild invocation launched within this environment to emit binary logs and additional debugging information to `<DIRECTORY>`. Useful when debugging build or evaluation issues when you can't directly influence the MSBuild invocation, such as in Visual Studio. More details on [capturing binary logs](./Providing-Binary-Logs.md)
-* `MSBUILDTARGETOUTPUTLOGGING=1`
-   * Set this to enable [printing all target outputs to the log](https://learn.microsoft.com/archive/blogs/msbuild/displaying-target-output-items-using-the-console-logger).
-* `MSBUILDLOGTASKINPUTS=1`
-   * Log task inputs (not needed if there are any diagnostic loggers already).
- * `MSBUILDEMITSOLUTION=1`
-   * Save the generated .proj file for the .sln that is used to build the solution. The generated files are emitted into a binary log by default and their presence on disk can break subsequent builds.
-* `MSBUILDENABLEALLPROPERTYFUNCTIONS=1`
-   * Enable [additional property functions](https://devblogs.microsoft.com/visualstudio/msbuild-property-functions/). If you need this level of detail you are generally served better with a binary log than the text log.
-* `MSBUILDLOGVERBOSERARSEARCHRESULTS=1`
-   * In ResolveAssemblyReference task, log verbose search results.
-* `MSBUILDLOGCODETASKFACTORYOUTPUT=1`
-   * Dump generated code for task to a <GUID>.txt file in the TEMP directory
-* `MSBUILDDISABLENODEREUSE=1`
-   * Set this to not leave MSBuild processes behind (see `/nr:false`, but the environment variable is useful to also set this for Visual Studio for example).
-* `MSBUILDLOGASYNC=1`
-   * Enable asynchronous logging.
-* `MSBUILDDEBUGONSTART=1`
-   * Launches debugger on build start. Works on Windows operating systems only.  
-   * Setting the value of 2 allows for manually attaching a debugger to a process ID. This works on Windows and non-Windows operating systems.
-* `MSBUILDDEBUGSCHEDULER=1` & `MSBUILDDEBUGPATH=<DIRECTORY>`
-   * Dumps scheduler state at specified directory.
-
-* `MsBuildSkipEagerWildCardEvaluationRegexes`
-  *  If specified, overrides the default behavior of glob expansion. During glob expansion, if the path with wildcards that is being processed matches one of the regular expressions provided in the [environment variable](#msbuildskipeagerwildcardevaluationregexes), the path is not processed (expanded). 
-  * The value of the environment  variable is a list of regular expressions, separated by semicolon (;).
+- `MSBuildDebugEngine=1` & `MSBUILDDEBUGPATH=<DIRECTORY>`
+  - Set this to cause any MSBuild invocation launched within this environment to emit binary logs and additional debugging information to `<DIRECTORY>`. Useful when debugging build or evaluation issues when you can't directly influence the MSBuild invocation, such as in Visual Studio. More details on [capturing binary logs](./Providing-Binary-Logs.md)
+- `MSBUILDTARGETOUTPUTLOGGING=1`
+  - Set this to enable [printing all target outputs to the log](https://learn.microsoft.com/archive/blogs/msbuild/displaying-target-output-items-using-the-console-logger).
+- `MSBUILDLOGTASKINPUTS=1`
+  - Log task inputs (not needed if there are any diagnostic loggers already).
+- `MSBUILDEMITSOLUTION=1`
+  - Save the generated .proj file for the .sln that is used to build the solution. The generated files are emitted into a binary log by default and their presence on disk can break subsequent builds.
+- `MSBUILDENABLEALLPROPERTYFUNCTIONS=1`
+  - Enable [additional property functions](https://devblogs.microsoft.com/visualstudio/msbuild-property-functions/). If you need this level of detail you are generally served better with a binary log than the text log.
+- `MSBUILDLOGVERBOSERARSEARCHRESULTS=1`
+  - In ResolveAssemblyReference task, log verbose search results.
+- `MSBUILDLOGCODETASKFACTORYOUTPUT=1`
+  - Dump generated code for task to a `<GUID>.txt` file in the TEMP directory
+- `MSBUILDDISABLENODEREUSE=1`
+  - Set this to not leave MSBuild processes behind (see `/nr:false`, but the environment variable is useful to also set this for Visual Studio for example).
+- `MSBUILDLOGASYNC=1`
+  - Enable asynchronous logging.
+- `MSBUILDDEBUGONSTART=1`
+  - Launches debugger on build start. Works on Windows operating systems only.
+  - Setting the value of 2 allows for manually attaching a debugger to a process ID. This works on Windows and non-Windows operating systems.
+- `MSBUILDDEBUGSCHEDULER=1` & `MSBUILDDEBUGPATH=<DIRECTORY>`
+  - Dumps scheduler state at specified directory.
+- `MsBuildSkipEagerWildCardEvaluationRegexes`
+  - If specified, overrides the default behavior of glob expansion. During glob expansion, if the path with wildcards that is being processed matches one of the regular expressions provided in the [environment variable](#msbuildskipeagerwildcardevaluationregexes), the path is not processed (expanded).
+  - The value of the environment variable is a list of regular expressions, separated by semicolon (;).
+- `MSBUILDFORCEALLTASKSOUTOFPROCESS`
+  - Set this to force all tasks to run out of process (except inline tasks).
+- `MSBUILDFORCEINLINETASKFACTORIESOUTOFPROC`
+  - Set this to force all inline tasks to run out of process. It is not compatible with custom TaskFactories.

documentation/wiki/Tasks.md

@@ -1,56 +1,72 @@
 # Tasks
+
 A Task is a unit of execution in a Target and a method of extensibility in MSBuild.
+
 ## Basics
+
 A task is a class implementing [`ITask`](https://github.com/dotnet/msbuild/blob/main/src/Framework/ITask.cs).
-- The notable method of this interface is `bool Execute()`. Code in it will get executed when the task is run. 
+
+- The notable method of this interface is `bool Execute()`. Code in it will get executed when the task is run.
 - A Task can have public properties that can be set by the user in the project file.
-    - These properties can be `string`, `bool`, `ITaskItem` (representation of a file system object), `string[]`, `bool[]`, `ITaskItem[]`
-    - the properties can have attributes `[Required]` which causes the engine to check that it has a value when the task is run and `[Output]` which exposes the property to be used again in XML
+  - These properties can be `string`, `bool`, `ITaskItem` (representation of a file system object), `string[]`, `bool[]`, `ITaskItem[]`
+  - the properties can have attributes `[Required]` which causes the engine to check that it has a value when the task is run and `[Output]` which exposes the property to be used again in XML
+
 - Tasks have the `Log` property set by the engine to log messages/errors/warnings.
 
 ## Internals
+
 - [`TaskRegistry`](https://github.com/dotnet/msbuild/blob/main/src/Build/Instance/TaskRegistry.cs) - has a list of all available tasks for the build, and resolves them.
 - [`TaskExecutionHost`](https://github.com/dotnet/msbuild/tree/main/src/Build/BackEnd/TaskExecutionHost) - finds task in TaskRegistry, calls TaskFactory to create an instance of the task and sets its properties using reflection from the values in the XML. Then calls Execute on the task and gathers the outputs.
 - TaskFactory - initializes the task, creates instance of the task
 - ITask class - runs `Execute()` method
 
 ## Custom Tasks
+
 Users can implement `ITask` and compile it into a .dll.
 Then they can use in project file:
+
 ```xml
 <UsingTask TaskName="MyTaskClass" AssemblyFile="MyTasks.dll"/>
 ```
+
 This uses the AssemblyTaskFactory to load the task from the .dll and create an instance of it.
 
 ## Diagram of task lifecycle
 
 ```mermaid
-graph 
+graph
 
 I["Implement:<br/>extend ITask interface in .dll"] --> R["Register:<br/>&lt;UsingTask /&gt;"] --> U["Use in XML:<br/>&lt;Target&gt;<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MyTask /&gt;<br/>&lt;/Target&gt;"] --> In["Initialize:<br/> compile inline or load from assembly <br/>(TaskFactory)"] --> S["Setup:<br/> Set input properties<br/> (TaskExecutionHost)"] --> E["ITask.Execute()"] --> O["Gather outputs: <br/> (TaskExecutionHost)"]
 ```
 
 ## Task Factories
+
 Task factories create instances of tasks. They implement [`ITaskFactory`](https://github.com/dotnet/msbuild/blob/main/src/Framework/ITaskFactory.cs) or [`ITaskFactory2`](https://github.com/dotnet/msbuild/blob/main/src/Framework/ITaskFactory2.cs).
 This interface defines `bool Initialize(...)` and `ITask CreateTask(...)`.
 They are e.g. responsible for loading a task from an assembly and initializing it.
 
+The trait `MSBUILDFORCEINLINETASKFACTORIESOUTOFPROC` forces inline tasks in an out of process TaskHost. It is not compatible with custom TaskFactories.
+
 ### Built-in Task Factories
+
 - [`AssemblyTaskFactory`](https://github.com/dotnet/msbuild/blob/main/src/Build/Instance/TaskFactories/AssemblyTaskFactory.cs) - constructs tasks from .NET assemblies
 - [`RoslynCodeTaskFactory`](https://github.com/dotnet/msbuild/blob/main/src/Tasks/RoslynCodeTaskFactory/RoslynCodeTaskFactory.cs) - inline code tasks
 - CodeTaskFactory, XamlTaskFactory - old, rarely used
 
 ### Custom Task Factories
+
 This is a rarely used method of extensibility.
 Users can implement `ITaskFactory` to create custom task factories.
 Then they can use in project file:
+
 ```xml
 <UsingTask TaskName="MyTask" AssemblyFile="Factory.dll" Factory="MyTaskFactory">
     <Task>Insides that the MyTaskFactory uses to initialize</Task>
 </UsingTask>
 ```
 
-# Microsoft Learn Resources
+## Microsoft Learn Resources
+
 - [MSBuild task](https://learn.microsoft.com/visualstudio/msbuild/msbuild-task)
 - [Task reference](https://learn.microsoft.com/visualstudio/msbuild/msbuild-task-reference)
 - [Task Writing](https://learn.microsoft.com/visualstudio/msbuild/task-writing)

src/Build/BackEnd/BuildManager/BuildManager.cs

@@ -1148,6 +1148,11 @@ public void EndBuild()
                     Reset();
                     _buildManagerState = BuildManagerState.Idle;
 
+                    if (Traits.Instance.ForceTaskFactoryOutOfProc)
+                    {
+                        TaskFactoryUtilities.CleanCurrentProcessInlineTaskDirectory();
+                    }
+
                     MSBuildEventSource.Log.BuildStop();
 
                     _threadException?.Throw();

src/Build/BackEnd/TaskExecutionHost/TaskExecutionHost.cs

@@ -980,14 +980,37 @@ private ITask InstantiateTask(IDictionary<string, string> taskIdentityParameters
                 else
                 {
                     TaskFactoryLoggingHost loggingHost = new TaskFactoryLoggingHost(_buildEngine.IsRunningMultipleNodes, _taskLocation, _taskLoggingContext);
+                    bool isTaskHost = false;
                     try
                     {
-                        task = _taskFactoryWrapper.TaskFactory is ITaskFactory2 taskFactory2 ?
-                            taskFactory2.CreateTask(loggingHost, taskIdentityParameters) :
-                            _taskFactoryWrapper.TaskFactory.CreateTask(loggingHost);
+                        // Check if we should force out-of-process execution for non-AssemblyTaskFactory instances
+                        // IntrinsicTaskFactory tasks run in proc always
+                        if (Traits.Instance.ForceTaskFactoryOutOfProc && _taskFactoryWrapper.TaskFactory is not IntrinsicTaskFactory)
+                        {
+                            // Custom Task factories are not supported, internal TaskFactories implement this marker interface
+                            if (_taskFactoryWrapper.TaskFactory is not IOutOfProcTaskFactory outOfProcTaskFactory)
+                            {
+                                _taskLoggingContext.LogError(
+                                    new BuildEventFileInfo(_taskLocation),
+                                    "CustomTaskFactoryOutOfProcNotSupported",
+                                    _taskFactoryWrapper.TaskFactory.FactoryName,
+                                    _taskName);
+                                return null;
+                            }
+
+                            task = CreateTaskHostTaskForOutOfProcFactory(taskIdentityParameters, loggingHost, outOfProcTaskFactory);
+                            isTaskHost = true;
+                        }
+                        else
+                        {
+                            // Normal in-process execution for custom task factories
+                            task = _taskFactoryWrapper.TaskFactory is ITaskFactory2 taskFactory2 ?
+                                taskFactory2.CreateTask(loggingHost, taskIdentityParameters) :
+                                _taskFactoryWrapper.TaskFactory.CreateTask(loggingHost);
+                        }
 
-                        // Track telemetry for non-AssemblyTaskFactory task factories. No task can go to the task host.
-                        _taskLoggingContext?.TargetLoggingContext?.ProjectLoggingContext?.ProjectTelemetry?.AddTaskExecution(_taskFactoryWrapper.TaskFactory.GetType().FullName, isTaskHost: false);
+                        // Track telemetry for non-AssemblyTaskFactory task factories
+                        _taskLoggingContext?.TargetLoggingContext?.ProjectLoggingContext?.ProjectTelemetry?.AddTaskExecution(_taskFactoryWrapper.TaskFactory.GetType().FullName, isTaskHost);
                     }
                     finally
                     {
@@ -1703,5 +1726,78 @@ private void DisplayCancelWaitMessage()
                 // if the task logging context is no longer valid, we choose to eat the exception because we can't log the message anyway.
             }
         }
+
+        /// <summary>
+        /// Creates a <see cref="TaskHostTask"/> wrapper to run a non-AssemblyTaskFactory task out of process.
+        /// This is used when Traits.Instance.ForceTaskFactoryOutOfProc is true to ensure
+        /// non-AssemblyTaskFactory tasks run in isolation.
+        /// </summary>
+        /// <param name="taskIdentityParameters">Task identity parameters.</param>
+        /// <param name="loggingHost">The logging host to use for the task.</param>
+        /// <param name="outOfProcTaskFactory">The out-of-process task factory instance.</param>
+        /// <returns>A TaskHostTask that will execute the inner task out of process, or <code>null</code> if task creation fails.</returns>
+        private ITask CreateTaskHostTaskForOutOfProcFactory(IDictionary<string, string> taskIdentityParameters, TaskFactoryLoggingHost loggingHost, IOutOfProcTaskFactory outOfProcTaskFactory)
+        {
+            ITask innerTask;
+
+            innerTask = _taskFactoryWrapper.TaskFactory is ITaskFactory2 taskFactory2 ?
+                taskFactory2.CreateTask(loggingHost, taskIdentityParameters) :
+                _taskFactoryWrapper.TaskFactory.CreateTask(loggingHost);
+
+            if (innerTask == null)
+            {
+                return null;
+            }
+
+            // Create a LoadedType for the actual task type so we can wrap it in TaskHostTask
+            Type taskType = innerTask.GetType();
+
+            // For out-of-process inline tasks, get the assembly path from the factory
+            // (Assembly.Location is typically empty for inline tasks loaded from bytes)
+            string resolvedAssemblyLocation = outOfProcTaskFactory.GetAssemblyPath();
+
+            // This should never happen - if the factory can create a task, it should know where the assembly is
+            ErrorUtilities.VerifyThrow(!string.IsNullOrEmpty(resolvedAssemblyLocation), 
+                $"IOutOfProcTaskFactory {_taskFactoryWrapper.TaskFactory.FactoryName} created a task but returned null/empty assembly path");
+
+            LoadedType taskLoadedType = new LoadedType(
+                taskType,
+                AssemblyLoadInfo.Create(null, resolvedAssemblyLocation),
+                taskType.Assembly,
+                typeof(ITaskItem));
+
+            // Default task host parameters for out-of-process execution for inline tasks
+            Dictionary<string, string> taskHostParameters = new Dictionary<string, string>
+            {
+                [XMakeAttributes.runtime] = XMakeAttributes.GetCurrentMSBuildRuntime(),
+                [XMakeAttributes.architecture] = XMakeAttributes.GetCurrentMSBuildArchitecture()
+            };
+
+            // Merge with any existing task identity parameters
+            if (taskIdentityParameters?.Count > 0)
+            {
+                foreach (var kvp in taskIdentityParameters.Where(kvp => !taskHostParameters.ContainsKey(kvp.Key)))
+                {
+                    taskHostParameters[kvp.Key] = kvp.Value;
+                }
+            }
+
+            // Clean up the original task since we're going to wrap it
+            _taskFactoryWrapper.TaskFactory.CleanupTask(innerTask);
+
+#pragma warning disable SA1111, SA1009 // Closing parenthesis should be on line of last parameter
+            return new TaskHostTask(
+                _taskLocation,
+                _taskLoggingContext,
+                _buildComponentHost,
+                taskHostParameters,
+                taskLoadedType,
+                true
+#if FEATURE_APPDOMAIN
+                , AppDomainSetup
+#endif
+                );
+#pragma warning restore SA1111, SA1009 // Closing parenthesis should be on line of last parameter
+        }
     }
 }

src/Build/Instance/TaskFactories/TaskHostTask.cs

@@ -277,6 +277,13 @@ public bool Execute()
                 ErrorUtilities.VerifyThrowInternalNull(_taskHostProvider, "taskHostProvider");
             }
 
+            string taskLocation = AssemblyUtilities.GetAssemblyLocation(_taskType.Type.GetTypeInfo().Assembly);
+            if (string.IsNullOrEmpty(taskLocation))
+            {
+                // fall back to the AssemblyLoadInfo location for inline tasks loaded from bytes
+                taskLocation = _taskType?.Assembly?.AssemblyLocation ?? string.Empty;
+            }
+
             TaskHostConfiguration hostConfiguration =
                 new TaskHostConfiguration(
                         _buildComponentHost.BuildParameters.NodeId,
@@ -292,7 +299,7 @@ public bool Execute()
                         BuildEngine.ProjectFileOfTaskNode,
                         BuildEngine.ContinueOnError,
                         _taskType.Type.FullName,
-                        AssemblyUtilities.GetAssemblyLocation(_taskType.Type.GetTypeInfo().Assembly),
+                        taskLocation,
                         _buildComponentHost.BuildParameters.LogTaskInputs,
                         _setParameters,
                         new Dictionary<string, string>(_buildComponentHost.BuildParameters.GlobalProperties),

src/Build/Instance/TaskRegistry.cs

@@ -1277,7 +1277,7 @@ public bool ComputeIfCustom()
                          !FileClassifier.IsMicrosoftAssembly(_taskFactoryAssemblyLoadInfo.AssemblyName)) ||
                         (!string.IsNullOrEmpty(_taskFactoryAssemblyLoadInfo.AssemblyFile) &&
                          // This condition will as well capture Microsoft tasks pulled from NuGet cache - since we decide based on assembly name.
-                         // Hence we do not have to add the 'IsMicrosoftPackageInNugetCache' call anywhere here 
+                         // Hence we do not have to add the 'IsMicrosoftPackageInNugetCache' call anywhere here
                          !FileClassifier.IsMicrosoftAssembly(Path.GetFileName(_taskFactoryAssemblyLoadInfo.AssemblyFile)) &&
                          !FileClassifier.Shared.IsBuiltInLogic(_taskFactoryAssemblyLoadInfo.AssemblyFile)))
                     // and let's consider all tasks imported by common targets as non custom logic.

src/Build/Microsoft.Build.csproj

@@ -95,6 +95,7 @@
       <Link>Collections\ReadOnlyEmptyCollection.cs</Link>
     </Compile>
     <Compile Include="..\Shared\BufferedReadStream.cs" />
+    <Compile Include="..\Shared\TaskFactoryUtilities.cs" />
     <Compile Include="..\GlobalUsings.cs" Link="GlobalUsings.cs" />
     <Compile Include="..\Shared\TaskHostConfiguration.cs" />
     <Compile Include="..\Shared\TaskHostTaskCancelled.cs" />