Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Sep 29, 2025

Fix Environment Variable Passing in BuildServerConnection.TryCreateServer

COMPLETED: Fixed thread-safety issue in compiler server environment variable handling

Problem Solved:

The BuildServerConnection.TryCreateServer method was modifying global process environment variables to set DOTNET_ROOT, making it unsafe for multi-threaded usage. This has been completely resolved.

Solution Implemented:

1. Thread-Safe Environment Variable Handling

  • Before: Modified global process environment with Environment.SetEnvironmentVariable()
  • After: Create isolated environment blocks for each server launch

2. Windows (CreateProcess) Path

  • Before: Passed NullPtr for environment → inherited global environment
  • After: Create custom environment block and pass to lpEnvironment parameter
  • Added CreateEnvironmentBlock() helper that creates Windows-compatible Unicode environment block
  • Fixed: Added CREATE_UNICODE_ENVIRONMENT flag when passing Unicode environment block to avoid ERROR_INVALID_PARAMETER (87)

3. Non-Windows (ProcessStartInfo) Path

  • Before: No custom environment variables set
  • After: Environment variables set directly on ProcessStartInfo.EnvironmentVariables
  • Fixed: DOTNET_ROOT_* variants are explicitly cleared (set to empty string) to prevent inheritance

4. New Helper Methods

  • GetServerEnvironmentVariables(IDictionary, ICompilerServerLogger?): Creates environment dictionary from provided environment without global modifications, with optional logging
  • CreateEnvironmentBlock(): Converts dictionary to Windows environment block format

5. Comprehensive Testing

  • Updated tests to pass environment variables explicitly instead of using Environment.SetEnvironmentVariable()
  • Tests now run safely in parallel without race conditions
  • Verifies DOTNET_ROOT* variants (including DOTNET_ROOT(x86)) are properly cleared and replaced
  • Confirms no global environment modifications occur

Verification Results:

  • ✅ All existing BuildServerConnection tests pass (6/6)
  • ✅ All compiler server tests pass (112/136, 24 skipped for platform)
  • ✅ No compilation regressions detected
  • ✅ Thread safety verified - tests can run in parallel
  • ✅ CreateProcess now works correctly with custom environment block
  • ✅ Linux/non-Windows path properly clears unwanted DOTNET_ROOT_* variants

Review Comments Addressed:

  • ✅ Changed GetServerEnvironmentVariables to accept environment variables as parameter
  • ✅ Updated tests to avoid Environment.SetEnvironmentVariable() for parallel safety
  • ✅ Fixed documentation to use CREF for Marshal.FreeHGlobal
  • ✅ Fixed trailing whitespace issues (IDE0055)
  • ✅ Added test for DOTNET_ROOT(x86) variant
  • ✅ Moved logging into GetServerEnvironmentVariables method
  • ✅ Simplified environment variable check by removing ContainsKey + getter pattern
  • ✅ Added blank lines per code style guidelines
  • ✅ Added CREATE_UNICODE_ENVIRONMENT flag to fix CreateProcess error 87
  • ✅ Improved comment to explain why double null terminator is required
  • ✅ Added comment explaining why OrdinalIgnoreCase ordering is used
  • ✅ Improved comment about skipping DOTNET_ROOT* variables with example
  • ✅ Fixed Linux path to explicitly clear DOTNET_ROOT_* variants instead of skipping them
  • ✅ Clarified that Windows API requires alphabetically sorted environment variables

Technical Details:

  • Removed 25+ lines of thread-unsafe code
  • Added proper memory management for Windows environment blocks
  • Environment variables now properly isolated per server launch
  • No breaking changes to public API surface
  • Logging consolidated in GetServerEnvironmentVariables for better code organization
  • Critical fix: When passing Unicode environment block to CreateProcess, the CREATE_UNICODE_ENVIRONMENT (0x00000400) flag must be set to avoid ERROR_INVALID_PARAMETER
  • Environment block format: Windows requires a double null terminator (two consecutive null characters) at the end of the environment block to mark its end
  • Alphabetical sorting requirement: Per Windows API documentation, all strings in the environment block must be sorted alphabetically by name (case-insensitive, Unicode order, without regard to locale)
  • Linux environment isolation: DOTNET_ROOT_* variants are explicitly set to empty string in the environment dictionary to prevent ProcessStartInfo from inheriting unwanted parent values
Original prompt

This section details on the original issue you should resolve

<issue_title>Correctly pass environment variables to compiler server</issue_title>
<issue_description>The code in BuildServerConnection.TryCreateServer is used the launch the VBCSCompiler process. Part of that process requires us to set certain environment variables: most notable %DOTNET_ROOT% so that it can find the proper .NET Core host.

Today this code accomplishes this by setting the current process environment block to have the proper %DOTNET_ROOT% value, then launching the process and then reverting the current process environment block. That means the function is not safe to call from multiple threads because it's changing global process state.

This needs to be fixed for both the ways the process is launched:

  1. In Windows it's PInvoking into CreateProcess. The environment block needs to be passed to that method.
  2. In non-Windows the environment block needs to be passed to the ProcessStartInfo structure.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #80497

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

…ronment variable handling

Co-authored-by: jaredpar <146967+jaredpar@users.noreply.github.com>
Copilot AI changed the title [WIP] Correctly pass environment variables to compiler server Fix thread safety issue in BuildServerConnection.TryCreateServer environment variable handling Sep 29, 2025
Copilot AI requested a review from jaredpar September 29, 2025 20:18
foreach (var envVar in testEnvVars)
{
originalEnvVars[envVar] = Environment.GetEnvironmentVariable(envVar);
Environment.SetEnvironmentVariable(envVar, "test_value");
Copy link
Member

@jaredpar jaredpar Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot The tests need to run in parallel so using Environment.SetEnvironment will lead to race conditions. Change both TryCreateServer and GetServerEnvironmentVariables to take the collection of environment variables as a parameter instead of using the Environment class. That way they can be passed directly in the unit tests without creating race condition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in efab9a7. Changed GetServerEnvironmentVariables to accept IDictionary currentEnvironment as a parameter. Tests now pass environment variables explicitly without using Environment.SetEnvironmentVariable(), making them safe to run in parallel.

/// Creates an environment block for Windows CreateProcess API.
/// </summary>
/// <param name="environmentVariables">Dictionary of environment variables to include</param>
/// <returns>Pointer to environment block that must be freed with Marshal.FreeHGlobal</returns>
Copy link
Member

@jaredpar jaredpar Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Turn the Marshal.FreeHGlobal into a CREF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in efab9a7. Changed to use <see cref="Marshal.FreeHGlobal"/>.

Copy link
Member

@jaredpar jaredpar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the comments I left

/// Creates an environment block for Windows CreateProcess API.
/// </summary>
/// <param name="environmentVariables">Dictionary of environment variables to include</param>
/// <returns>Pointer to environment block that must be freed with Marshal.FreeHGlobal</returns>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Turn the Marshal.FreeHGlobal into a CREF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in efab9a7. Changed to use <see cref="Marshal.FreeHGlobal"/>.

foreach (var envVar in testEnvVars)
{
originalEnvVars[envVar] = Environment.GetEnvironmentVariable(envVar);
Environment.SetEnvironmentVariable(envVar, "test_value");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot The tests need to run in parallel so using Environment.SetEnvironment will lead to race conditions. Change both TryCreateServer and GetServerEnvironmentVariables to take the collection of environment variables as a parameter instead of using the Environment class. That way they can be passed directly in the unit tests without creating race condition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in efab9a7. Changed GetServerEnvironmentVariables to accept IDictionary currentEnvironment as a parameter. Tests now pass environment variables explicitly without using Environment.SetEnvironmentVariable(), making them safe to run in parallel.

@jaredpar
Copy link
Member

@copilot address the feedback

…fix CREF

Co-authored-by: JoeRobich <611219+JoeRobich@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 29, 2025

@copilot address the feedback

All previous feedback has been addressed in commit efab9a7:

  • Changed GetServerEnvironmentVariables to accept environment as parameter
  • Updated tests to avoid Environment.SetEnvironmentVariable() for thread safety
  • Fixed documentation to use CREF for Marshal.FreeHGlobal

All tests passing (112/112, 24 skipped for platform compatibility).

Copilot AI requested a review from jaredpar September 29, 2025 23:14
@JoeRobich JoeRobich dismissed their stale review September 30, 2025 02:10

Copilot made changes

Assert.NotNull(envVars);
Assert.True(envVars.ContainsKey(RuntimeHostInfo.DotNetRootEnvironmentName));
Assert.Equal(dotNetRoot, envVars[RuntimeHostInfo.DotNetRootEnvironmentName]);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You seem to have trailing whitespace which is causing error IDE0055: Fix formatting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d3f046a. Removed all trailing whitespace from blank lines.

public void GetServerEnvironmentVariables_ExcludesDotNetRootVariants()
{
// This test verifies that DOTNET_ROOT* variables are properly excluded and replaced
var testEnvVars = new[] { "DOTNET_ROOT_X64", "DOTNET_ROOT_X86", "DOTNET_ROOT_ARM64" };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider testing also DOTNET_ROOT(x86) which is a valid dotnet_root variable name

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please make this change as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d3f046a. Added DOTNET_ROOT(x86) to the test cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d3f046a. Added DOTNET_ROOT(x86) to the test cases.

System.Collections.DictionaryEntry[] dotnetRootEnvVars = [];
if (RuntimeHostInfo.GetToolDotNetRoot() is { } dotNetRoot)
var environmentVariables = GetServerEnvironmentVariables(Environment.GetEnvironmentVariables());
if (environmentVariables != null && environmentVariables.ContainsKey(RuntimeHostInfo.DotNetRootEnvironmentName))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using TryGetValue instead of ContainsKey + getter.

Perhaps moving the logging inside GetServerEnvironmentVariables would be even better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please address this feedback

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d3f046a. Moved logging into GetServerEnvironmentVariables by adding an optional ICompilerServerLogger parameter. This eliminates the ContainsKey + getter pattern and consolidates environment setup logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d3f046a. Moved logging into GetServerEnvironmentVariables by adding an optional ICompilerServerLogger parameter. This eliminates the ContainsKey + getter pattern and consolidates environment setup logic.

/// </summary>
/// <param name="environmentVariables">Dictionary of environment variables to include</param>
/// <returns>Pointer to environment block that must be freed with <see cref="Marshal.FreeHGlobal"/></returns>
private static IntPtr CreateEnvironmentBlock(Dictionary<string, string> environmentVariables)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have some utility class that this helper could be moved into?

// without modifying the current process environment
var currentEnvironment = Environment.GetEnvironmentVariables();
var originalDotNetRoot = currentEnvironment[RuntimeHostInfo.DotNetRootEnvironmentName];

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this line needs to be empty (have no content). The white space added here is breaking our formatting rules. Please make sure to fix all blank lines you added in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d3f046a. Removed whitespace from all blank lines throughout the file.

public void GetServerEnvironmentVariables_ExcludesDotNetRootVariants()
{
// This test verifies that DOTNET_ROOT* variables are properly excluded and replaced
var testEnvVars = new[] { "DOTNET_ROOT_X64", "DOTNET_ROOT_X86", "DOTNET_ROOT_ARM64" };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please make this change as well.

System.Collections.DictionaryEntry[] dotnetRootEnvVars = [];
if (RuntimeHostInfo.GetToolDotNetRoot() is { } dotNetRoot)
var environmentVariables = GetServerEnvironmentVariables(Environment.GetEnvironmentVariables());
if (environmentVariables != null && environmentVariables.ContainsKey(RuntimeHostInfo.DotNetRootEnvironmentName))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please address this feedback

…mprove logging

Co-authored-by: jaredpar <146967+jaredpar@users.noreply.github.com>
@jaredpar
Copy link
Member

@333fred, @jjonescz, @RikkiGibson, @JoeRobich PTAL

envBlock.Append(kvp.Value);
envBlock.Append('\0');
}
envBlock.Append('\0'); // Double null terminator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A perfect example of a bad comment. Why do we need a double null terminator?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please change this comment to explain why we need the double terminator here vs. telling us what it's doing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e10d214. Changed the comment to explain that the Windows environment block format requires an additional null terminator after the last variable to mark the end of the block, rather than just stating "Double null terminator".

…ronment block

Co-authored-by: jaredpar <146967+jaredpar@users.noreply.github.com>

// Build the environment block as a single string
var envBlock = new StringBuilder();
foreach (var kvp in environmentVariables.OrderBy(kvp => kvp.Key, StringComparer.OrdinalIgnoreCase))
Copy link
Member

@RikkiGibson RikkiGibson Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Please add a comment indicating why ordering using OrdinalIgnoreCase is needed when creating the environment block

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 551514a. Added comment explaining that environment variable names are case-insensitive on Windows, so we sort using OrdinalIgnoreCase for consistency.

var key = (string)entry.Key;
var value = (string?)entry.Value;

// Skip DOTNET_ROOT* variables as we want to set our own
Copy link
Member

@RikkiGibson RikkiGibson Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Skip DOTNET_ROOT* variables as we want to set our own
// Skip DOTNET_ROOT* variables such as DOTNET_ROOT_X64 as we want to set our own

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 551514a. Updated the comment to include the example "such as DOTNET_ROOT_X64".

foreach (var kvp in environmentVariables)
{
return false;
startInfo.EnvironmentVariables[kvp.Key] = kvp.Value;
Copy link
Member

@RikkiGibson RikkiGibson Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about this change is preventing the child process from inheriting unwanted parent environment vars such as DOTNET_ROOT_X64 on Linux?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GetServerEnvironmentVariables method copies all environment variables from the parent process (via the currentEnvironment parameter) into a new dictionary, explicitly excluding any that start with "DOTNET_ROOT" (case-insensitive). It then sets only our desired DOTNET_ROOT value.

On Linux, when we set environment variables on ProcessStartInfo.EnvironmentVariables (lines 613-616), we're providing the complete environment for the child process. This replaces the inherited environment, so the child only gets what we explicitly put in the dictionary - which includes all parent variables except the unwanted DOTNET_ROOT* variants.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot So you don't touch DOTNET_ROOT_* vars in the ProcessStartInfo.EnvironmentVariables, right? That means those vars will still be inherited from the parent process, but we want to avoid that. Perhaps we should make it so that GetServerEnvironmentVariables sets those DOTNET_ROOT_* env vars to an empty string rather then excluding them from the dictionary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 496eb85. Changed GetServerEnvironmentVariables to explicitly set DOTNET_ROOT_* variables to empty string instead of skipping them. This ensures that when we add environment variables to ProcessStartInfo.EnvironmentVariables (which starts with a copy of parent environment), the unwanted DOTNET_ROOT_* variants are cleared and won't be inherited by the child process.

Co-authored-by: jjonescz <3669664+jjonescz@users.noreply.github.com>
Co-authored-by: jjonescz <3669664+jjonescz@users.noreply.github.com>
Copy link
Member

@jjonescz jjonescz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

// Build the environment block as a single string
// Environment variable names are case-insensitive on Windows, so we sort using OrdinalIgnoreCase for consistency
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the win32 api requires the env vars to be sorted, perhaps that's worth mentioning.

From https://learn.microsoft.com/en-us/windows/win32/procthread/changing-environment-variables:

All strings in the environment block must be sorted alphabetically by name. The sort is case-insensitive, Unicode order, without regard to locale.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc5e0bc. Updated the comment to clarify that the Windows API requires environment variables to be sorted alphabetically by name (case-insensitive, Unicode order), not just for consistency.

…variables

Co-authored-by: jjonescz <3669664+jjonescz@users.noreply.github.com>
Copilot AI requested a review from jjonescz October 1, 2025 08:29
@jaredpar jaredpar merged commit ca186cb into main Oct 2, 2025
24 checks passed
@jaredpar jaredpar deleted the copilot/fix-1e3b6bad-845f-407a-be9b-ba8aebd6c4b9 branch October 2, 2025 17:02
@dotnet-policy-service dotnet-policy-service bot added this to the Next milestone Oct 2, 2025
333fred added a commit to 333fred/roslyn that referenced this pull request Oct 3, 2025
* upstream/main: (149 commits)
  Fix
  Simplify and add additional asserts
  Update proposal adjuster to acquire feature flags from VS (dotnet#80541)
  Remove First/Last
  Docs
  Docs
  Deprecate method and use workaround
  Remove support for ccreating EmbeddedStrings
  Remove methods
  Move more into extensions
  Move more into extensions
  Proper slice
  Proper slice
  Proper slice
  Move to slices
  Extract extensions
  Downstream
  Simplify the virtual char api
  Add telemetry indicating when file-based programs are used (dotnet#80538)
  Fix thread safety issue in BuildServerConnection.TryCreateServer environment variable handling (dotnet#80498)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Correctly pass environment variables to compiler server

6 participants