Add documentation for enabling binlog collection via env var #12805

YuliiaKovalova · 2025-11-21T15:05:44Z

Document the process for enabling binary logging in CI/CD pipelines using environment variables, including supported arguments, argument processing order, and implementation flow.

Fixes #12804

Document the process for enabling binary logging in CI/CD pipelines using environment variables, including supported arguments, argument processing order, and implementation flow.

baronfel · 2025-11-21T16:30:26Z

documentation/specs/enable-binlog-collection-by-env-var

+
+## Warning Messages
+
+All issues are logged as **warnings**, never errors. Builds must not fail due to environment variable processing.


Would these be Messages logged through the configured Loggers, or raw CLI output? If the latter, we should strongly consider writing these to stderr - in general all 'verbose', non-Logger-provided output should go to that channel.

I agree in principal but I'm not sure that this specific scenario is the best place to start using stderr--I suspect many build scripts don't capture it well.

We've started breaking that seal in the dotnet CLI in 10.0.1xx, and busted it wide open in 10.0.200 already. It's a bigger unknown for msbuild.exe for sure though.

jaredpar · 2025-11-26T13:27:24Z

documentation/specs/enable-binlog-collection-by-env-var

@@ -0,0 +1,113 @@
+## Purpose
+
+Enable binary logging in CI/CD pipelines when `-noAutoResponse` blocks response files, without modifying build scripts or project files.


Think that this is a problem even without -noAutoResponse. Using an RSP file would necessitate writing artifacts to disk or modifying an existing RSP file in place. The goal I was thinking of is how can you enable collection without touching the artifacts on disk at all.

without touching the artifacts on disk at all

Strong agree.

jaredpar · 2025-11-26T13:28:29Z

documentation/specs/enable-binlog-collection-by-env-var

+- `-bl` / `/bl` / `-binarylogger` / `/binarylogger` (with optional parameters)
+- `-check` / `/check` (with optional parameters - requires code modification to support `deferred` mode)
+
+**All other switches are blocked** as security risks. These two switches are diagnostic/monitoring tools that don't affect build behavior or enable code execution.


What are the security risks that this poses? My mental model is that once you're able to inject environment variables into the build then you can change a lot of operations.

Not disputing this angle, just trying to get my head into the right place here.

environment variables are assumed to be trusted in our threat model, you can specify loading of dlls or proj/targets/props files that will be executed as a part of the build

IMO this restriction is not a security measure, it's more about diagnosability. It is already very annoying to discover that stray environment variables have acted as build inputs (Platform is an old favorite env var that horks builds). The reason I prefer a restricted logging-only env var is to avoid being able to do so in a new, even harder-to-understand way.

We currently log the entire command line in the binlog, but we do not 'classify' where arguments of that command line came from. Same is true for various trait flag. It might be an interesting future enhancement to add this provenance information to the binlog.

documentation/specs/enable-binlog-collection-by-env-var

jaredpar · 2025-11-26T13:34:13Z

documentation/specs/enable-binlog-collection-by-env-var

+
+**Implementation Note**: MSBuild's existing argument parser handles merging logic. The environment variable arguments are prepended to the command line before parsing, ensuring natural precedence.
+
+## Implementation Flow


Where all would this end up impacting the builds? Certainly expect that dotnet build and msbuild will process these args but what else? Guessing that say the builds done in a Roslyn msbuild workspace would not emit logs (or would they)?

Correct--only things that go through MSBuild's Main(), so dotnet build/dotnet msbuild/MSBuild.exe and not MSBuildWorkspace or VS or MyFancyCustomBuildDriverViaAPI.exe.

The only thing that makes me super worried there is devenv.exe /build which is definitely used in official-build scenarios in the wild (though is rare inside Microsoft).

We do have the ability to inject a binlog into even API-driven builds (MSBUILDDEBUGENGINE, which we use for debugging VS builds). We could use that instead of new functionality. I think this spec should discuss and address that.

added this section

Copilot

Pull request overview

This PR adds documentation for a proposed feature that would enable binary logging in CI/CD pipelines through the MSBUILD_LOGGING_ARGS environment variable. The feature is designed to work even when -noAutoResponse blocks response files, providing an alternative mechanism for enabling diagnostic logging.

Key changes:

Documents the MSBUILD_LOGGING_ARGS environment variable and its purpose
Specifies supported arguments (-bl and -check) and argument processing order
Describes parameter merging behavior and implementation flow

documentation/specs/enable-binlog-collection-by-env-var

### Context Docs-only PRs fail the Code Coverage CI job because tests are skipped (no coverage artifacts generated), but the job unconditionally attempts to download and publish those artifacts. ### Changes Made Modified `.vsts-dotnet-ci.yml` CodeCoverage job to respect the `onlyDocChanged` flag: - Added dependency on `IfOnlyDocumentionChanged` job to access the flag - Added `onlyDocChanged` variable to job variables - Conditioned all artifact downloads on `eq(variables.onlyDocChanged, 0)` - Conditioned coverage processing and publishing on `eq(variables.onlyDocChanged, 0)` This aligns with how other jobs (BootstrapMSBuildOnFullFrameworkWindows, CoreBootstrappedOnLinux, etc.) already handle docs-only PRs. ### Testing Pipeline configuration change - will be validated on next PR run. When `onlyDocChanged = 1`, all coverage tasks skip; when `onlyDocChanged = 0`, coverage runs normally. ### Notes Pre-existing typos in job/variable names (`Documention`, `Varibale`) intentionally preserved for consistency across the pipeline.  <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>docs only PRs fail code coverage CI check</issue_title> > <issue_description>#12805 > #12857 > > ``` > Code Coverage failed > [3 errors / 0 warnings](https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1217301) > > Annotations > [Check failure on line 9 in Build log](https://github.com/dotnet/msbuild/pull/12857/files#annotation_42398343450) > > @azure-pipelines > azure-pipelines > / msbuild-pr (Code Coverage) > Build log #L9 > > Publishing build artifacts failed with an error: Not found PathtoPublish: D:\a\1\s\artifacts\CoverageResults\merged.coverage > [Check failure on line 11 in Build log](https://github.com/dotnet/msbuild/pull/12857/files#annotation_42398343487) > > @azure-pipelines > azure-pipelines > / msbuild-pr (Code Coverage) > Build log #L11 > > Artifact LinuxCoreCoverage was not found for build 1217301. > [Check failure on line 9 in Build log](https://github.com/dotnet/msbuild/pull/12857/files#annotation_42398343497) > > @azure-pipelines > azure-pipelines > / msbuild-pr (Code Coverage) > Build log #L9 > > Publishing build artifacts failed with an error: Not found PathtoPublish: D:\a\1\s\artifacts\CoverageResults\merged.cobertura.xml > ``` > > This is probably because testing is skipped on docs only PRs, in that case code coverage should also be skipped</issue_description> > > <agent_instructions>fix this issue, note that the CI pipeline definition is called .vsts-dotnet-ci.yml</agent_instructions> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> - Fixes #12878  --- ✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/msbuild/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: JanProvaznik <25267098+JanProvaznik@users.noreply.github.com>

rainersigwald · 2025-12-05T16:18:46Z

documentation/specs/enable-binlog-collection-by-env-var

@@ -0,0 +1,113 @@
+## Purpose
+
+Enable binary logging in CI/CD pipelines when `-noAutoResponse` blocks response files, without modifying build scripts or project files.


without touching the artifacts on disk at all

Strong agree.

rainersigwald · 2025-12-05T16:21:18Z

documentation/specs/enable-binlog-collection-by-env-var

+- `-bl` / `/bl` / `-binarylogger` / `/binarylogger` (with optional parameters)
+- `-check` / `/check` (with optional parameters - requires code modification to support `deferred` mode)
+
+**All other switches are blocked** as security risks. These two switches are diagnostic/monitoring tools that don't affect build behavior or enable code execution.


IMO this restriction is not a security measure, it's more about diagnosability. It is already very annoying to discover that stray environment variables have acted as build inputs (Platform is an old favorite env var that horks builds). The reason I prefer a restricted logging-only env var is to avoid being able to do so in a new, even harder-to-understand way.

documentation/specs/enable-binlog-collection-by-env-var

rainersigwald · 2025-12-05T16:23:11Z

documentation/specs/enable-binlog-collection-by-env-var

+3. **MSBUILD_LOGGING_ARGS** - always processed, regardless of `-noAutoResponse`
+4. **Command-line arguments** - highest precedence
+
+The environment variable is processed **after** checking for `-noAutoResponse`, making it a reliable fallback when response files are disabled.


I don't see how this matters, since it's additive-only.

rainersigwald · 2025-12-05T16:31:18Z

documentation/specs/enable-binlog-collection-by-env-var

+**Binary Logger (-bl)**
+- Multiple `-bl` arguments are **allowed and cumulative**
+- Each creates a separate binlog file
+- Example: `MSBUILD_LOGGING_ARGS=-bl:env.binlog` + command line `-bl:cmd.binlog` → both files created


This is already handled by the existing CLI logic, right? Or will be after #12706?

it will be in place after #12706

documentation/specs/enable-binlog-collection-by-env-var

rainersigwald · 2025-12-05T16:39:02Z

documentation/specs/enable-binlog-collection-by-env-var

+
+## Supported Arguments
+
+- `-bl` / `/bl` / `-binarylogger` / `/binarylogger` (with optional parameters)


Should we be even more specific than this? I think for the use case we're thinking of the only bl arguments that make sense have a path and a {} placeholder (to allow multiple build invocations without overwriting logs).

added this to the document

rainersigwald · 2025-12-05T16:40:37Z

documentation/specs/enable-binlog-collection-by-env-var

+
+## Warning Messages
+
+All issues are logged as **warnings**, never errors. Builds must not fail due to environment variable processing.


Note that new warnings can easily be errors in a customer build. I think our position is that "if you opt into new functionality by setting this environment variable, you are also opting into those warnings", right?

rainersigwald · 2025-12-05T16:41:14Z

documentation/specs/enable-binlog-collection-by-env-var

+
+- **Informational**: "Using arguments from MSBUILD_LOGGING_ARGS environment variable: {0}"
+- **Rejected Argument**: "MSBUILD_LOGGING_ARGS: Ignoring unsupported argument '{0}'. Only -bl and -check arguments are allowed."
+- **Processing Error**: "Error processing MSBUILD_LOGGING_ARGS environment variable: {0}"


In this last error case, what should MSBuild do concretely? What's the difference between "ignoring an argument" and "error"?

updated the section

documentation/specs/enable-binlog-collection-by-env-var

Add documentation for enabling binlog collection via env var

1f96d76

Document the process for enabling binary logging in CI/CD pipelines using environment variables, including supported arguments, argument processing order, and implementation flow.

baronfel reviewed Nov 21, 2025

View reviewed changes

Merge branch 'main' into dev/ykovalova/binlog_env_var_spec

e77a44d

jaredpar reviewed Nov 26, 2025

View reviewed changes

YuliiaKovalova self-assigned this Dec 3, 2025

YuliiaKovalova marked this pull request as ready for review December 5, 2025 07:58

Copilot AI review requested due to automatic review settings December 5, 2025 07:58

Copilot started reviewing on behalf of YuliiaKovalova December 5, 2025 07:58 View session

Copilot finished reviewing on behalf of YuliiaKovalova December 5, 2025 08:02

Copilot AI reviewed Dec 5, 2025

View reviewed changes

JanProvaznik mentioned this pull request Dec 5, 2025

docs only PRs fail code coverage CI check #12878

Closed

Copilot AI mentioned this pull request Dec 5, 2025

Skip code coverage tasks for docs-only PRs #12879

Merged

Merge branch 'main' into dev/ykovalova/binlog_env_var_spec

30fb3e8

rainersigwald reviewed Dec 5, 2025

View reviewed changes

YuliiaKovalova added 2 commits December 8, 2025 11:32

fix review comments

e7a7da1

more cleanup

3ed3407


		## Warning Messages

		All issues are logged as warnings, never errors. Builds must not fail due to environment variable processing.

		@@ -0,0 +1,113 @@
		## Purpose

		Enable binary logging in CI/CD pipelines when `-noAutoResponse` blocks response files, without modifying build scripts or project files.


		Implementation Note: MSBuild's existing argument parser handles merging logic. The environment variable arguments are prepended to the command line before parsing, ensuring natural precedence.

		## Implementation Flow


		## Supported Arguments

		- `-bl` / `/bl` / `-binarylogger` / `/binarylogger` (with optional parameters)

Add documentation for enabling binlog collection via env var #12805

Are you sure you want to change the base?

Add documentation for enabling binlog collection via env var #12805

Conversation

YuliiaKovalova commented Nov 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants