Skip to content

Track subclasses of MSBuild tasks for telemetry#12623

Merged
YuliiaKovalova merged 5 commits intomainfrom
copilot/collect-task-telemetry-data
Oct 9, 2025
Merged

Track subclasses of MSBuild tasks for telemetry#12623
YuliiaKovalova merged 5 commits intomainfrom
copilot/collect-task-telemetry-data

Conversation

Copy link
Contributor

Copilot AI commented Oct 8, 2025

Summary

Adds telemetry collection to track user-authored subclasses of Microsoft-owned MSBuild tasks. This data will help inform decisions about making tasks sealed and other API changes. This addresses the requirements outlined in #10855.

Changes

Telemetry Collection

  • New telemetry event: build/tasks/msbuild-subclassed tracks subclasses of Microsoft MSBuild tasks
  • Data collected: Dictionary mapping Microsoft task names to counts of their subclass usage
    • Example: Microsoft_Build_Utilities_Task: 2 indicates 2 user tasks inherit from Microsoft.Build.Utilities.Task
    • Tracks user-authored (non-Microsoft) tasks regardless of sealed status
    • Property names use underscores instead of dots for telemetry compatibility

Implementation Details

Modified ProjectTelemetry.cs:

  • Added TrackTaskSubclassing(Type, bool) method that:
    • Walks the entire inheritance hierarchy to find Microsoft-owned base classes
    • Tracks all user-authored (not Microsoft-owned) subclasses
    • Stores counts in a dictionary mapping base task names to subclass usage
  • Telemetry is aggregated per project and logged when the project build completes

Modified AssemblyTaskFactory.cs:

  • Added IsMicrosoftAuthoredTask() helper that identifies Microsoft tasks by checking:
    • Assembly name starts with "Microsoft."
    • Assembly is from built-in MSBuild logic location
    • Assembly is a Microsoft package from NuGet cache
  • Calls TrackTaskSubclassing() for every task instantiation in CreateTaskInstance()
  • Passes whether the task is Microsoft-owned to enable proper filtering

New test file ProjectTelemetry_Tests.cs:

  • 6 comprehensive unit tests covering all scenarios
  • Tests tracking of both sealed and non-sealed subclasses, Microsoft-owned filtering
  • Integration test that verifies telemetry collection during actual builds

Example Output

For a build where users have created 2 custom tasks inheriting from Microsoft.Build.Utilities.Task:

TelemetryEvent: build/tasks/msbuild-subclassed
Properties:
  - Microsoft_Build_Utilities_Task: 2

If users also subclassed other Microsoft tasks:

TelemetryEvent: build/tasks/msbuild-subclassed
Properties:
  - Microsoft_Build_Utilities_Task: 2
  - Microsoft_Build_Tasks_Csc: 1

Performance Considerations

  • Minimal overhead: inspects every task but only stores data for subclasses
  • Uses a simple dictionary, no complex data structures
  • No additional reflection beyond what MSBuild already does
  • Telemetry logged once per project, not per task execution

Extensibility

The implementation walks the entire inheritance hierarchy, making it easy to:

  • Track subclasses of any Microsoft task at any depth
  • Add additional filtering criteria in the future
  • Extend to other task characteristics as needed

SDK Integration

Follows the existing telemetry pattern where the SDK's MSBuildLogger aggregates integer properties across builds and sends them to the telemetry backend. See: https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet/Commands/MSBuild/MSBuildLogger.cs

Testing

All tests pass:

  • ✅ 6 new ProjectTelemetry tests
  • ✅ 84 existing TaskExecutionHost tests
  • ✅ 26 existing AssemblyTaskFactory tests
  • ✅ Full MSBuild build succeeds
  • ✅ Sample projects build successfully

Fixes #10855

Original prompt

This section details on the original issue you should resolve

<issue_title>Collect telemetry about MSBuild- and SDK-delivered Tasks that are loaded during the build</issue_title>
<issue_description>### Summary

We want to mark our tasks sealed, but this would be an API breaking change. To justify this change we need data. So we want to collect data about MSBuild, SDK, and generally Microsoft-authored Task usage to drive this decision making.

Background and Motivation

Today tasks shipped by MSBuild and the SDK generally are not sealed. This means that folks consuming our NuGet packages can extend the Tasks and use them in their own code. This presents problems when we want to do things like enlighten our Tasks to support multithreading.

Proposed Feature

When loading Tasks for execution, we should collect metrics on the characteristics of that Task if that Task is shipped by MSBuild.

  • if the Task is sealed or not
  • if the Task inherits from Microsoft.Build.Utilities.Task (our helper base class)
  • (once the multithreaded APIs are in) if the task directly implements IMultiThreadedTask
  • (once the multithreaded APIs are in) if the task supports multithreading through the attribute-detection mechanism

In the future we may want expand the set of Task information gathered, and we may want to allow external MSBuild users (like the dotnet SDK) to add Tasks to the set of Tasks to collect this information about, so there should be some extensibility mechanism for both of these.

Alternative Designs

No response</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #12622

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

@dotnet-policy-service
Copy link
Contributor

Hello @@copilot, I noticed that you’re changing an .swr file or any file under src/Package/MSBuild.VSSetup.. Please make sure to validate this change by an experimental VS insertion. This is accomplished by pushing to an exp/* branch, which requires write permissions to this repo.

Copilot AI and others added 2 commits October 8, 2025 15:32
- Extended ProjectTelemetry to track Microsoft task characteristics
- Added AddMicrosoftTaskLoaded method to collect task metrics
- Track whether tasks are sealed
- Track whether tasks inherit from Microsoft.Build.Utilities.Task
- Collect telemetry in AssemblyTaskFactory when tasks are instantiated
- Added comprehensive unit tests for new functionality

Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Copilot AI changed the title [WIP] Collect telemetry about MSBuild and SDK tasks Collect telemetry about MSBuild- and SDK-delivered Tasks Oct 8, 2025
Copilot AI requested a review from baronfel October 8, 2025 15:46
…tasks

- Renamed event from build/tasks/microsoft to build/tasks/msbuild-subclassed
- Changed from counting all Microsoft tasks to tracking non-sealed subclasses
- Now stores dictionary mapping Microsoft task names to their subclass usage counts
- Only tracks user-authored non-sealed subclasses of Microsoft tasks
- Updated all tests to match new behavior

Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Copilot AI changed the title Collect telemetry about MSBuild- and SDK-delivered Tasks Track non-sealed subclasses of MSBuild tasks for telemetry Oct 8, 2025
Copilot AI requested a review from baronfel October 8, 2025 16:04
- Removed the check that excluded sealed tasks from tracking
- Now tracks both sealed and non-sealed subclasses of Microsoft tasks
- Updated tests to reflect this change
- Updated documentation to remove references to non-sealed filtering

Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Copilot AI changed the title Track non-sealed subclasses of MSBuild tasks for telemetry Track subclasses of MSBuild tasks for telemetry Oct 8, 2025
Copilot AI requested a review from baronfel October 8, 2025 16:14
@baronfel baronfel marked this pull request as ready for review October 8, 2025 16:15
Copilot AI review requested due to automatic review settings October 8, 2025 16:15
@baronfel baronfel requested a review from a team October 8, 2025 16:16
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements telemetry collection to track user-authored subclasses of Microsoft-owned MSBuild tasks. The purpose is to gather data about task inheritance patterns to inform decisions about making tasks sealed, addressing requirements from issue #10855.

Key changes:

  • Adds a new telemetry event build/tasks/msbuild-subclassed that tracks subclass usage counts
  • Implements task inheritance hierarchy analysis to identify Microsoft base classes
  • Provides comprehensive test coverage for the new telemetry functionality

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
src/Build/Instance/TaskFactories/AssemblyTaskFactory.cs Adds Microsoft task detection logic and integrates telemetry tracking into task instantiation
src/Build/BackEnd/Components/Logging/ProjectTelemetry.cs Implements core telemetry collection with inheritance hierarchy walking and data aggregation
src/Build.UnitTests/BackEnd/ProjectTelemetry_Tests.cs Provides comprehensive unit tests covering all telemetry scenarios and edge cases

@YuliiaKovalova YuliiaKovalova merged commit 5fbd693 into main Oct 9, 2025
10 checks passed
@YuliiaKovalova YuliiaKovalova deleted the copilot/collect-task-telemetry-data branch October 9, 2025 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Collect telemetry about MSBuild- and SDK-delivered Tasks that are loaded during the build

4 participants