Skip to content

[Linux][EventPipe][UserEvents] Add user events eventpipe support #115265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Jun 18, 2025

Conversation

mdh1418
Copy link
Member

@mdh1418 mdh1418 commented May 3, 2025

Runtime support for dotnet/diagnostics#5454

This PR adds user_events support for native runtime events leveraging EventPipe infrastructure, following the IPC Protocol Specification for CollectTracing5 discussed in dotnet/diagnostics#5454.

A new User_events EventPipe Session type is added to the runtime and enabled through the CollectTracing5 IPC Command. The User_events EventPipe session deviates from the typical IPC Streaming session by writing events directly to Tracepoints configured through the initial IPC Message, instead of using the buffer manager.

In addition, an event filter can be configured for more precise control over which events are enabled for each IPC initialized EventPipe Session.

The changes in this PR are as follows:

  • Adds EventPipe structures to hold tracepoint configuration
  • Adds a new User_events EventPipe session type
  • Refactors the DiagnosticServer EventPipe deserializers for more modular+flexible parsing
  • Registers Tracepoints configured through the IPC Command with the file descriptor passed through the continuation stream & Unregisters tracepoints upon session disablement
  • Writes EventPipe Events according to the Tracepoint Format Version 1 outlined in the IPC Protocol Spec.
  • Adds an EventFilter to allow additional control over event enablement. 
  • Refactoring/Cleanup of preexisting eventpipe/diagnostic server logic

Work planned for follow-up PRs

  • Optimize sending Event metadata to only be sent on the first unique Event write per session
  • Integrating the User_events_data_fd into the IPCStream structures ( See comment ,comment, comment)
  • Optimizing Tracepoint retrieval + cache (See comment)
  • Upgrade EventPipeEvent Metadata to be built using NetTrace V6 spec (from V5)
  • Fix inconsistency in Metadata version from Process Info event (See comment)
  • Add Unit tests

Testing

Performed manual end-to-end testing with:

  • C program emitting bytes over the dotnet diagnostics socket following the IPC Protocol Spec + user_events_data fd over continuation stream
  • Validating the registered user_events tracepoints with sudo perf list user_events*
  • Enabled the registered tracepoints through echo 1 > /sys/kernel/tracing/events/user_events/<tracepoints>/enable
  • Verified the trace data via sudo cat /sys/kernel/tracing/trace
  • Tested with NativeAOT Compatibility
  • Tested StopSession command

@Copilot Copilot AI review requested due to automatic review settings May 3, 2025 02:46
@ghost ghost added the area-Tracing-coreclr label May 3, 2025
@mdh1418 mdh1418 marked this pull request as draft May 3, 2025 02:47
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces Linux user events support for EventPipe by adding a new session type (EP_SESSION_TYPE_USEREVENTS) and integrating tracepoint registration and event filtering for user events.

  • Added new overloads and structures (e.g. EventPipeTracepoint, EventPipeEventFilter, ProviderTracepointConfiguration) to support user events.
  • Updated session allocation, option validation, and event writing to incorporate handling of user events.
  • Fixed minor naming issues and adjusted IPC protocol definitions for consistency.

Reviewed Changes

Copilot reviewed 20 out of 22 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/native/eventpipe/ep.c Adds new overloads for provider config initialization and adjusts option checks for USEREVENTS.
src/native/eventpipe/ep-types.h Introduces new structures and getters for tracepoint and event filtering support.
src/native/eventpipe/ep-types-forward.h Adds new session type EP_SESSION_TYPE_USEREVENTS.
src/native/eventpipe/ep-session.h & .c Adds user events-related fields and functions (tracepoint registration, event writing).
src/native/eventpipe/ep-session-provider.h & .c Updates provider allocation and filtering to include event filtering and tracepoint configuration.
src/native/eventpipe/ep-provider.c Updates provider enable mask computation to factor in event filtering via event id.
src/native/eventpipe/ds-types.h Cleans up duplicate definitions for command IDs.
src/native/eventpipe/ds-server.c, ds-protocol.* Fixes naming inconsistencies and updates IPC protocol references.
src/native/eventpipe/ds-ipc* files Adds and updates support functions for receiving file descriptors over IPC.
Files not reviewed (2)
  • src/native/eventpipe/configure.cmake: Language not supported
  • src/native/eventpipe/ep-shared-config.h.in: Language not supported

@mdh1418
Copy link
Member Author

mdh1418 commented May 3, 2025

@noahfalk @beaubelgrave @brianrob This is still a WIP, but has everything from deserializing the IPC messages, to registering the tracepoints, and writes dummy data to the tracepoint.

CC: @lateralusX FYI, adding some eventpipe changes

@mdh1418
Copy link
Member Author

mdh1418 commented May 6, 2025

CC: @agocke

Copy link
Member

@noahfalk noahfalk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basic stuff all looks good. I commented on a few things that happened to catch my eye but I mostly avoided the details assuming that things are still changing. I'll take a look again once its more complete.

@mdh1418 mdh1418 force-pushed the add_user_events_eventpipe_support branch 3 times, most recently from 1c0b495 to 5c7c64a Compare May 29, 2025 17:30
@mdh1418 mdh1418 marked this pull request as ready for review May 29, 2025 17:30
@mdh1418
Copy link
Member Author

mdh1418 commented May 29, 2025

I still need to go over the user_events tracepoint registration/writing commits, and I'm trying to look through our types for a faster event_filter check, maybe umap or simdhash?

But the first 6 commits which handle the IPC stream deserialization and initializing the user_events eventpipe session are ready for review

@noahfalk @lateralusX

@noahfalk
Copy link
Member

noahfalk commented May 29, 2025

I'm trying to look through our types for a faster event_filter check, maybe umap or simdhash?

I'd suggest use whichever is simpler to code up, probably umap. Event filters seem unlikely to have more than 1000 elements and processing it once at session startup isn't particularly perf sensitive. At that scale even O(N^2) vector searching is fine, but assuming there is minimal work to swap vector to map I'd go for it just to eliminate the risk if the filters ever scaled up really big.

@mdh1418
Copy link
Member Author

mdh1418 commented May 30, 2025

After I finish revising the Tracepoint registration + Tracepoint write and existing feedback, will rebase for a cleaner commit history

@mdh1418 mdh1418 marked this pull request as draft May 30, 2025 20:02
mdh1418 and others added 10 commits May 31, 2025 20:11
Visual reorder of Diagnostics enum structs
Fix a couple naming typos
Spaces to tabs consistency
To support the new user_events-based eventpipe session, add
a more modular and flexible EventPipeProviderConfiguration deserializer.
The user_events_data file is required to register user_events tracepoints.
Require profilers to pass over the corresponding file descriptor with
SCM_RIGHTS in order to initialize the user_events EventPipe session.
@mdh1418 mdh1418 force-pushed the add_user_events_eventpipe_support branch from 24504c5 to d6648a2 Compare June 2, 2025 05:36
Fix eventpipe_collect_tracing_command_free_tracepoint_sets logic
Clarify len variable names
Consistently use -1 as invalid file descriptor
Update event_pipe to use ep_rt_utf16_to_utf8_string/free
Copy link
Member

@lateralusX lateralusX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Greate work!

@mdh1418 mdh1418 removed NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) labels Jun 18, 2025
@brianrob
Copy link
Member

I have not done a full review of this PR. Just wanted to voice my appreciation for you doing this work!

@mdh1418
Copy link
Member Author

mdh1418 commented Jun 18, 2025

/ba-g "HostActivation.Tests is #116520 and iOS/tvOS System.Runtime.Tests is #116714"

@mdh1418 mdh1418 merged commit ce67cef into dotnet:main Jun 18, 2025
156 of 169 checks passed
@mdh1418 mdh1418 deleted the add_user_events_eventpipe_support branch June 24, 2025 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants