Skip to content

diagnostics_channel: add USDT probes - #62118

Open
bengl wants to merge 2 commits into
nodejs:mainfrom
bengl:bengl/usdt-dc
Open

bengl wants to merge 2 commits into
nodejs:mainfrom
bengl:bengl/usdt-dc

Conversation

@bengl

@bengl bengl commented Mar 5, 2026 •

Copy link
Copy Markdown
Member

Adds a dc__publish USDT probe for diagnostics channel publish events.

Attaching a tracer does not activate inactive channels or bypass subscriber checks in publishers.

Includes Linux CI coverage with and without USDT, plus a bpftrace snapshot-restore regression test. Both CI builds assert the expected USDT support.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/gyp

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Mar 5, 2026
Comment thread doc/api/diagnostics_channel.md Outdated

#### Platform support

At `./configure` time, Node.js checks for a working `dtrace` tool and

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have removed dtrace support long time ago. Wouldn't it require us to include a new suite for testing it on that environment?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what our build setup looks like now, but yes, everything that was required for the previous incarnation of probes in Node.js would likely be required again with this PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we tested the dtrace stuff previously. I think when it was removed it had be broken for a while (or maybe that was one of the other non-tested removed features).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a possible path forward here where headers are pre-generated and added to git (much like we do for other cases of generated headers, in dependencies for example), and then a userland, self-contained test (at least on linux) checks for the placement and activation of the probe.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I've made some changes (many months later..)

Linux is now default-on with no dtrace build dependency. src/node_provider_linux.h is committed.
It can be regenerated via tools/usdt/generate_headers.py and a --check drift job runs in CI against it.
Default ./configure enables USDT whenever <sys/sdt.h> is present (systemtap-sdt-dev / systemtap-sdt-devel is the only build requirement on Linux).

macOS is opt-in via ./configure --with-dtrace (which at build-time calls dtrace -h -xnolibs).
--without-dtrace still disables everything.

New test-usdt CI job in test-linux.yml:

  • default: end-to-end bpftrace test as root (verifies the probe fires with the channel name) plus the committed-header drift check
  • --without-dtrace: pins the no-op tier

Bench (aarch64 VM, benchmark/diagnostics_channel/publish.js, subscribers=1, avg of 2): default ~282M ops/s vs --without-dtrace ~314M ops/s — ~10% on this microbench.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

This pull request has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

@github-actions github-actions Bot added the stale Issues and PRs marked stale due to inactivity and scheduled for automatic closure. label Aug 3, 2026
bengl added a commit to bengl/node that referenced this pull request Sep 9, 2026
Implements the approach from the nodejs#62118 review discussion to remove
the build-time 'dtrace' dependency on Linux:

* Commit the SystemTap-generated probe header
  (src/node_provider_linux.h, regenerate with
  tools/usdt/generate_headers.py).  USDT support is now on by
  default on Linux whenever <sys/sdt.h> is available
  (systemtap-sdt-dev on Debian/Ubuntu, systemtap-sdt-devel on
  Fedora/RHEL) and never needs a 'dtrace' tool at build time.  A
  committed-header drift check runs in CI.
* Make native DTrace opt-in on macOS via the new
  ./configure --with-dtrace; FreeBSD/illumos remain unsupported
  pending a 'dtrace -G' link step.  --without-dtrace still disables
  probes everywhere.  The always-on <sys/sdt.h> fallback tier is
  gone.
* Add a path-gated test-usdt job to the Linux CI workflow with a
  default leg that runs the end-to-end bpftrace probe test as
  root, and a --without-dtrace leg that pins the no-op tier.

The generated header is excluded from cpplint like
src/node_root_certs.h.
bengl added a commit to bengl/node that referenced this pull request Sep 9, 2026
Implements the approach from the nodejs#62118 review discussion to remove
the build-time 'dtrace' dependency on Linux:

* Commit the SystemTap-generated probe header
  (src/node_provider_linux.h, regenerate with
  tools/usdt/generate_headers.py).  USDT support is now on by
  default on Linux whenever <sys/sdt.h> is available
  (systemtap-sdt-dev on Debian/Ubuntu, systemtap-sdt-devel on
  Fedora/RHEL) and never needs a 'dtrace' tool at build time.  A
  committed-header drift check runs in CI.
* Make native DTrace opt-in on macOS via the new
  ./configure --with-dtrace; FreeBSD/illumos remain unsupported
  pending a 'dtrace -G' link step.  --without-dtrace still disables
  probes everywhere.  The always-on <sys/sdt.h> fallback tier is
  gone.
* Add a path-gated test-usdt job to the Linux CI workflow with a
  default leg that runs the end-to-end bpftrace probe test as
  root, and a --without-dtrace leg that pins the no-op tier.

The generated header is excluded from cpplint like
src/node_root_certs.h.

Signed-off-by: Bryan English <bryan@bryanenglish.com>
@github-actions github-actions Bot removed the stale Issues and PRs marked stale due to inactivity and scheduled for automatic closure. label Sep 10, 2026
@bengl
bengl marked this pull request as ready for review September 10, 2026 20:56
Comment thread src/node_usdt.h
Comment thread src/node_diagnostics_channel.cc
@codecov

codecov Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 46.15385% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.43%. Comparing base (4bf4c00) to head (8ada985).
⚠️ Report is 596 commits behind head on main.

Files with missing lines Patch % Lines
src/node_diagnostics_channel.cc 6.66% 10 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #62118      +/-   ##
==========================================
+ Coverage   90.17%   90.43%   +0.26%     
==========================================
  Files         771      791      +20     
  Lines      265496   276504   +11008     
  Branches    50483    53094    +2611     
==========================================
+ Hits       239401   250046   +10645     
+ Misses      17052    16858     -194     
- Partials     9043     9600     +557     
Files with missing lines Coverage Δ
lib/diagnostics_channel.js 97.66% <100.00%> (+0.31%) ⬆️
src/node_diagnostics_channel.h 71.42% <ø> (+14.28%) ⬆️
src/node_diagnostics_channel.cc 80.64% <6.66%> (-3.01%) ⬇️

... and 312 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bengl
bengl force-pushed the bengl/usdt-dc branch 2 times, most recently from 5a684c8 to 6645936 Compare September 11, 2026 13:20
@bengl bengl added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 23, 2026
@panva panva added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 24, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Sep 24, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@panva panva added the resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. label Sep 24, 2026
@github-actions github-actions Bot added resume-ci-failed Resuming CI with the resume-ci label failed and requires manual intervention. and removed resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. labels Sep 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Failed to resume CI

✖ CI job 77839 is in status RUNNING, skipping resume

Full Auto Start CI output
�[36m⠋�[39m Validating Jenkins credentials
�[36m⠋�[39m Validating Jenkins credentials
✔  Jenkins credentials valid
�[36m⠙�[39m Looking for CI runs for pull request 62118
�[36m⠙�[39m Looking for CI runs for pull request 62118
�[36m⠙�[39m Getting PR from nodejs/node/pull/62118
�[36m⠙�[39m Getting reviews from nodejs/node/pull/62118
�[36m⠙�[39m Getting comments from nodejs/node/pull/62118
✔  Found PR CI job 77839
�[36m⠹�[39m Querying data for job/node-test-pull-request/77839/
�[36m⠹�[39m Querying data for job/node-test-pull-request/77839/
�[36m⠹�[39m Querying API for job/node-test-pull-request/77839/
✔  Build data downloaded
   ✖  CI job 77839 is in status RUNNING, skipping resume

View workflow run

@panva panva added resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. and removed resume-ci-failed Resuming CI with the resume-ci label failed and requires manual intervention. labels Sep 24, 2026
@panva panva removed the resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. label Sep 26, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment thread lib/diagnostics_channel.js Outdated
Comment thread src/node_diagnostics_channel.cc Outdated
Comment thread doc/api/diagnostics_channel.md Outdated
Comment thread .github/workflows/test-linux.yml Outdated
Comment thread .github/workflows/test-linux.yml Outdated
@panva panva removed the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Sep 28, 2026
@bengl
bengl force-pushed the bengl/usdt-dc branch 2 times, most recently from 40f7683 to 591cf2f Compare October 6, 2026 13:12
@bengl

bengl commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

@panva I think your comments are addressed now. PTAL.

@bengl bengl added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 6, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 6, 2026
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@bengl bengl added the author ready PRs with CI started, the required approvals, and no outstanding review comments. label Oct 7, 2026
Comment on lines 20 to 23

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a platform assert to ensure that probeSemaphore is not undefined on Linux, where the support is enabled by default?

This ensures that the test does run on Linux CI.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I think the GitHub UI is broken for my comment. I mean something like this:

// test/common/usdt.js

function skipIfUsdtIsDisabled() {
  if (!process.config.variables.node_no_usdt && process.config.variables.node_use_dtrace) {
    return;
  }
  common.skip("USDT is not enabled);
}

and in this test:

skipIfUsdtIsDisabled();

assert.ok(
    probeSemaphore instanceof Uint16Array,
    `Expected probeSemaphore to be Uint16Array, got ${typeof probeSemaphore}`,
  );
assert.ok(
      probeSemaphore[0] === 0 || probeSemaphore[0] === 1,
      `Expected semaphore to be 0 or 1, got ${probeSemaphore[0]}`,
    );

@legendecas legendecas Oct 7, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, it would be helpful to add a test condition in test/common/index.js (or a new test/common/usdt.js as this accesses an internal binding) to ensure that the test is determined to be enabled when the USDT support is supposed to be enabled.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread src/node_diagnostics_channel.cc Outdated

@Qard Qard left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to the comments from @legendecas, but also, it sounds like JS subscribers are needed for the publishes to actually fire the probes? This appears to not be the case for the native-side publishes. This seems a bit unexpected to me. Is there some way we could read the semaphore or the bit the kernel modifies to detect liveness and use that to do the mode-swap on the channel somehow so it becomes active if a probe is attached?

Also, how exactly would the message object be consumed? It seems to be getting passed, but as just a void* which I'm not sure if there's any useful way to consume that externally? Is that actually useful?

bengl added 2 commits October 7, 2026 14:21
Fire a `dc__publish` USDT probe for every diagnostics_channel
publish, passing the channel name, so tracers such as bpftrace,
perf, or SystemTap can observe publish traffic with near-zero cost
when nothing is attached. Probes are enabled by default on Linux;
--without-dtrace disables them.

The probe semaphore is exposed to JS as a Uint16Array over the
native semaphore so the publish hot path can gate on one indexed
load instead of a binding call. The view is read through the
binding on every publish: it is created once per context, and any
cached reference (for example one captured while building a
startup snapshot) would be a stale copy after the snapshot is
deserialized, leaving probes disabled even while a tracer is
attached. On builds without USDT support the property is absent
and the hot path stays branch-only.

Signed-off-by: Bryan English <bryan@bryanenglish.com>
Assisted-by: Pi using GLM-5.3
Remove the build-time dtrace dependency on Linux, following the
approach discussed in the PR review: generate the SystemTap
provider header once with tools/usdt/generate_headers.py and commit
it as src/node_provider_linux.h, so Linux builds need only
<sys/sdt.h> and never a dtrace tool. --with-dtrace becomes the
opt-in switch for macOS; --without-dtrace disables probes entirely.

Add a path-filtered test-usdt job to the Linux CI workflow that
installs bpftrace and systemtap-sdt-dev, verifies the committed
header stays in sync with the generator, builds with and without
dtrace, and runs the USDT tests, including a root-only bpftrace
end-to-end test that attaches to the dc__publish probe and asserts
the channel name argument.

Signed-off-by: Bryan English <bryan@bryanenglish.com>
Assisted-by: Pi using GLM-5.3
@legendecas legendecas added the diagnostics_channel Issues and PRs related to the diagnostics_channel module. label Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs with CI started, the required approvals, and no outstanding review comments. diagnostics_channel Issues and PRs related to the diagnostics_channel module. lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants