feat: add additional tracing on websockets #13332

quinna-h · 2025-05-05T21:46:28Z

Add websocket tracing according to RFC

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2025-05-05T21:47:04Z

CODEOWNERS have been resolved as:

releasenotes/notes/asgi-websocket-tracing-support-0b785b921385dfeb.yaml  @DataDog/apm-python
tests/snapshots/tests.contrib.fastapi.test_fastapi.test_long_running_websocket_session.json  @DataDog/apm-python
tests/snapshots/tests.contrib.fastapi.test_fastapi.test_websocket_sampling_not_inherited.json  @DataDog/apm-python
tests/snapshots/tests.contrib.fastapi.test_fastapi.test_websocket_tracing_not_separate_traces.json  @DataDog/apm-python
ddtrace/contrib/internal/asgi/middleware.py                             @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/django/patch.py                                @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/fastapi/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/starlette/patch.py                             @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/fastapi/app.py                                            @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/fastapi/test_fastapi.py                                   @DataDog/apm-core-python @DataDog/apm-idm-python
tests/snapshots/tests.contrib.fastapi.test_fastapi.test_traced_websocket.json  @DataDog/apm-python

github-actions · 2025-05-05T22:07:14Z

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 289 ± 6 ms.

The average import time from base is: 290 ± 5 ms.

The import time difference between this PR and base is: -0.9 ± 0.2 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 1.294 ms (0.45%)

ddtrace 0.684 ms (0.24%)

ddtrace.internal._unpatched 0.031 ms (0.01%)

json 0.031 ms (0.01%)

json.decoder 0.031 ms (0.01%)

re 0.031 ms (0.01%)

enum 0.031 ms (0.01%)

types 0.031 ms (0.01%)

ddtrace.bootstrap.sitecustomize 0.610 ms (0.21%)

ddtrace.bootstrap.preload 0.610 ms (0.21%)

ddtrace.internal.remoteconfig.client 0.610 ms (0.21%)

pr-commenter · 2025-05-05T22:36:25Z

Benchmarks

Benchmark execution time: 2025-06-12 21:23:50

Comparing candidate commit cdae31c in PR branch quinna.halim/websockets-tracing-asgi with baseline commit b588b17 in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 463 metrics, 2 unstable metrics.

scenario:iastaspectsospath-ospathsplitext_aspect

🟥 execution_time [+605.977ns; +675.077ns] or [+13.434%; +14.965%]

quinna-h · 2025-05-06T14:06:38Z

ddtrace/contrib/internal/asgi/middleware.py

+                            ws_span.set_metric("_dd.dm.inherited", 1)
+                            parent_span = self.tracer.current_root_span()
+                            if parent_span is not None:
+                                ws_span.set_tag_str("_dd.dm.service", parent_span.service)


should be the handshake span's service and resource

datadog-datadog-prod-us1 · 2025-05-30T17:07:33Z

ddtrace/contrib/internal/asgi/middleware.py

+                                    ipaddress.ip_address(client_ip)  # validate ip address
+                                    span.set_tag_str("network.client.ip", client_ip)
+                                except ValueError:
+                                    pass


🔴 Code Quality Violation

silent exception (...read more)

Using the pass statement in an exception block ignores the exception. Exceptions should never be ignored. Instead, the user must add code to notify an exception occurred and attempt to handle it or recover from it.

The exception to this rule is the use of StopIteration or StopAsyncIteration when implementing a custom iterator (as those errors are used to acknowledge the end of a successful iteration).

datadog-datadog-prod-us1 · 2025-06-02T15:39:46Z

ddtrace/contrib/internal/asgi/middleware.py

+                                    ipaddress.ip_address(client_ip)  # validate ip address
+                                    span.set_tag_str("network.client.ip", client_ip)
+                                except ValueError:
+                                    pass


🔴 Code Quality Violation

silent exception (...read more)

Using the pass statement in an exception block ignores the exception. Exceptions should never be ignored. Instead, the user must add code to notify an exception occurred and attempt to handle it or recover from it.

The exception to this rule is the use of StopIteration or StopAsyncIteration when implementing a custom iterator (as those errors are used to acknowledge the end of a successful iteration).

ddtrace/contrib/internal/asgi/middleware.py

datadog-datadog-prod-us1 · 2025-06-04T15:27:23Z

ddtrace/contrib/internal/asgi/middleware.py

+                                    ipaddress.ip_address(client_ip)  # validate ip address
+                                    span.set_tag_str("network.client.ip", client_ip)
+                                except ValueError:
+                                    pass


🔴 Code Quality Violation

silent exception (...read more)

Using the pass statement in an exception block ignores the exception. Exceptions should never be ignored. Instead, the user must add code to notify an exception occurred and attempt to handle it or recover from it.

The exception to this rule is the use of StopIteration or StopAsyncIteration when implementing a custom iterator (as those errors are used to acknowledge the end of a successful iteration).

quinna-h · 2025-06-04T15:31:10Z

tests/snapshots/tests.contrib.fastapi.test_fastapi.test_traced_websocket.json

+    "resource": "websocket /ws",
+    "trace_id": 2,
+    "span_id": 1,
+    "parent_id": 0,
    "type": "web",
    "error": 0,
    "meta": {
      "_dd.base_service": "tests.contrib.fastapi",
      "_dd.p.dm": "-0",


should the fastapi.request span also have the sampling decision maker set here?

0 means default (unset). Why it differs from the value that has the websocket.receive above? If you want to set it either it has to be manually kept in the test either you can set a rule for it

@amarziali I manually set the sampling on the root span here: https://github.com/DataDog/dd-trace-py/pull/13332/files#diff-4b6de063e968d66cb554265c5ebd1d9caeb1d876dba16014255903f50d1fabddR262

You should do in the test and not in the instrumentation code. Otherwise you will change the sampling decision.

wip

tests/contrib/fastapi/test_fastapi.py

update sampling on root span wip wip: style

ddtrace/contrib/internal/asgi/middleware.py

ddtrace/contrib/internal/django/patch.py

change to False by default

…kets-tracing-asgi' into quinna.halim/websockets-tracing-asgi

update

wip

formatting

formatting

eb4773d

quinna-h changed the title ~~add tracing on websockets~~ add additional tracing on websockets May 5, 2025

quinna-h commented May 6, 2025

View reviewed changes

add config variables and sampling rules

2cb44a0

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from 47131b0 to 2cb44a0 Compare May 8, 2025 18:58

Merge branch 'main' into quinna.halim/websockets-tracing-asgi

9bfe36c

datadog-datadog-prod-us1 bot reviewed May 30, 2025

View reviewed changes

update to use correct resource name and apply sampling

fc2a026

datadog-datadog-prod-us1 bot reviewed Jun 2, 2025

View reviewed changes

quinna-h added 2 commits June 2, 2025 14:51

support separate traces and not-inherited sampling

3b99182

add configuration settings

a55d348

quinna-h changed the title ~~add additional tracing on websockets~~ feat: add increased tracing on websockets Jun 2, 2025

quinna-h changed the title ~~feat: add increased tracing on websockets~~ feat: add additional tracing on websockets Jun 2, 2025

quinna-h marked this pull request as ready for review June 2, 2025 20:49

quinna-h requested review from a team as code owners June 2, 2025 20:49

quinna-h requested review from wantsui, Yun-Kim and juanjux June 2, 2025 20:49

format snapshot

b3c5f96

amarziali reviewed Jun 3, 2025

View reviewed changes

ddtrace/contrib/internal/asgi/middleware.py Outdated Show resolved Hide resolved

amarziali reviewed Jun 3, 2025

View reviewed changes

ddtrace/contrib/internal/asgi/middleware.py Outdated Show resolved Hide resolved

quinna-h added 2 commits June 3, 2025 13:25

only set sampling on outgoing messages

ad9c0f1

update snapshot tests and sampling

669dde3

datadog-datadog-prod-us1 bot reviewed Jun 4, 2025

View reviewed changes

quinna-h commented Jun 4, 2025

View reviewed changes

rename test for separate traces

ccdb2bf

wip

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from d02b4a1 to ccdb2bf Compare June 4, 2025 17:14

quinna-h commented Jun 4, 2025

View reviewed changes

tests/contrib/fastapi/test_fastapi.py Outdated Show resolved Hide resolved

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from 16d0a8b to dee2fd3 Compare June 5, 2025 14:43

style

ac4d82e

update sampling on root span wip wip: style

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from dee2fd3 to ac4d82e Compare June 5, 2025 14:55

wantsui reviewed Jun 5, 2025

View reviewed changes

ddtrace/contrib/internal/asgi/middleware.py Show resolved Hide resolved

wantsui reviewed Jun 5, 2025

View reviewed changes

ddtrace/contrib/internal/django/patch.py Show resolved Hide resolved

quinna-h added 2 commits June 6, 2025 13:24

update sampling

507c497

add releasenote

f428bcb

change to False by default

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from a052ecf to f428bcb Compare June 6, 2025 18:00

quinna-h added 5 commits June 9, 2025 10:32

Merge branch 'main' into quinna.halim/websockets-tracing-asgi

452fde2

update tests

d52ef25

Merge remote-tracking branch 'refs/remotes/origin/quinna.halim/websoc…

bf27515

…kets-tracing-asgi' into quinna.halim/websockets-tracing-asgi

fix django tests

a6a9e46

update snapshots and inherit sampling

ebd86fb

update

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from 309f38d to ebd86fb Compare June 10, 2025 16:15

delete old snapshot

b4c016f

wip

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from e1f8525 to b4c016f Compare June 10, 2025 17:01

quinna-h requested a review from amarziali June 10, 2025 17:35

quinna-h added 3 commits June 12, 2025 16:10

active context before starting span on receive

c227c7f

Merge branch 'main' into quinna.halim/websockets-tracing-asgi

c700c4c

wip

cdae31c

formatting

quinna-h force-pushed the quinna.halim/websockets-tracing-asgi branch from 41fa395 to cdae31c Compare June 12, 2025 20:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add additional tracing on websockets #13332

feat: add additional tracing on websockets #13332

quinna-h commented May 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 5, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented May 5, 2025 •

edited

Loading

Uh oh!

quinna-h May 6, 2025

Uh oh!

datadog-datadog-prod-us1 bot May 30, 2025

Uh oh!

datadog-datadog-prod-us1 bot Jun 2, 2025

Uh oh!

Uh oh!

Uh oh!

datadog-datadog-prod-us1 bot Jun 4, 2025

Uh oh!

quinna-h Jun 4, 2025

Uh oh!

amarziali Jun 5, 2025

Uh oh!

quinna-h Jun 5, 2025

Uh oh!

amarziali Jun 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feat: add additional tracing on websockets #13332

Are you sure you want to change the base?

feat: add additional tracing on websockets #13332

Conversation

quinna-h commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Reviewer Checklist

Uh oh!

github-actions bot commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bootstrap import analysis

Summary

Import time breakdown

Uh oh!

pr-commenter bot commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

scenario:iastaspectsospath-ospathsplitext_aspect

Uh oh!

quinna-h May 6, 2025

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1 bot May 30, 2025

Choose a reason for hiding this comment

🔴 Code Quality Violation

Uh oh!

datadog-datadog-prod-us1 bot Jun 2, 2025

Choose a reason for hiding this comment

🔴 Code Quality Violation

Uh oh!

Uh oh!

Uh oh!

datadog-datadog-prod-us1 bot Jun 4, 2025

Choose a reason for hiding this comment

🔴 Code Quality Violation

Uh oh!

quinna-h Jun 4, 2025

Choose a reason for hiding this comment

Uh oh!

amarziali Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

quinna-h Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

amarziali Jun 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

quinna-h commented May 5, 2025 •

edited

Loading

github-actions bot commented May 5, 2025 •

edited

Loading

github-actions bot commented May 5, 2025 •

edited

Loading

pr-commenter bot commented May 5, 2025 •

edited

Loading