[WIP] openai-java instrumentation #9959

ygree · 2025-11-13T20:25:51Z

What Does This Do

The openai-java 1.0+ instrumentation for completions, chat completions, embeddings, and responses.

Currently it only creates APM spans:

test_openai.py::TestOpenAiApm::test_chat_completion[java-test-ml-app-tcp-True] PASSED [ 12%]
test_openai.py::TestOpenAiApm::test_chat_completion[java-test-ml-app-tcp-False] PASSED [ 25%]
test_openai.py::TestOpenAiApm::test_completion[java-test-ml-app-tcp] PASSED [ 37%]
test_openai.py::TestOpenAiApm::test_embedding[java-test-ml-app-tcp] PASSED [ 50%]
test_openai.py::TestOpenAiApm::test_chat_completion_tool_call[java-test-ml-app-tcp-True] PASSED [ 62%]
test_openai.py::TestOpenAiApm::test_chat_completion_tool_call[java-test-ml-app-tcp-False] PASSED [ 75%]
test_openai.py::TestOpenAiApm::test_responses_create[java-test-ml-app-tcp-True] PASSED [ 87%]
test_openai.py::TestOpenAiApm::test_responses_create[java-test-ml-app-tcp-False] PASSED [100%]

TBD: llmobs specifics
TBD: telemetry

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

…stead of forcing it

…d "streamed async request completion test with withRawResponse"

Call decorateWithResponse from the wrappers

Rename span resources to be aligned with trace-py Add http.client resource assertion

Test case renaming

Reorder tests by synch, async

Fix Embeddings fixture for the latestDepTest when base64

datadog-datadog-prod-us1 · 2025-11-13T20:51:15Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 59.65% (+0.02%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 637f8e5 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

pr-commenter · 2025-11-13T22:44:05Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ygree/openai-java
git_commit_date	1763128431	1763168244
git_commit_sha	`b436de8`	`637f8e5`
release_version	1.56.0-SNAPSHOT~b436de808f	1.55.0-SNAPSHOT~637f8e5d72

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763170242	1763170242
ci_job_id	1237184535	1237184535
ci_pipeline_id	82565005	82565005
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-r9kbw6p3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-r9kbw6p3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 6 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:tracing:Remote Config	better [-48.292µs; -22.791µs] or [-7.621%; -3.597%]	598.124µs	633.666µs

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.095 s) : 0, 1095305
Total [baseline] (10.714 s) : 0, 10714023
Agent [candidate] (1.114 s) : 0, 1114081
Total [candidate] (10.827 s) : 0, 10826965
section appsec
Agent [baseline] (1.278 s) : 0, 1277687
Total [baseline] (11.052 s) : 0, 11051692
Agent [candidate] (1.285 s) : 0, 1284597
Total [candidate] (11.11 s) : 0, 11109918
section iast
Agent [baseline] (1.239 s) : 0, 1239016
Total [baseline] (11.183 s) : 0, 11183321
Agent [candidate] (1.238 s) : 0, 1238168
Total [candidate] (11.143 s) : 0, 11143263
section profiling
Agent [baseline] (1.226 s) : 0, 1225622
Total [baseline] (11.094 s) : 0, 11093990
Agent [candidate] (1.237 s) : 0, 1236864
Total [candidate] (11.146 s) : 0, 11145632

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.095 s	-
Agent	appsec	1.278 s	182.382 ms (16.7%)
Agent	iast	1.239 s	143.712 ms (13.1%)
Agent	profiling	1.226 s	130.318 ms (11.9%)
Total	tracing	10.714 s	-
Total	appsec	11.052 s	337.668 ms (3.2%)
Total	iast	11.183 s	469.298 ms (4.4%)
Total	profiling	11.094 s	379.967 ms (3.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.114 s	-
Agent	appsec	1.285 s	170.516 ms (15.3%)
Agent	iast	1.238 s	124.088 ms (11.1%)
Agent	profiling	1.237 s	122.783 ms (11.0%)
Total	tracing	10.827 s	-
Total	appsec	11.11 s	282.953 ms (2.6%)
Total	iast	11.143 s	316.298 ms (2.9%)
Total	profiling	11.146 s	318.667 ms (2.9%)

gantt
    title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.455 ms) : 0, 1455
crashtracking [candidate] (1.473 ms) : 0, 1473
BytebuddyAgent [baseline] (703.428 ms) : 0, 703428
BytebuddyAgent [candidate] (717.426 ms) : 0, 717426
GlobalTracer [baseline] (247.614 ms) : 0, 247614
GlobalTracer [candidate] (251.099 ms) : 0, 251099
AppSec [baseline] (31.988 ms) : 0, 31988
AppSec [candidate] (32.495 ms) : 0, 32495
Debugger [baseline] (63.705 ms) : 0, 63705
Debugger [candidate] (63.924 ms) : 0, 63924
Remote Config [baseline] (639.327 µs) : 0, 639
Remote Config [candidate] (628.243 µs) : 0, 628
Telemetry [baseline] (8.128 ms) : 0, 8128
Telemetry [candidate] (8.25 ms) : 0, 8250
Flare Poller [baseline] (3.603 ms) : 0, 3603
Flare Poller [candidate] (3.729 ms) : 0, 3729
section appsec
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (729.414 ms) : 0, 729414
BytebuddyAgent [candidate] (733.887 ms) : 0, 733887
GlobalTracer [baseline] (239.858 ms) : 0, 239858
GlobalTracer [candidate] (241.346 ms) : 0, 241346
IAST [baseline] (24.862 ms) : 0, 24862
IAST [candidate] (24.879 ms) : 0, 24879
AppSec [baseline] (173.635 ms) : 0, 173635
AppSec [candidate] (173.757 ms) : 0, 173757
Debugger [baseline] (60.807 ms) : 0, 60807
Debugger [candidate] (61.449 ms) : 0, 61449
Remote Config [baseline] (729.491 µs) : 0, 729
Remote Config [candidate] (675.551 µs) : 0, 676
Telemetry [baseline] (8.243 ms) : 0, 8243
Telemetry [candidate] (8.257 ms) : 0, 8257
Flare Poller [baseline] (3.767 ms) : 0, 3767
Flare Poller [candidate] (3.977 ms) : 0, 3977
section iast
crashtracking [baseline] (1.449 ms) : 0, 1449
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (829.91 ms) : 0, 829910
BytebuddyAgent [candidate] (832.526 ms) : 0, 832526
GlobalTracer [baseline] (238.294 ms) : 0, 238294
GlobalTracer [candidate] (237.133 ms) : 0, 237133
IAST [baseline] (31.752 ms) : 0, 31752
IAST [candidate] (33.884 ms) : 0, 33884
AppSec [baseline] (29.646 ms) : 0, 29646
AppSec [candidate] (26.519 ms) : 0, 26519
Debugger [baseline] (61.393 ms) : 0, 61393
Debugger [candidate] (60.147 ms) : 0, 60147
Remote Config [baseline] (565.218 µs) : 0, 565
Remote Config [candidate] (533.451 µs) : 0, 533
Telemetry [baseline] (7.679 ms) : 0, 7679
Telemetry [candidate] (7.658 ms) : 0, 7658
Flare Poller [baseline] (3.511 ms) : 0, 3511
Flare Poller [candidate] (3.475 ms) : 0, 3475
section profiling
crashtracking [baseline] (1.433 ms) : 0, 1433
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (728.026 ms) : 0, 728026
BytebuddyAgent [candidate] (737.715 ms) : 0, 737715
GlobalTracer [baseline] (221.654 ms) : 0, 221654
GlobalTracer [candidate] (223.791 ms) : 0, 223791
AppSec [baseline] (32.184 ms) : 0, 32184
AppSec [candidate] (32.356 ms) : 0, 32356
Debugger [baseline] (62.816 ms) : 0, 62816
Debugger [candidate] (62.564 ms) : 0, 62564
Remote Config [baseline] (654.481 µs) : 0, 654
Remote Config [candidate] (649.419 µs) : 0, 649
Telemetry [baseline] (7.965 ms) : 0, 7965
Telemetry [candidate] (7.925 ms) : 0, 7925
Flare Poller [baseline] (3.736 ms) : 0, 3736
Flare Poller [candidate] (3.832 ms) : 0, 3832
ProfilingAgent [baseline] (97.613 ms) : 0, 97613
ProfilingAgent [candidate] (96.537 ms) : 0, 96537
Profiling [baseline] (98.202 ms) : 0, 98202
Profiling [candidate] (97.138 ms) : 0, 97138

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.099 s) : 0, 1098980
Total [baseline] (8.817 s) : 0, 8816947
Agent [candidate] (1.099 s) : 0, 1099316
Total [candidate] (8.868 s) : 0, 8868318
section iast
Agent [baseline] (1.237 s) : 0, 1236672
Total [baseline] (9.512 s) : 0, 9511774
Agent [candidate] (1.239 s) : 0, 1238755
Total [candidate] (9.54 s) : 0, 9539906

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.099 s	-
Agent	iast	1.237 s	137.692 ms (12.5%)
Total	tracing	8.817 s	-
Total	iast	9.512 s	694.827 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.099 s	-
Agent	iast	1.239 s	139.439 ms (12.7%)
Total	tracing	8.868 s	-
Total	iast	9.54 s	671.588 ms (7.6%)

gantt
    title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.454 ms) : 0, 1454
crashtracking [candidate] (1.467 ms) : 0, 1467
BytebuddyAgent [baseline] (705.716 ms) : 0, 705716
BytebuddyAgent [candidate] (708.408 ms) : 0, 708408
GlobalTracer [baseline] (248.385 ms) : 0, 248385
GlobalTracer [candidate] (247.987 ms) : 0, 247987
AppSec [baseline] (32.519 ms) : 0, 32519
AppSec [candidate] (32.119 ms) : 0, 32119
Debugger [baseline] (63.568 ms) : 0, 63568
Debugger [candidate] (62.339 ms) : 0, 62339
Remote Config [baseline] (633.666 µs) : 0, 634
Remote Config [candidate] (598.124 µs) : 0, 598
Telemetry [baseline] (8.236 ms) : 0, 8236
Telemetry [candidate] (8.026 ms) : 0, 8026
Flare Poller [baseline] (3.699 ms) : 0, 3699
Flare Poller [candidate] (3.603 ms) : 0, 3603
section iast
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (829.316 ms) : 0, 829316
BytebuddyAgent [candidate] (832.588 ms) : 0, 832588
GlobalTracer [baseline] (237.904 ms) : 0, 237904
GlobalTracer [candidate] (237.117 ms) : 0, 237117
AppSec [baseline] (32.278 ms) : 0, 32278
AppSec [candidate] (32.472 ms) : 0, 32472
Debugger [baseline] (59.982 ms) : 0, 59982
Debugger [candidate] (60.225 ms) : 0, 60225
Remote Config [baseline] (541.924 µs) : 0, 542
Remote Config [candidate] (551.129 µs) : 0, 551
Telemetry [baseline] (7.673 ms) : 0, 7673
Telemetry [candidate] (7.726 ms) : 0, 7726
Flare Poller [baseline] (3.444 ms) : 0, 3444
Flare Poller [candidate] (3.437 ms) : 0, 3437
IAST [baseline] (29.165 ms) : 0, 29165
IAST [candidate] (28.23 ms) : 0, 28230

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ygree/openai-java
git_commit_date	1763128431	1763168244
git_commit_sha	`b436de8`	`637f8e5`
release_version	1.56.0-SNAPSHOT~b436de808f	1.55.0-SNAPSHOT~637f8e5d72

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1763170947	1763170947
ci_job_id	1237184536	1237184536
ci_pipeline_id	82565005	82565005
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-ehwz9ddl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-ehwz9ddl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 0 performance regressions! Performance is the same for 15 metrics, 18 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	better [-310.027µs; -151.083µs] or [-11.380%; -5.546%]	better [-751.359µs; -211.418µs] or [-9.593%; -2.699%]	unstable [-104.197op/s; +217.822op/s] or [-7.962%; +16.644%]	2.494ms	7.351ms	1365.562op/s	2.724ms	7.832ms	1308.750op/s
scenario:load:petclinic:appsec:high_load	better [-871.604µs; -405.150µs] or [-4.735%; -2.201%]	unsure [-1.616ms; -0.476ms] or [-5.344%; -1.575%]	unstable [-17.591op/s; +35.216op/s] or [-7.092%; +14.198%]	17.769ms	29.200ms	256.844op/s	18.407ms	30.246ms	248.031op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.197 ms) : 1186, 1209
.   : milestone, 1197,
iast (3.503 ms) : 3419, 3586
.   : milestone, 3503,
iast_FULL (5.993 ms) : 5932, 6054
.   : milestone, 5993,
iast_GLOBAL (3.679 ms) : 3624, 3733
.   : milestone, 3679,
profiling (2.113 ms) : 2093, 2133
.   : milestone, 2113,
tracing (1.824 ms) : 1810, 1839
.   : milestone, 1824,
section candidate
no_agent (1.188 ms) : 1176, 1199
.   : milestone, 1188,
iast (3.354 ms) : 3223, 3485
.   : milestone, 3354,
iast_FULL (5.981 ms) : 5920, 6042
.   : milestone, 5981,
iast_GLOBAL (3.731 ms) : 3676, 3786
.   : milestone, 3731,
profiling (2.335 ms) : 2312, 2358
.   : milestone, 2335,
tracing (1.783 ms) : 1768, 1797
.   : milestone, 1783,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.197 ms [1.186 ms, 1.209 ms]	-
iast	3.503 ms [3.419 ms, 3.586 ms]	2.305 ms (192.5%)
iast_FULL	5.993 ms [5.932 ms, 6.054 ms]	4.795 ms (400.5%)
iast_GLOBAL	3.679 ms [3.624 ms, 3.733 ms]	2.481 ms (207.2%)
profiling	2.113 ms [2.093 ms, 2.133 ms]	915.761 µs (76.5%)
tracing	1.824 ms [1.81 ms, 1.839 ms]	626.882 µs (52.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.188 ms [1.176 ms, 1.199 ms]	-
iast	3.354 ms [3.223 ms, 3.485 ms]	2.166 ms (182.4%)
iast_FULL	5.981 ms [5.92 ms, 6.042 ms]	4.793 ms (403.5%)
iast_GLOBAL	3.731 ms [3.676 ms, 3.786 ms]	2.543 ms (214.1%)
profiling	2.335 ms [2.312 ms, 2.358 ms]	1.147 ms (96.6%)
tracing	1.783 ms [1.768 ms, 1.797 ms]	594.904 µs (50.1%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.099 ms) : 16925, 17274
.   : milestone, 17099,
appsec (18.816 ms) : 18626, 19006
.   : milestone, 18816,
code_origins (17.739 ms) : 17563, 17916
.   : milestone, 17739,
iast (17.507 ms) : 17332, 17682
.   : milestone, 17507,
profiling (18.42 ms) : 18238, 18602
.   : milestone, 18420,
tracing (18.039 ms) : 17859, 18219
.   : milestone, 18039,
section candidate
no_agent (17.9 ms) : 17720, 18079
.   : milestone, 17900,
appsec (18.165 ms) : 17981, 18349
.   : milestone, 18165,
code_origins (18.214 ms) : 18032, 18396
.   : milestone, 18214,
iast (17.616 ms) : 17443, 17789
.   : milestone, 17616,
profiling (18.357 ms) : 18172, 18542
.   : milestone, 18357,
tracing (17.748 ms) : 17571, 17925
.   : milestone, 17748,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.099 ms [16.925 ms, 17.274 ms]	-
appsec	18.816 ms [18.626 ms, 19.006 ms]	1.716 ms (10.0%)
code_origins	17.739 ms [17.563 ms, 17.916 ms]	640.024 µs (3.7%)
iast	17.507 ms [17.332 ms, 17.682 ms]	407.35 µs (2.4%)
profiling	18.42 ms [18.238 ms, 18.602 ms]	1.32 ms (7.7%)
tracing	18.039 ms [17.859 ms, 18.219 ms]	939.972 µs (5.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.9 ms [17.72 ms, 18.079 ms]	-
appsec	18.165 ms [17.981 ms, 18.349 ms]	265.245 µs (1.5%)
code_origins	18.214 ms [18.032 ms, 18.396 ms]	314.45 µs (1.8%)
iast	17.616 ms [17.443 ms, 17.789 ms]	-283.922 µs (-1.6%)
profiling	18.357 ms [18.172 ms, 18.542 ms]	457.152 µs (2.6%)
tracing	17.748 ms [17.571 ms, 17.925 ms]	-151.434 µs (-0.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ygree/openai-java
git_commit_date	1763128431	1763168244
git_commit_sha	`b436de8`	`637f8e5`
release_version	1.56.0-SNAPSHOT~b436de808f	1.55.0-SNAPSHOT~637f8e5d72

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1763170349	1763170349
ci_job_id	1237184537	1237184537
ci_pipeline_id	82565005	82565005
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-7y6p4jq8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-7y6p4jq8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.988 s) : 14988000, 14988000
.   : milestone, 14988000,
appsec (14.831 s) : 14831000, 14831000
.   : milestone, 14831000,
iast (18.61 s) : 18610000, 18610000
.   : milestone, 18610000,
iast_GLOBAL (17.903 s) : 17903000, 17903000
.   : milestone, 17903000,
profiling (14.825 s) : 14825000, 14825000
.   : milestone, 14825000,
tracing (14.49 s) : 14490000, 14490000
.   : milestone, 14490000,
section candidate
no_agent (15.548 s) : 15548000, 15548000
.   : milestone, 15548000,
appsec (14.793 s) : 14793000, 14793000
.   : milestone, 14793000,
iast (18.673 s) : 18673000, 18673000
.   : milestone, 18673000,
iast_GLOBAL (17.794 s) : 17794000, 17794000
.   : milestone, 17794000,
profiling (14.857 s) : 14857000, 14857000
.   : milestone, 14857000,
tracing (14.685 s) : 14685000, 14685000
.   : milestone, 14685000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.988 s [14.988 s, 14.988 s]	-
appsec	14.831 s [14.831 s, 14.831 s]	-157.0 ms (-1.0%)
iast	18.61 s [18.61 s, 18.61 s]	3.622 s (24.2%)
iast_GLOBAL	17.903 s [17.903 s, 17.903 s]	2.915 s (19.4%)
profiling	14.825 s [14.825 s, 14.825 s]	-163.0 ms (-1.1%)
tracing	14.49 s [14.49 s, 14.49 s]	-498.0 ms (-3.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.548 s [15.548 s, 15.548 s]	-
appsec	14.793 s [14.793 s, 14.793 s]	-755.0 ms (-4.9%)
iast	18.673 s [18.673 s, 18.673 s]	3.125 s (20.1%)
iast_GLOBAL	17.794 s [17.794 s, 17.794 s]	2.246 s (14.4%)
profiling	14.857 s [14.857 s, 14.857 s]	-691.0 ms (-4.4%)
tracing	14.685 s [14.685 s, 14.685 s]	-863.0 ms (-5.6%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~637f8e5d72, baseline=1.56.0-SNAPSHOT~b436de808f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (2.466 ms) : 2415, 2517
.   : milestone, 2466,
iast (2.216 ms) : 2152, 2280
.   : milestone, 2216,
iast_GLOBAL (2.256 ms) : 2192, 2320
.   : milestone, 2256,
profiling (2.075 ms) : 2022, 2128
.   : milestone, 2075,
tracing (2.032 ms) : 1982, 2081
.   : milestone, 2032,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (3.628 ms) : 3414, 3843
.   : milestone, 3628,
iast (2.214 ms) : 2150, 2278
.   : milestone, 2214,
iast_GLOBAL (2.261 ms) : 2197, 2326
.   : milestone, 2261,
profiling (2.084 ms) : 2030, 2138
.   : milestone, 2084,
tracing (2.024 ms) : 1975, 2074
.   : milestone, 2024,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.474 ms [1.463 ms, 1.486 ms]	-
appsec	2.466 ms [2.415 ms, 2.517 ms]	991.837 µs (67.3%)
iast	2.216 ms [2.152 ms, 2.28 ms]	742.205 µs (50.4%)
iast_GLOBAL	2.256 ms [2.192 ms, 2.32 ms]	782.211 µs (53.1%)
profiling	2.075 ms [2.022 ms, 2.128 ms]	600.968 µs (40.8%)
tracing	2.032 ms [1.982 ms, 2.081 ms]	557.682 µs (37.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	3.628 ms [3.414 ms, 3.843 ms]	2.156 ms (146.5%)
iast	2.214 ms [2.15 ms, 2.278 ms]	741.538 µs (50.4%)
iast_GLOBAL	2.261 ms [2.197 ms, 2.326 ms]	789.219 µs (53.6%)
profiling	2.084 ms [2.03 ms, 2.138 ms]	611.696 µs (41.5%)
tracing	2.024 ms [1.975 ms, 2.074 ms]	551.929 µs (37.5%)

…Support openai-java v3+

test_openai.py::TestOpenAiApm PASS

ygree added 30 commits October 24, 2025 18:03

Initial APM-only openai-java instrumentation with a unit test.

e7bd8d9

Start llmobs system in tests and create llmobs span.

6feb61e

streamed request completion test

d541adb

README

c209bd0

Instrument sync streamed completion

a82856d

Mock Streamed Completion in Tests

3d3def9

Add failing test for Async Completion

3542b9e

Async Single Completion. Simplify tests. Remove println. Cleanup.

c12cd0a

Extract DDHttpResponseFor to intercept when the response is parsed in…

2af4e5f

…stead of forcing it

Instrument and test Async Stream Completion

cec33af

More tests "streamed request completion test with withRawResponse" an…

b2b4141

…d "streamed async request completion test with withRawResponse"

Fix muzzle check

b1b6db2

Add openai APM tags, assert tags, todo tags

baa687d

Wrap HttpResponseFor instead of forcing parsing. Add TODOs

b9ad963

Set response model tag

5bf16f4

Set "openai.organization.name"

be20fa1

Call decorateWithResponse from the wrappers

Set ratelimit metrics

dd230b7

api_base

c3f94b0

"openai.request.method" & "openai.request.endpoint"

ddf0e09

createChatCompletion instrumentation

fb9fd04

Reorder tests single then stream

a4358a8

Async Single Chat Completion

99dedfe

Async Streamed Chat Completion

d66a5ee

Rename assertChatCompletionTrace

f8af3ae

Instrument Embeddings

846e945

Rename span resources to be aligned with trace-py Add http.client resource assertion

ResponseService WIP

b14bc4b

Test case renaming

ResponseService synch

e8daf08

Reorder tests by synch, async

ResponseServiceAsyncInstrumentation

c8a9e6e

Fix Embeddings fixture for the latestDepTest when base64

Setup httpClient for tests WIP

9754cd8

Intercept Http req/resp with TestOpenAiHttpClient

aeaae2a

Implement req/resp recorder and mock backend for tests

7e50d1d

ygree self-assigned this Nov 13, 2025

ygree added tag: do not merge Do not merge changes inst: others All other instrumentations labels Nov 13, 2025

Add lockfile

8e85447

Minor changes to the records format

ea2f465

ygree added 14 commits November 13, 2025 15:05

OpenAiHttpClient for tests. Record only if the record doesn't exist. …

1f3aa6c

…Support openai-java v3+

Rename TestOpenAiHttpClient

0c28997

Do not dump a record if already exists

6a41720

Fix format

6050515

Fix linter errors

c8a1e94

Fix unused imports

20c88d7

Fix format

001ce0e

Fix format

96f4353

Fix format

46c2d3a

Merge branch 'master' into ygree/openai-java

29ae26c

Fix format

8de5108

Remove unexisting helper class that failed the test

fb74ed2

Extract response model.

9a2bc61

test_openai.py::TestOpenAiApm PASS

Extract response model.

637f8e5

test_openai.py::TestOpenAiApm PASS

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] openai-java instrumentation #9959

[WIP] openai-java instrumentation #9959

Uh oh!

ygree commented Nov 13, 2025 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Nov 13, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[WIP] openai-java instrumentation #9959

Are you sure you want to change the base?

[WIP] openai-java instrumentation #9959

Uh oh!

Conversation

ygree commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ygree commented Nov 13, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Nov 13, 2025 •

edited

Loading

pr-commenter bot commented Nov 13, 2025 •

edited

Loading