Skip to content

Fix CPU overhead in LLM Obs eval processor #9765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Oct 17, 2025
Merged

Fix CPU overhead in LLM Obs eval processor #9765

merged 2 commits into from
Oct 17, 2025

Conversation

@XG-xin
Copy link
Contributor

@XG-xin XG-xin commented Oct 15, 2025 •
edited
Loading

What Does This Do

Enabling LLM observability on an application will result in CPU overhead when the evaluation metrics processor thread is idle.
This PR polls the queue with a timeout to prevent constant attempts to drain the queue and flush.

Testing:

  1. Run an application that doesn't submit llmobs evaluation metrics on a docker container.

Before: the CPU usage was above 100%:
image

After: it decreased to 8%, then dropped further to around 2%.
image

  1. Run an application that submits LLM evaluation metrics locally on a Mac machine.

The CPU usage is around 8.6% after this new implementation.
image

Motivation

Solves MLOS-268

Additional Notes

Contributor Checklist

Jira ticket: MLOS-268

@datadog-datadog-prod-us1
Copy link
Contributor

datadog-datadog-prod-us1 bot commented Oct 15, 2025 •
edited by datadog-official bot
Loading

🎯 Code Coverage
• Patch Coverage: 0.00%
• Total Coverage: 59.80% (-0.03%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8347f7f | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Oct 15, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master xinyuan/handle-empty-llm-eval-metric
git_commit_date 1760639037 1760645292
git_commit_sha 33e27c7 8347f7f
release_version 1.55.0-SNAPSHOT~33e27c7cea 1.55.0-SNAPSHOT~8347f7f8ff
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1760647221 1760647221
ci_job_id 1183896506 1183896506
ci_pipeline_id 79562452 79562452
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-7xhmfede 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-7xhmfede 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:profiling:Agent worse
[+243.966ms; +650.008ms] or [+2.262%; +6.026%]		 11.233s 10.786s
scenario:startup:petclinic:tracing:Remote Config worse
[+27.887µs; +60.195µs] or [+4.161%; +8.982%]		 714.217µs 670.176µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.017 s) : 0, 1016512
Total [baseline] (10.77 s) : 0, 10770342
Agent [candidate] (1.017 s) : 0, 1017350
Total [candidate] (10.722 s) : 0, 10721871
section appsec
Agent [baseline] (1.204 s) : 0, 1204056
Total [baseline] (10.91 s) : 0, 10909804
Agent [candidate] (1.194 s) : 0, 1194202
Total [candidate] (11.023 s) : 0, 11023241
section iast
Agent [baseline] (1.16 s) : 0, 1159553
Total [baseline] (11.126 s) : 0, 11126419
Agent [candidate] (1.148 s) : 0, 1148037
Total [candidate] (10.955 s) : 0, 10955130
section profiling
Agent [baseline] (1.161 s) : 0, 1161309
Total [baseline] (10.786 s) : 0, 10786137
Agent [candidate] (1.173 s) : 0, 1173280
Total [candidate] (11.233 s) : 0, 11233124
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.017 s -
Agent appsec 1.204 s 187.544 ms (18.4%)
Agent iast 1.16 s 143.041 ms (14.1%)
Agent profiling 1.161 s 144.797 ms (14.2%)
Total tracing 10.77 s -
Total appsec 10.91 s 139.461 ms (1.3%)
Total iast 11.126 s 356.076 ms (3.3%)
Total profiling 10.786 s 15.795 ms (0.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.017 s -
Agent appsec 1.194 s 176.852 ms (17.4%)
Agent iast 1.148 s 130.687 ms (12.8%)
Agent profiling 1.173 s 155.93 ms (15.3%)
Total tracing 10.722 s -
Total appsec 11.023 s 301.37 ms (2.8%)
Total iast 10.955 s 233.259 ms (2.2%)
Total profiling 11.233 s 511.253 ms (4.8%) 
gantt
    title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.493 ms) : 0, 1493
crashtracking [candidate] (1.482 ms) : 0, 1482
BytebuddyAgent [baseline] (692.498 ms) : 0, 692498
BytebuddyAgent [candidate] (693.906 ms) : 0, 693906
GlobalTracer [baseline] (242.911 ms) : 0, 242911
GlobalTracer [candidate] (242.043 ms) : 0, 242043
AppSec [baseline] (32.533 ms) : 0, 32533
AppSec [candidate] (32.217 ms) : 0, 32217
Debugger [baseline] (6.383 ms) : 0, 6383
Debugger [candidate] (6.485 ms) : 0, 6485
Remote Config [baseline] (670.176 µs) : 0, 670
Remote Config [candidate] (714.217 µs) : 0, 714
Telemetry [baseline] (9.415 ms) : 0, 9415
Telemetry [candidate] (9.326 ms) : 0, 9326
Flare Poller [baseline] (9.49 ms) : 0, 9490
Flare Poller [candidate] (9.987 ms) : 0, 9987
section appsec
crashtracking [baseline] (1.48 ms) : 0, 1480
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (724.043 ms) : 0, 724043
BytebuddyAgent [candidate] (717.97 ms) : 0, 717970
GlobalTracer [baseline] (236.877 ms) : 0, 236877
GlobalTracer [candidate] (234.786 ms) : 0, 234786
AppSec [baseline] (175.934 ms) : 0, 175934
AppSec [candidate] (175.063 ms) : 0, 175063
Debugger [baseline] (6.116 ms) : 0, 6116
Debugger [candidate] (6.064 ms) : 0, 6064
Remote Config [baseline] (634.009 µs) : 0, 634
Remote Config [candidate] (630.577 µs) : 0, 631
Telemetry [baseline] (8.548 ms) : 0, 8548
Telemetry [candidate] (8.513 ms) : 0, 8513
Flare Poller [baseline] (4.034 ms) : 0, 4034
Flare Poller [candidate] (3.915 ms) : 0, 3915
IAST [baseline] (25.08 ms) : 0, 25080
IAST [candidate] (24.669 ms) : 0, 24669
section iast
crashtracking [baseline] (1.495 ms) : 0, 1495
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (820.879 ms) : 0, 820879
BytebuddyAgent [candidate] (813.394 ms) : 0, 813394
GlobalTracer [baseline] (233.804 ms) : 0, 233804
GlobalTracer [candidate] (230.819 ms) : 0, 230819
AppSec [baseline] (35.116 ms) : 0, 35116
AppSec [candidate] (34.94 ms) : 0, 34940
Debugger [baseline] (6.172 ms) : 0, 6172
Debugger [candidate] (6.108 ms) : 0, 6108
Remote Config [baseline] (622.548 µs) : 0, 623
Remote Config [candidate] (612.878 µs) : 0, 613
Telemetry [baseline] (8.722 ms) : 0, 8722
Telemetry [candidate] (8.764 ms) : 0, 8764
Flare Poller [baseline] (4.234 ms) : 0, 4234
Flare Poller [candidate] (4.193 ms) : 0, 4193
IAST [baseline] (26.851 ms) : 0, 26851
IAST [candidate] (26.275 ms) : 0, 26275
section profiling
crashtracking [baseline] (1.422 ms) : 0, 1422
crashtracking [candidate] (1.437 ms) : 0, 1437
BytebuddyAgent [baseline] (719.714 ms) : 0, 719714
BytebuddyAgent [candidate] (726.845 ms) : 0, 726845
GlobalTracer [baseline] (218.378 ms) : 0, 218378
GlobalTracer [candidate] (220.351 ms) : 0, 220351
AppSec [baseline] (32.331 ms) : 0, 32331
AppSec [candidate] (32.983 ms) : 0, 32983
Debugger [baseline] (7.478 ms) : 0, 7478
Debugger [candidate] (8.276 ms) : 0, 8276
Remote Config [baseline] (724.357 µs) : 0, 724
Remote Config [candidate] (784.561 µs) : 0, 785
Telemetry [baseline] (15.182 ms) : 0, 15182
Telemetry [candidate] (14.545 ms) : 0, 14545
Flare Poller [baseline] (4.171 ms) : 0, 4171
Flare Poller [candidate] (4.178 ms) : 0, 4178
ProfilingAgent [baseline] (108.579 ms) : 0, 108579
ProfilingAgent [candidate] (110.407 ms) : 0, 110407
Profiling [baseline] (109.746 ms) : 0, 109746
Profiling [candidate] (111.426 ms) : 0, 111426
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.031 s) : 0, 1031496
Total [baseline] (8.697 s) : 0, 8697434
Agent [candidate] (1.027 s) : 0, 1027332
Total [candidate] (8.715 s) : 0, 8715178
section iast
Agent [baseline] (1.155 s) : 0, 1155333
Total [baseline] (9.303 s) : 0, 9303016
Agent [candidate] (1.149 s) : 0, 1149045
Total [candidate] (9.331 s) : 0, 9330776
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.031 s -
Agent iast 1.155 s 123.837 ms (12.0%)
Total tracing 8.697 s -
Total iast 9.303 s 605.582 ms (7.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent iast 1.149 s 121.714 ms (11.8%)
Total tracing 8.715 s -
Total iast 9.331 s 615.598 ms (7.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.496 ms) : 0, 1496
crashtracking [candidate] (1.492 ms) : 0, 1492
BytebuddyAgent [baseline] (703.19 ms) : 0, 703190
BytebuddyAgent [candidate] (700.034 ms) : 0, 700034
GlobalTracer [baseline] (244.696 ms) : 0, 244696
GlobalTracer [candidate] (243.481 ms) : 0, 243481
AppSec [baseline] (32.952 ms) : 0, 32952
AppSec [candidate] (32.743 ms) : 0, 32743
Debugger [baseline] (6.498 ms) : 0, 6498
Debugger [candidate] (6.5 ms) : 0, 6500
Remote Config [baseline] (688.982 µs) : 0, 689
Remote Config [candidate] (710.468 µs) : 0, 710
Telemetry [baseline] (9.62 ms) : 0, 9620
Telemetry [candidate] (9.29 ms) : 0, 9290
Flare Poller [baseline] (10.914 ms) : 0, 10914
Flare Poller [candidate] (11.808 ms) : 0, 11808
section iast
crashtracking [baseline] (1.508 ms) : 0, 1508
crashtracking [candidate] (1.486 ms) : 0, 1486
BytebuddyAgent [baseline] (818.278 ms) : 0, 818278
BytebuddyAgent [candidate] (813.567 ms) : 0, 813567
GlobalTracer [baseline] (232.471 ms) : 0, 232471
GlobalTracer [candidate] (231.218 ms) : 0, 231218
AppSec [baseline] (35.241 ms) : 0, 35241
AppSec [candidate] (34.965 ms) : 0, 34965
Debugger [baseline] (6.237 ms) : 0, 6237
Debugger [candidate] (6.224 ms) : 0, 6224
Remote Config [baseline] (616.389 µs) : 0, 616
Remote Config [candidate] (623.569 µs) : 0, 624
Telemetry [baseline] (8.688 ms) : 0, 8688
Telemetry [candidate] (8.758 ms) : 0, 8758
Flare Poller [baseline] (4.242 ms) : 0, 4242
Flare Poller [candidate] (4.157 ms) : 0, 4157
IAST [baseline] (26.395 ms) : 0, 26395
IAST [candidate] (26.511 ms) : 0, 26511
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master xinyuan/handle-empty-llm-eval-metric
git_commit_date 1760639037 1760645292
git_commit_sha 33e27c7 8347f7f
release_version 1.55.0-SNAPSHOT~33e27c7cea 1.55.0-SNAPSHOT~8347f7f8ff
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1760646881 1760646881
ci_job_id 1183896507 1183896507
ci_pipeline_id 79562452 79562452
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-cop2d98o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-cop2d98o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 3 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load better
[-320.213µs; -217.669µs] or [-7.277%; -4.946%]		 unstable
[-60.231op/s; +193.356op/s] or [-5.772%; +18.529%]		 4.132ms 1110.094op/s 4.400ms 1043.531op/s
scenario:load:insecure-bank:iast:high_load better
[-870.458µs; -511.668µs] or [-8.552%; -5.027%]		 unstable
[-25.904op/s; +91.904op/s] or [-5.675%; +20.134%]		 9.488ms 489.469op/s 10.179ms 456.469op/s
scenario:load:petclinic:code_origins:high_load worse
[+1.567ms; +2.364ms] or [+3.709%; +5.594%]		 unstable
[-11.351op/s; +4.130op/s] or [-10.247%; +3.729%]		 44.218ms 107.165op/s 42.253ms 110.775op/s
scenario:load:petclinic:tracing:high_load worse
[+3.220ms; +4.038ms] or [+7.660%; +9.605%]		 unstable
[-15.076op/s; +0.020op/s] or [-13.545%; +0.018%]		 45.671ms 103.772op/s 42.042ms 111.300op/s
scenario:load:petclinic:iast:high_load worse
[+2.474ms; +3.321ms] or [+5.693%; +7.643%]		 unstable
[-12.819op/s; +1.900op/s] or [-11.902%; +1.764%]		 46.351ms 102.241op/s 43.454ms 107.700op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.663 ms) : 36378, 36948
.   : milestone, 36663,
appsec (47.219 ms) : 46786, 47652
.   : milestone, 47219,
code_origins (42.253 ms) : 41902, 42603
.   : milestone, 42253,
iast (43.454 ms) : 43079, 43829
.   : milestone, 43454,
profiling (48.571 ms) : 48113, 49029
.   : milestone, 48571,
tracing (42.042 ms) : 41681, 42403
.   : milestone, 42042,
section candidate
no_agent (37.398 ms) : 37097, 37698
.   : milestone, 37398,
appsec (47.285 ms) : 46862, 47707
.   : milestone, 47285,
code_origins (44.218 ms) : 43829, 44606
.   : milestone, 44218,
iast (46.351 ms) : 45940, 46763
.   : milestone, 46351,
profiling (47.917 ms) : 47481, 48354
.   : milestone, 47917,
tracing (45.671 ms) : 45273, 46069
.   : milestone, 45671,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.663 ms [36.378 ms, 36.948 ms] -
appsec 47.219 ms [46.786 ms, 47.652 ms] 10.556 ms (28.8%)
code_origins 42.253 ms [41.902 ms, 42.603 ms] 5.59 ms (15.2%)
iast 43.454 ms [43.079 ms, 43.829 ms] 6.791 ms (18.5%)
profiling 48.571 ms [48.113 ms, 49.029 ms] 11.908 ms (32.5%)
tracing 42.042 ms [41.681 ms, 42.403 ms] 5.379 ms (14.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.398 ms [37.097 ms, 37.698 ms] -
appsec 47.285 ms [46.862 ms, 47.707 ms] 9.887 ms (26.4%)
code_origins 44.218 ms [43.829 ms, 44.606 ms] 6.82 ms (18.2%)
iast 46.351 ms [45.94 ms, 46.763 ms] 8.954 ms (23.9%)
profiling 47.917 ms [47.481 ms, 48.354 ms] 10.52 ms (28.1%)
tracing 45.671 ms [45.273 ms, 46.069 ms] 8.273 ms (22.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.4 ms) : 4351, 4450
.   : milestone, 4400,
iast (10.179 ms) : 10002, 10356
.   : milestone, 10179,
iast_FULL (14.565 ms) : 14274, 14856
.   : milestone, 14565,
iast_GLOBAL (10.816 ms) : 10621, 11010
.   : milestone, 10816,
profiling (9.071 ms) : 8928, 9215
.   : milestone, 9071,
tracing (7.895 ms) : 7781, 8009
.   : milestone, 7895,
section candidate
no_agent (4.132 ms) : 4086, 4177
.   : milestone, 4132,
iast (9.488 ms) : 9332, 9644
.   : milestone, 9488,
iast_FULL (14.721 ms) : 14424, 15018
.   : milestone, 14721,
iast_GLOBAL (10.909 ms) : 10713, 11105
.   : milestone, 10909,
profiling (8.965 ms) : 8825, 9106
.   : milestone, 8965,
tracing (8.134 ms) : 8016, 8252
.   : milestone, 8134,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.4 ms [4.351 ms, 4.45 ms] -
iast 10.179 ms [10.002 ms, 10.356 ms] 5.778 ms (131.3%)
iast_FULL 14.565 ms [14.274 ms, 14.856 ms] 10.165 ms (231.0%)
iast_GLOBAL 10.816 ms [10.621 ms, 11.01 ms] 6.415 ms (145.8%)
profiling 9.071 ms [8.928 ms, 9.215 ms] 4.671 ms (106.1%)
tracing 7.895 ms [7.781 ms, 8.009 ms] 3.494 ms (79.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.132 ms [4.086 ms, 4.177 ms] -
iast 9.488 ms [9.332 ms, 9.644 ms] 5.356 ms (129.6%)
iast_FULL 14.721 ms [14.424 ms, 15.018 ms] 10.59 ms (256.3%)
iast_GLOBAL 10.909 ms [10.713 ms, 11.105 ms] 6.777 ms (164.0%)
profiling 8.965 ms [8.825 ms, 9.106 ms] 4.834 ms (117.0%)
tracing 8.134 ms [8.016 ms, 8.252 ms] 4.003 ms (96.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master xinyuan/handle-empty-llm-eval-metric
git_commit_date 1760639037 1760645292
git_commit_sha 33e27c7 8347f7f
release_version 1.55.0-SNAPSHOT~33e27c7cea 1.55.0-SNAPSHOT~8347f7f8ff
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1760647456 1760647456
ci_job_id 1183896508 1183896508
ci_pipeline_id 79562452 79562452
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-uzwqevr8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-uzwqevr8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.482 ms) : 1470, 1493
.   : milestone, 1482,
appsec (2.479 ms) : 2427, 2530
.   : milestone, 2479,
iast (2.228 ms) : 2164, 2292
.   : milestone, 2228,
iast_GLOBAL (2.28 ms) : 2215, 2345
.   : milestone, 2280,
profiling (2.069 ms) : 2017, 2120
.   : milestone, 2069,
tracing (2.046 ms) : 1996, 2096
.   : milestone, 2046,
section candidate
no_agent (1.486 ms) : 1474, 1498
.   : milestone, 1486,
appsec (3.739 ms) : 3516, 3961
.   : milestone, 3739,
iast (2.221 ms) : 2157, 2285
.   : milestone, 2221,
iast_GLOBAL (2.266 ms) : 2201, 2331
.   : milestone, 2266,
profiling (2.076 ms) : 2024, 2129
.   : milestone, 2076,
tracing (2.041 ms) : 1990, 2091
.   : milestone, 2041,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.482 ms [1.47 ms, 1.493 ms] -
appsec 2.479 ms [2.427 ms, 2.53 ms] 997.129 µs (67.3%)
iast 2.228 ms [2.164 ms, 2.292 ms] 746.361 µs (50.4%)
iast_GLOBAL 2.28 ms [2.215 ms, 2.345 ms] 798.425 µs (53.9%)
profiling 2.069 ms [2.017 ms, 2.12 ms] 586.935 µs (39.6%)
tracing 2.046 ms [1.996 ms, 2.096 ms] 564.154 µs (38.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.486 ms [1.474 ms, 1.498 ms] -
appsec 3.739 ms [3.516 ms, 3.961 ms] 2.253 ms (151.6%)
iast 2.221 ms [2.157 ms, 2.285 ms] 735.151 µs (49.5%)
iast_GLOBAL 2.266 ms [2.201 ms, 2.331 ms] 779.873 µs (52.5%)
profiling 2.076 ms [2.024 ms, 2.129 ms] 590.441 µs (39.7%)
tracing 2.041 ms [1.99 ms, 2.091 ms] 554.917 µs (37.3%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~8347f7f8ff, baseline=1.55.0-SNAPSHOT~33e27c7cea
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.824 s) : 14824000, 14824000
.   : milestone, 14824000,
appsec (15.191 s) : 15191000, 15191000
.   : milestone, 15191000,
iast (18.591 s) : 18591000, 18591000
.   : milestone, 18591000,
iast_GLOBAL (17.881 s) : 17881000, 17881000
.   : milestone, 17881000,
profiling (15.243 s) : 15243000, 15243000
.   : milestone, 15243000,
tracing (14.99 s) : 14990000, 14990000
.   : milestone, 14990000,
section candidate
no_agent (15.496 s) : 15496000, 15496000
.   : milestone, 15496000,
appsec (15.09 s) : 15090000, 15090000
.   : milestone, 15090000,
iast (18.227 s) : 18227000, 18227000
.   : milestone, 18227000,
iast_GLOBAL (17.997 s) : 17997000, 17997000
.   : milestone, 17997000,
profiling (15.117 s) : 15117000, 15117000
.   : milestone, 15117000,
tracing (15.096 s) : 15096000, 15096000
.   : milestone, 15096000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.824 s [14.824 s, 14.824 s] -
appsec 15.191 s [15.191 s, 15.191 s] 367.0 ms (2.5%)
iast 18.591 s [18.591 s, 18.591 s] 3.767 s (25.4%)
iast_GLOBAL 17.881 s [17.881 s, 17.881 s] 3.057 s (20.6%)
profiling 15.243 s [15.243 s, 15.243 s] 419.0 ms (2.8%)
tracing 14.99 s [14.99 s, 14.99 s] 166.0 ms (1.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.496 s [15.496 s, 15.496 s] -
appsec 15.09 s [15.09 s, 15.09 s] -406.0 ms (-2.6%)
iast 18.227 s [18.227 s, 18.227 s] 2.731 s (17.6%)
iast_GLOBAL 17.997 s [17.997 s, 17.997 s] 2.501 s (16.1%)
profiling 15.117 s [15.117 s, 15.117 s] -379.0 ms (-2.4%)
tracing 15.096 s [15.096 s, 15.096 s] -400.0 ms (-2.6%)

@XG-xin XG-xin changed the title Fix CPU spike in LLM Obs eval processor Fix CPU overhead in LLM Obs eval processor Oct 15, 2025
@XG-xin XG-xin marked this pull request as ready for review October 15, 2025 19:26
@XG-xin XG-xin requested a review from a team as a code owner October 15, 2025 19:26
@github-actions
Copy link
Contributor

github-actions bot commented Oct 15, 2025

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@PerfectSlayer PerfectSlayer added type: bug Bug report and fix tag: performance Performance related changes comp: mlobs ML Observability (LLMObs) labels Oct 16, 2025
@PerfectSlayer PerfectSlayer requested review from a team and mhlidd October 16, 2025 09:10
gary-huang
gary-huang approved these changes Oct 16, 2025
View reviewed changes
Copy link
Contributor

@gary-huang gary-huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work!

@XG-xin XG-xin merged commit c60b3c3 into master Oct 17, 2025
534 of 535 checks passed
@XG-xin XG-xin deleted the xinyuan/handle-empty-llm-eval-metric branch October 17, 2025 14:24
@github-actions github-actions bot added this to the 1.55.0 milestone Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gary-huang gary-huang gary-huang approved these changes

@mhlidd mhlidd Awaiting requested review from mhlidd mhlidd was automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: mlobs ML Observability (LLMObs) tag: performance Performance related changes type: bug Bug report and fix

Projects

None yet

Milestone

1.55.0

Development

Successfully merging this pull request may close these issues.

4 participants

@XG-xin @gary-huang @PerfectSlayer