Skip to content

Add line probe exploration tests #8741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 19, 2025
Merged

Add line probe exploration tests #8741

merged 1 commit into from
May 19, 2025

Conversation

jpbempel
Copy link
Member

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Apr 28, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1747662906 1747663276
end_time 2025-05-19T13:56:25 2025-05-19T14:02:34
git_branch master jpbempel/itw-line-probe
git_commit_sha c06c54a f315d1c
start_time 2025-05-19T13:55:07 2025-05-19T14:01:17
See matching parameters
Baseline Candidate
ci_job_id 943840921 943840921
ci_pipeline_id 65482224 65482224
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1747662386 1747662386

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+762.748µs; +845.608µs] or [+7.630%; +8.459%]		 worse
[+776.279µs; +879.058µs] or [+7.646%; +8.658%]		 worse
[+739.247µs; +875.411µs] or [+7.208%; +8.536%]		 worse
[+706.307µs; +1084.312µs] or [+6.700%; +10.285%]		 worse
[-8.201op/s; -5.793op/s] or [-8.447%; -5.967%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-44.035µs; +12.201µs] or [-15.400%; +4.267%]		 unstable
[-62.176µs; +18.256µs] or [-18.838%; +5.531%]		 unstable
[-75.168µs; +28.697µs] or [-21.719%; +8.292%]		 unstable
[-138.321µs; +91.574µs] or [-21.958%; +14.537%]		 same
scenario:basic unsure
[-14.125µs; -0.710µs] or [-4.873%; -0.245%]		 same same unstable
[-57.194µs; +72.486µs] or [-10.313%; +13.071%]		 unstable
[-113.759op/s; +235.711op/s] or [-4.664%; +9.664%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (330.048 µs) : 281, 379
.   : milestone, 330,
basic (320.222 µs) : 308, 332
.   : milestone, 320,
loop (10.153 ms) : 10114, 10191
.   : milestone, 10153,
section candidate
noprobe (308.088 µs) : 286, 330
.   : milestone, 308,
basic (311.311 µs) : 303, 320
.   : milestone, 311,
loop (10.98 ms) : 10925, 11036
.   : milestone, 10980,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 330.048 µs [280.719 µs, 379.378 µs]
basic 320.222 µs [308.485 µs, 331.959 µs]
loop 10.153 ms [10.114 ms, 10.191 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 308.088 µs [285.683 µs, 330.493 µs]
basic 311.311 µs [302.892 µs, 319.729 µs]
loop 10.98 ms [10.925 ms, 11.036 ms]

@pr-commenter
Copy link

pr-commenter bot commented Apr 28, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/itw-line-probe
git_commit_date 1747662360 1747663619
git_commit_sha c06c54a d9dd963
release_version 1.50.0-SNAPSHOT~c06c54ac67 1.50.0-SNAPSHOT~d9dd963e45
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747666159 1747666159
ci_job_id 943906799 943906799
ci_pipeline_id 65485672 65485672
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-rjizbyln-project-304-concurrent-0-792moyql 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-rjizbyln-project-304-concurrent-0-792moyql 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.032 s) : 0, 1032123
Total [baseline] (8.655 s) : 0, 8655229
Agent [candidate] (1.025 s) : 0, 1025051
Total [candidate] (8.647 s) : 0, 8647100
section iast
Agent [baseline] (1.157 s) : 0, 1156577
Total [baseline] (9.207 s) : 0, 9207137
Agent [candidate] (1.155 s) : 0, 1155357
Total [candidate] (9.212 s) : 0, 9212414
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.153 s) : 0, 1152900
Total [baseline] (9.185 s) : 0, 9185470
Agent [candidate] (1.156 s) : 0, 1156452
Total [candidate] (9.198 s) : 0, 9198025
section iast_TELEMETRY_OFF
Agent [baseline] (1.143 s) : 0, 1143332
Total [baseline] (9.23 s) : 0, 9230460
Agent [candidate] (1.146 s) : 0, 1145548
Total [candidate] (9.212 s) : 0, 9212165
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.032 s -
Agent iast 1.157 s 124.453 ms (12.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.153 s 120.777 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.143 s 111.209 ms (10.8%)
Total tracing 8.655 s -
Total iast 9.207 s 551.908 ms (6.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.185 s 530.241 ms (6.1%)
Total iast_TELEMETRY_OFF 9.23 s 575.231 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.025 s -
Agent iast 1.155 s 130.306 ms (12.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.156 s 131.401 ms (12.8%)
Agent iast_TELEMETRY_OFF 1.146 s 120.497 ms (11.8%)
Total tracing 8.647 s -
Total iast 9.212 s 565.314 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.198 s 550.926 ms (6.4%)
Total iast_TELEMETRY_OFF 9.212 s 565.065 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.288 ms) : 0, 691288
BytebuddyAgent [candidate] (685.606 ms) : 0, 685606
GlobalTracer [baseline] (241.733 ms) : 0, 241733
GlobalTracer [candidate] (240.953 ms) : 0, 240953
AppSec [baseline] (54.828 ms) : 0, 54828
AppSec [candidate] (54.555 ms) : 0, 54555
Debugger [baseline] (9.089 ms) : 0, 9089
Debugger [candidate] (9.739 ms) : 0, 9739
Remote Config [baseline] (702.552 µs) : 0, 703
Remote Config [candidate] (687.224 µs) : 0, 687
Telemetry [baseline] (10.644 ms) : 0, 10644
Telemetry [candidate] (9.8 ms) : 0, 9800
section iast
BytebuddyAgent [baseline] (807.66 ms) : 0, 807660
BytebuddyAgent [candidate] (806.37 ms) : 0, 806370
GlobalTracer [baseline] (231.873 ms) : 0, 231873
GlobalTracer [candidate] (231.547 ms) : 0, 231547
IAST [baseline] (29.651 ms) : 0, 29651
IAST [candidate] (29.581 ms) : 0, 29581
AppSec [baseline] (49.464 ms) : 0, 49464
AppSec [candidate] (48.974 ms) : 0, 48974
Debugger [baseline] (5.863 ms) : 0, 5863
Debugger [candidate] (5.942 ms) : 0, 5942
Remote Config [baseline] (590.343 µs) : 0, 590
Remote Config [candidate] (605.471 µs) : 0, 605
Telemetry [baseline] (7.862 ms) : 0, 7862
Telemetry [candidate] (7.978 ms) : 0, 7978
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (805.974 ms) : 0, 805974
BytebuddyAgent [candidate] (807.214 ms) : 0, 807214
GlobalTracer [baseline] (230.479 ms) : 0, 230479
GlobalTracer [candidate] (231.99 ms) : 0, 231990
IAST [baseline] (27.333 ms) : 0, 27333
IAST [candidate] (27.881 ms) : 0, 27881
AppSec [baseline] (50.442 ms) : 0, 50442
AppSec [candidate] (51.397 ms) : 0, 51397
Debugger [baseline] (5.893 ms) : 0, 5893
Debugger [candidate] (5.884 ms) : 0, 5884
Remote Config [baseline] (577.178 µs) : 0, 577
Remote Config [candidate] (589.693 µs) : 0, 590
Telemetry [baseline] (7.872 ms) : 0, 7872
Telemetry [candidate] (7.878 ms) : 0, 7878
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (797.051 ms) : 0, 797051
BytebuddyAgent [candidate] (798.942 ms) : 0, 798942
GlobalTracer [baseline] (230.181 ms) : 0, 230181
GlobalTracer [candidate] (230.577 ms) : 0, 230577
IAST [baseline] (22.278 ms) : 0, 22278
IAST [candidate] (23.118 ms) : 0, 23118
AppSec [baseline] (56.014 ms) : 0, 56014
AppSec [candidate] (54.957 ms) : 0, 54957
Debugger [baseline] (5.914 ms) : 0, 5914
Debugger [candidate] (6.009 ms) : 0, 6009
Remote Config [baseline] (591.607 µs) : 0, 592
Remote Config [candidate] (602.934 µs) : 0, 603
Telemetry [baseline] (7.722 ms) : 0, 7722
Telemetry [candidate] (7.804 ms) : 0, 7804
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.032 s) : 0, 1031858
Total [baseline] (10.48 s) : 0, 10479931
Agent [candidate] (1.02 s) : 0, 1019741
Total [candidate] (10.437 s) : 0, 10437078
section appsec
Agent [baseline] (1.166 s) : 0, 1166277
Total [baseline] (10.63 s) : 0, 10629615
Agent [candidate] (1.165 s) : 0, 1164800
Total [candidate] (10.662 s) : 0, 10662175
section iast
Agent [baseline] (1.159 s) : 0, 1159136
Total [baseline] (10.854 s) : 0, 10854493
Agent [candidate] (1.164 s) : 0, 1164176
Total [candidate] (11.045 s) : 0, 11044582
section profiling
Agent [baseline] (1.277 s) : 0, 1277280
Total [baseline] (10.85 s) : 0, 10849791
Agent [candidate] (1.278 s) : 0, 1277883
Total [candidate] (10.784 s) : 0, 10784195
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.032 s -
Agent appsec 1.166 s 134.419 ms (13.0%)
Agent iast 1.159 s 127.278 ms (12.3%)
Agent profiling 1.277 s 245.422 ms (23.8%)
Total tracing 10.48 s -
Total appsec 10.63 s 149.684 ms (1.4%)
Total iast 10.854 s 374.563 ms (3.6%)
Total profiling 10.85 s 369.86 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.02 s -
Agent appsec 1.165 s 145.059 ms (14.2%)
Agent iast 1.164 s 144.435 ms (14.2%)
Agent profiling 1.278 s 258.142 ms (25.3%)
Total tracing 10.437 s -
Total appsec 10.662 s 225.097 ms (2.2%)
Total iast 11.045 s 607.504 ms (5.8%)
Total profiling 10.784 s 347.117 ms (3.3%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.368 ms) : 0, 691368
BytebuddyAgent [candidate] (681.313 ms) : 0, 681313
GlobalTracer [baseline] (242.53 ms) : 0, 242530
GlobalTracer [candidate] (240.021 ms) : 0, 240021
AppSec [baseline] (54.881 ms) : 0, 54881
AppSec [candidate] (55.263 ms) : 0, 55263
Debugger [baseline] (8.562 ms) : 0, 8562
Debugger [candidate] (7.623 ms) : 0, 7623
Remote Config [baseline] (701.77 µs) : 0, 702
Remote Config [candidate] (680.4 µs) : 0, 680
Telemetry [baseline] (9.979 ms) : 0, 9979
Telemetry [candidate] (11.231 ms) : 0, 11231
section appsec
BytebuddyAgent [baseline] (704.669 ms) : 0, 704669
BytebuddyAgent [candidate] (703.252 ms) : 0, 703252
GlobalTracer [baseline] (236.971 ms) : 0, 236971
GlobalTracer [candidate] (237.042 ms) : 0, 237042
IAST [baseline] (21.778 ms) : 0, 21778
IAST [candidate] (21.617 ms) : 0, 21617
AppSec [baseline] (175.464 ms) : 0, 175464
AppSec [candidate] (176.009 ms) : 0, 176009
Debugger [baseline] (5.941 ms) : 0, 5941
Debugger [candidate] (5.952 ms) : 0, 5952
Remote Config [baseline] (625.015 µs) : 0, 625
Remote Config [candidate] (620.413 µs) : 0, 620
Telemetry [baseline] (8.08 ms) : 0, 8080
Telemetry [candidate] (7.414 ms) : 0, 7414
section iast
BytebuddyAgent [baseline] (809.775 ms) : 0, 809775
BytebuddyAgent [candidate] (813.324 ms) : 0, 813324
GlobalTracer [baseline] (232.083 ms) : 0, 232083
GlobalTracer [candidate] (233.032 ms) : 0, 233032
IAST [baseline] (30.205 ms) : 0, 30205
IAST [candidate] (28.51 ms) : 0, 28510
AppSec [baseline] (48.78 ms) : 0, 48780
AppSec [candidate] (50.835 ms) : 0, 50835
Debugger [baseline] (5.976 ms) : 0, 5976
Debugger [candidate] (6.036 ms) : 0, 6036
Remote Config [baseline] (602.012 µs) : 0, 602
Remote Config [candidate] (610.415 µs) : 0, 610
Telemetry [baseline] (7.985 ms) : 0, 7985
Telemetry [candidate] (8.041 ms) : 0, 8041
section profiling
BytebuddyAgent [baseline] (672.562 ms) : 0, 672562
BytebuddyAgent [candidate] (672.529 ms) : 0, 672529
GlobalTracer [baseline] (374.381 ms) : 0, 374381
GlobalTracer [candidate] (374.671 ms) : 0, 374671
AppSec [baseline] (61.584 ms) : 0, 61584
AppSec [candidate] (61.804 ms) : 0, 61804
Debugger [baseline] (6.198 ms) : 0, 6198
Debugger [candidate] (6.298 ms) : 0, 6298
Remote Config [baseline] (642.45 µs) : 0, 642
Remote Config [candidate] (644.578 µs) : 0, 645
Telemetry [baseline] (8.179 ms) : 0, 8179
Telemetry [candidate] (8.205 ms) : 0, 8205
ProfilingAgent [baseline] (102.987 ms) : 0, 102987
ProfilingAgent [candidate] (102.967 ms) : 0, 102967
Profiling [baseline] (103.01 ms) : 0, 103010
Profiling [candidate] (102.99 ms) : 0, 102990
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-19T14:17:50 2025-05-19T14:25:35
git_branch master jpbempel/itw-line-probe
git_commit_date 1747662360 1747663619
git_commit_sha c06c54a d9dd963
release_version 1.50.0-SNAPSHOT~c06c54ac67 1.50.0-SNAPSHOT~d9dd963e45
start_time 2025-05-19T14:17:36 2025-05-19T14:25:21
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747665134 1747665134
ci_job_id 943906800 943906800
ci_pipeline_id 65485672 65485672
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-rjizbyln-project-304-concurrent-1-xvzjocsk 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-rjizbyln-project-304-concurrent-1-xvzjocsk 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67
    dateFormat X
    axisFormat %s
section baseline
no_agent (382.929 µs) : 362, 403
.   : milestone, 383,
iast (535.974 µs) : 513, 559
.   : milestone, 536,
iast_FULL (740.473 µs) : 719, 762
.   : milestone, 740,
iast_GLOBAL (570.809 µs) : 549, 592
.   : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (526.782 µs) : 504, 550
.   : milestone, 527,
iast_INACTIVE (465.558 µs) : 443, 488
.   : milestone, 466,
iast_TELEMETRY_OFF (512.224 µs) : 489, 536
.   : milestone, 512,
tracing (463.459 µs) : 441, 486
.   : milestone, 463,
section candidate
no_agent (386.115 µs) : 366, 406
.   : milestone, 386,
iast (518.31 µs) : 497, 540
.   : milestone, 518,
iast_FULL (736.507 µs) : 715, 759
.   : milestone, 737,
iast_GLOBAL (566.634 µs) : 545, 588
.   : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (529.516 µs) : 507, 552
.   : milestone, 530,
iast_INACTIVE (467.886 µs) : 446, 490
.   : milestone, 468,
iast_TELEMETRY_OFF (521.03 µs) : 498, 544
.   : milestone, 521,
tracing (459.527 µs) : 438, 481
.   : milestone, 460,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.929 µs [362.365 µs, 403.493 µs] -
iast 535.974 µs [512.504 µs, 559.444 µs] 153.045 µs (40.0%)
iast_FULL 740.473 µs [718.65 µs, 762.296 µs] 357.544 µs (93.4%)
iast_GLOBAL 570.809 µs [549.482 µs, 592.136 µs] 187.88 µs (49.1%)
iast_HARDCODED_SECRET_DISABLED 526.782 µs [503.713 µs, 549.85 µs] 143.852 µs (37.6%)
iast_INACTIVE 465.558 µs [443.266 µs, 487.851 µs] 82.629 µs (21.6%)
iast_TELEMETRY_OFF 512.224 µs [488.551 µs, 535.897 µs] 129.295 µs (33.8%)
tracing 463.459 µs [441.31 µs, 485.607 µs] 80.529 µs (21.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.115 µs [366.172 µs, 406.058 µs] -
iast 518.31 µs [496.824 µs, 539.796 µs] 132.195 µs (34.2%)
iast_FULL 736.507 µs [714.508 µs, 758.507 µs] 350.392 µs (90.7%)
iast_GLOBAL 566.634 µs [545.207 µs, 588.06 µs] 180.518 µs (46.8%)
iast_HARDCODED_SECRET_DISABLED 529.516 µs [506.572 µs, 552.46 µs] 143.401 µs (37.1%)
iast_INACTIVE 467.886 µs [445.51 µs, 490.263 µs] 81.771 µs (21.2%)
iast_TELEMETRY_OFF 521.03 µs [498.013 µs, 544.046 µs] 134.915 µs (34.9%)
tracing 459.527 µs [437.996 µs, 481.058 µs] 73.412 µs (19.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1333, 1372
.   : milestone, 1352,
appsec (1.734 ms) : 1710, 1757
.   : milestone, 1734,
appsec_no_iast (1.723 ms) : 1699, 1747
.   : milestone, 1723,
code_origins (1.691 ms) : 1665, 1718
.   : milestone, 1691,
iast (1.515 ms) : 1491, 1538
.   : milestone, 1515,
profiling (1.507 ms) : 1483, 1531
.   : milestone, 1507,
tracing (1.507 ms) : 1482, 1532
.   : milestone, 1507,
section candidate
no_agent (1.376 ms) : 1357, 1396
.   : milestone, 1376,
appsec (1.736 ms) : 1713, 1759
.   : milestone, 1736,
appsec_no_iast (1.721 ms) : 1698, 1744
.   : milestone, 1721,
code_origins (1.68 ms) : 1654, 1706
.   : milestone, 1680,
iast (1.513 ms) : 1489, 1538
.   : milestone, 1513,
profiling (1.532 ms) : 1509, 1555
.   : milestone, 1532,
tracing (1.503 ms) : 1479, 1527
.   : milestone, 1503,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.333 ms, 1.372 ms] -
appsec 1.734 ms [1.71 ms, 1.757 ms] 381.208 µs (28.2%)
appsec_no_iast 1.723 ms [1.699 ms, 1.747 ms] 370.686 µs (27.4%)
code_origins 1.691 ms [1.665 ms, 1.718 ms] 338.946 µs (25.1%)
iast 1.515 ms [1.491 ms, 1.538 ms] 162.123 µs (12.0%)
profiling 1.507 ms [1.483 ms, 1.531 ms] 154.115 µs (11.4%)
tracing 1.507 ms [1.482 ms, 1.532 ms] 154.863 µs (11.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.376 ms [1.357 ms, 1.396 ms] -
appsec 1.736 ms [1.713 ms, 1.759 ms] 359.586 µs (26.1%)
appsec_no_iast 1.721 ms [1.698 ms, 1.744 ms] 344.538 µs (25.0%)
code_origins 1.68 ms [1.654 ms, 1.706 ms] 303.826 µs (22.1%)
iast 1.513 ms [1.489 ms, 1.538 ms] 137.1 µs (10.0%)
profiling 1.532 ms [1.509 ms, 1.555 ms] 155.399 µs (11.3%)
tracing 1.503 ms [1.479 ms, 1.527 ms] 126.82 µs (9.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/itw-line-probe
git_commit_date 1747662360 1747663619
git_commit_sha c06c54a d9dd963
release_version 1.50.0-SNAPSHOT~c06c54ac67 1.50.0-SNAPSHOT~d9dd963e45
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1747665696 1747665696
ci_job_id 943906801 943906801
ci_pipeline_id 65485672 65485672
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-luwakes2-project-304-concurrent-0-f1bualew 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-luwakes2-project-304-concurrent-0-f1bualew 6.8.0-1028-aws #30~22.04.1-Ubuntu SMP Sun Apr 20 06:03:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.468 s) : 15468000, 15468000
.   : milestone, 15468000,
appsec (15.003 s) : 15003000, 15003000
.   : milestone, 15003000,
iast (18.361 s) : 18361000, 18361000
.   : milestone, 18361000,
iast_GLOBAL (18.022 s) : 18022000, 18022000
.   : milestone, 18022000,
profiling (15.323 s) : 15323000, 15323000
.   : milestone, 15323000,
tracing (15.175 s) : 15175000, 15175000
.   : milestone, 15175000,
section candidate
no_agent (15.292 s) : 15292000, 15292000
.   : milestone, 15292000,
appsec (15.225 s) : 15225000, 15225000
.   : milestone, 15225000,
iast (18.223 s) : 18223000, 18223000
.   : milestone, 18223000,
iast_GLOBAL (17.993 s) : 17993000, 17993000
.   : milestone, 17993000,
profiling (14.858 s) : 14858000, 14858000
.   : milestone, 14858000,
tracing (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.468 s [15.468 s, 15.468 s] -
appsec 15.003 s [15.003 s, 15.003 s] -465.0 ms (-3.0%)
iast 18.361 s [18.361 s, 18.361 s] 2.893 s (18.7%)
iast_GLOBAL 18.022 s [18.022 s, 18.022 s] 2.554 s (16.5%)
profiling 15.323 s [15.323 s, 15.323 s] -145.0 ms (-0.9%)
tracing 15.175 s [15.175 s, 15.175 s] -293.0 ms (-1.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.292 s [15.292 s, 15.292 s] -
appsec 15.225 s [15.225 s, 15.225 s] -67.0 ms (-0.4%)
iast 18.223 s [18.223 s, 18.223 s] 2.931 s (19.2%)
iast_GLOBAL 17.993 s [17.993 s, 17.993 s] 2.701 s (17.7%)
profiling 14.858 s [14.858 s, 14.858 s] -434.0 ms (-2.8%)
tracing 14.915 s [14.915 s, 14.915 s] -377.0 ms (-2.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~d9dd963e45, baseline=1.50.0-SNAPSHOT~c06c54ac67
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.417 ms) : 2367, 2466
.   : milestone, 2417,
iast (2.192 ms) : 2130, 2254
.   : milestone, 2192,
iast_GLOBAL (2.233 ms) : 2170, 2295
.   : milestone, 2233,
profiling (2.034 ms) : 1985, 2084
.   : milestone, 2034,
tracing (2.027 ms) : 1979, 2075
.   : milestone, 2027,
section candidate
no_agent (1.479 ms) : 1467, 1490
.   : milestone, 1479,
appsec (2.411 ms) : 2362, 2461
.   : milestone, 2411,
iast (2.194 ms) : 2133, 2256
.   : milestone, 2194,
iast_GLOBAL (2.24 ms) : 2178, 2302
.   : milestone, 2240,
profiling (2.048 ms) : 1997, 2099
.   : milestone, 2048,
tracing (2.016 ms) : 1968, 2064
.   : milestone, 2016,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 2.417 ms [2.367 ms, 2.466 ms] 936.596 µs (63.3%)
iast 2.192 ms [2.13 ms, 2.254 ms] 711.79 µs (48.1%)
iast_GLOBAL 2.233 ms [2.17 ms, 2.295 ms] 752.718 µs (50.9%)
profiling 2.034 ms [1.985 ms, 2.084 ms] 554.255 µs (37.4%)
tracing 2.027 ms [1.979 ms, 2.075 ms] 546.747 µs (36.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.467 ms, 1.49 ms] -
appsec 2.411 ms [2.362 ms, 2.461 ms] 932.424 µs (63.0%)
iast 2.194 ms [2.133 ms, 2.256 ms] 715.456 µs (48.4%)
iast_GLOBAL 2.24 ms [2.178 ms, 2.302 ms] 760.978 µs (51.5%)
profiling 2.048 ms [1.997 ms, 2.099 ms] 569.098 µs (38.5%)
tracing 2.016 ms [1.968 ms, 2.064 ms] 536.761 µs (36.3%)

@jpbempel jpbempel added the tag: no release notes Changes to exclude from release notes label Apr 28, 2025
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch 2 times, most recently from b995155 to 625db29 Compare May 5, 2025 08:58
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch 5 times, most recently from 3423769 to ccf6196 Compare May 7, 2025 09:12
@jpbempel jpbempel marked this pull request as ready for review May 7, 2025 11:03
@jpbempel jpbempel requested review from a team as code owners May 7, 2025 11:03
@jpbempel jpbempel requested review from evanchooly and mcculls and removed request for a team May 7, 2025 11:03
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch 2 times, most recently from 9628084 to f315d1c Compare May 19, 2025 13:46
@jpbempel jpbempel requested a review from a team as a code owner May 19, 2025 13:46
@jpbempel jpbempel requested review from brettlangdon and removed request for a team May 19, 2025 13:46
@jpbempel
Add line probe exploration tests
d9dd963 
Refactor InstrumentTheWorld config to take a string indicating the
usage of method or line probe for the process.
Rename jobs to indicate line or method probes
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch from f315d1c to d9dd963 Compare May 19, 2025 14:07
@jpbempel jpbempel merged commit 872388f into master May 19, 2025
591 of 595 checks passed
@jpbempel jpbempel deleted the jpbempel/itw-line-probe branch May 19, 2025 15:15
@github-actions github-actions bot added this to the 1.50.0 milestone May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@evanchooly evanchooly evanchooly approved these changes

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

@brettlangdon brettlangdon Awaiting requested review from brettlangdon brettlangdon is a code owner automatically assigned from DataDog/apm-release-platform

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement
Projects
None yet
Milestone
1.50.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @evanchooly