Skip to content

Ensure net.bytebuddy.raw is set to true when ConfigTransformSpockExtension is installed #8728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 24, 2025
Merged

Ensure net.bytebuddy.raw is set to true when ConfigTransformSpockExtension is installed #8728

merged 1 commit into from
Apr 24, 2025

Conversation

mcculls
Copy link
Contributor

@mcculls mcculls commented Apr 24, 2025

What Does This Do

This matches the same setting used by AgentInstaller which avoids spurious test failures in agent-tooling, specifically TypeFactoryForkedTest. This requires raw-types enabled to match the behaviour when installing the tracer with -javaagent. Without this byte-buddy will use dufferent code paths involving generic checks, which can lead to test failures using recent versions of byte-buddy (specifically going between full and outline types, where outlines are always raw-types by their nature.)

Motivation

These failures would not happen when using -javaagent because AgentInstaller forces the use of the raw-types setting. ConfigTransformSpockExtension should therefore do the same.

Additional Notes

Note we can't set this property in TypeFactoryForkedTest because by then it is too late, byte-buddy is already confgured by ConfigTransformSpockExtension and the raw-types value cannot be changed.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mcculls
Ensure net.bytebuddy.raw is set to true when ConfigTransformSpockExte…
77cb569 
…nsion is installed

This matches the same setting used by AgentInstaller which avoids spurious test failures
in agent-tooling, specifically TypeFactoryForkedTest. This requires raw-types enabled to
match the behaviour when installing the tracer with -javaagent. Without this byte-buddy
will use dufferent code paths involving generic checks, which can lead to test failures
using recent versions of byte-buddy (specifically going between full and outline types,
where outlines are always raw-types by their nature.)

These failures would not happen when using -javaagent because AgentInstaller forces the
use of the raw-types setting. ConfigTransformSpockExtension should therefore do the same.

Note we can't set this property in TypeFactoryForkedTest because by then it is too late,
byte-buddy is already confgured by ConfigTransformSpockExtension and the raw-types value
cannot be changed.
@mcculls mcculls requested a review from a team as a code owner April 24, 2025 17:36
@mcculls mcculls requested review from dougqh and sarahchen6 April 24, 2025 17:36
sarahchen6
sarahchen6 approved these changes Apr 24, 2025
View reviewed changes
Copy link
Contributor

@sarahchen6 sarahchen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!!!

@pr-commenter
Copy link

pr-commenter bot commented Apr 24, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/fix-outline-test-environment
git_commit_date 1745495821 1745516160
git_commit_sha 44de9a3 77cb569
release_version 1.49.0-SNAPSHOT~44de9a3b51 1.49.0-SNAPSHOT~77cb569538
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745519333 1745519333
ci_job_id 911493484 911493484
ci_pipeline_id 63244666 63244666
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nsrtnna7-project-304-concurrent-0-dc57jkm9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nsrtnna7-project-304-concurrent-0-dc57jkm9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 16 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062689
Total [baseline] (8.734 s) : 0, 8733767
Agent [candidate] (1.062 s) : 0, 1061968
Total [candidate] (8.71 s) : 0, 8709947
section iast
Agent [baseline] (1.19 s) : 0, 1190182
Total [baseline] (9.306 s) : 0, 9305916
Agent [candidate] (1.188 s) : 0, 1187889
Total [candidate] (9.29 s) : 0, 9289599
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.195 s) : 0, 1195437
Total [baseline] (9.329 s) : 0, 9329405
Agent [candidate] (1.189 s) : 0, 1189390
Total [candidate] (9.262 s) : 0, 9261624
section iast_TELEMETRY_OFF
Agent [baseline] (1.203 s) : 0, 1203464
Total [baseline] (9.309 s) : 0, 9308685
Agent [candidate] (1.187 s) : 0, 1186804
Total [candidate] (9.286 s) : 0, 9286070
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.19 s 127.493 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.195 s 132.748 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.203 s 140.775 ms (13.2%)
Total tracing 8.734 s -
Total iast 9.306 s 572.149 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.329 s 595.638 ms (6.8%)
Total iast_TELEMETRY_OFF 9.309 s 574.918 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.188 s 125.921 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.189 s 127.421 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.187 s 124.836 ms (11.8%)
Total tracing 8.71 s -
Total iast 9.29 s 579.652 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.262 s 551.677 ms (6.3%)
Total iast_TELEMETRY_OFF 9.286 s 576.123 ms (6.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (724.912 ms) : 0, 724912
BytebuddyAgent [candidate] (723.961 ms) : 0, 723961
GlobalTracer [baseline] (240.937 ms) : 0, 240937
GlobalTracer [candidate] (239.792 ms) : 0, 239792
AppSec [baseline] (56.446 ms) : 0, 56446
AppSec [candidate] (55.784 ms) : 0, 55784
Debugger [baseline] (6.981 ms) : 0, 6981
Debugger [candidate] (8.461 ms) : 0, 8461
Remote Config [baseline] (700.317 µs) : 0, 700
Remote Config [candidate] (691.321 µs) : 0, 691
Telemetry [baseline] (9.121 ms) : 0, 9121
Telemetry [candidate] (9.763 ms) : 0, 9763
section iast
BytebuddyAgent [baseline] (842.392 ms) : 0, 842392
BytebuddyAgent [candidate] (842.17 ms) : 0, 842170
GlobalTracer [baseline] (230.899 ms) : 0, 230899
GlobalTracer [candidate] (229.964 ms) : 0, 229964
IAST [baseline] (24.496 ms) : 0, 24496
IAST [candidate] (26.608 ms) : 0, 26608
AppSec [baseline] (53.875 ms) : 0, 53875
AppSec [candidate] (50.153 ms) : 0, 50153
Debugger [baseline] (5.908 ms) : 0, 5908
Debugger [candidate] (5.818 ms) : 0, 5818
Remote Config [baseline] (597.902 µs) : 0, 598
Remote Config [candidate] (583.235 µs) : 0, 583
Telemetry [baseline] (7.891 ms) : 0, 7891
Telemetry [candidate] (7.734 ms) : 0, 7734
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (845.781 ms) : 0, 845781
BytebuddyAgent [candidate] (841.866 ms) : 0, 841866
GlobalTracer [baseline] (231.537 ms) : 0, 231537
GlobalTracer [candidate] (230.399 ms) : 0, 230399
IAST [baseline] (23.33 ms) : 0, 23330
IAST [candidate] (23.877 ms) : 0, 23877
AppSec [baseline] (53.552 ms) : 0, 53552
AppSec [candidate] (53.114 ms) : 0, 53114
Debugger [baseline] (6.042 ms) : 0, 6042
Debugger [candidate] (5.946 ms) : 0, 5946
Remote Config [baseline] (604.607 µs) : 0, 605
Remote Config [candidate] (594.93 µs) : 0, 595
Telemetry [baseline] (8.131 ms) : 0, 8131
Telemetry [candidate] (7.948 ms) : 0, 7948
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (853.222 ms) : 0, 853222
BytebuddyAgent [candidate] (839.87 ms) : 0, 839870
GlobalTracer [baseline] (232.566 ms) : 0, 232566
GlobalTracer [candidate] (230.356 ms) : 0, 230356
IAST [baseline] (22.806 ms) : 0, 22806
IAST [candidate] (22.368 ms) : 0, 22368
AppSec [baseline] (56.518 ms) : 0, 56518
AppSec [candidate] (56.381 ms) : 0, 56381
Debugger [baseline] (6.014 ms) : 0, 6014
Debugger [candidate] (5.982 ms) : 0, 5982
Remote Config [baseline] (627.094 µs) : 0, 627
Remote Config [candidate] (602.699 µs) : 0, 603
Telemetry [baseline] (7.963 ms) : 0, 7963
Telemetry [candidate] (7.875 ms) : 0, 7875
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057482
Total [baseline] (10.563 s) : 0, 10562525
Agent [candidate] (1.08 s) : 0, 1080254
Total [candidate] (10.599 s) : 0, 10599292
section appsec
Agent [baseline] (1.202 s) : 0, 1202438
Total [baseline] (10.789 s) : 0, 10789244
Agent [candidate] (1.204 s) : 0, 1204181
Total [candidate] (10.749 s) : 0, 10748648
section iast
Agent [baseline] (1.186 s) : 0, 1186008
Total [baseline] (11.003 s) : 0, 11003139
Agent [candidate] (1.188 s) : 0, 1188141
Total [candidate] (10.903 s) : 0, 10903122
section profiling
Agent [baseline] (1.315 s) : 0, 1314709
Total [baseline] (10.824 s) : 0, 10824232
Agent [candidate] (1.319 s) : 0, 1318913
Total [candidate] (10.764 s) : 0, 10763648
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.202 s 144.955 ms (13.7%)
Agent iast 1.186 s 128.525 ms (12.2%)
Agent profiling 1.315 s 257.227 ms (24.3%)
Total tracing 10.563 s -
Total appsec 10.789 s 226.719 ms (2.1%)
Total iast 11.003 s 440.614 ms (4.2%)
Total profiling 10.824 s 261.708 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.08 s -
Agent appsec 1.204 s 123.927 ms (11.5%)
Agent iast 1.188 s 107.887 ms (10.0%)
Agent profiling 1.319 s 238.659 ms (22.1%)
Total tracing 10.599 s -
Total appsec 10.749 s 149.356 ms (1.4%)
Total iast 10.903 s 303.83 ms (2.9%)
Total profiling 10.764 s 164.356 ms (1.6%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (723.51 ms) : 0, 723510
BytebuddyAgent [candidate] (736.918 ms) : 0, 736918
GlobalTracer [baseline] (239.802 ms) : 0, 239802
GlobalTracer [candidate] (243.902 ms) : 0, 243902
AppSec [baseline] (54.653 ms) : 0, 54653
AppSec [candidate] (56.38 ms) : 0, 56380
Debugger [baseline] (6.94 ms) : 0, 6940
Debugger [candidate] (8.428 ms) : 0, 8428
Remote Config [baseline] (696.297 µs) : 0, 696
Remote Config [candidate] (717.425 µs) : 0, 717
Telemetry [baseline] (8.418 ms) : 0, 8418
Telemetry [candidate] (9.951 ms) : 0, 9951
section appsec
BytebuddyAgent [baseline] (742.003 ms) : 0, 742003
BytebuddyAgent [candidate] (742.424 ms) : 0, 742424
GlobalTracer [baseline] (236.923 ms) : 0, 236923
GlobalTracer [candidate] (237.418 ms) : 0, 237418
AppSec [baseline] (174.548 ms) : 0, 174548
AppSec [candidate] (175.499 ms) : 0, 175499
Debugger [baseline] (6.336 ms) : 0, 6336
Debugger [candidate] (5.937 ms) : 0, 5937
Remote Config [baseline] (636.096 µs) : 0, 636
Remote Config [candidate] (627.926 µs) : 0, 628
Telemetry [baseline] (7.488 ms) : 0, 7488
Telemetry [candidate] (7.786 ms) : 0, 7786
IAST [baseline] (21.775 ms) : 0, 21775
IAST [candidate] (21.779 ms) : 0, 21779
section iast
BytebuddyAgent [baseline] (839.809 ms) : 0, 839809
BytebuddyAgent [candidate] (841.472 ms) : 0, 841472
GlobalTracer [baseline] (230.116 ms) : 0, 230116
GlobalTracer [candidate] (229.886 ms) : 0, 229886
AppSec [baseline] (50.842 ms) : 0, 50842
AppSec [candidate] (50.522 ms) : 0, 50522
Debugger [baseline] (5.838 ms) : 0, 5838
Debugger [candidate] (5.907 ms) : 0, 5907
Remote Config [baseline] (589.317 µs) : 0, 589
Remote Config [candidate] (586.511 µs) : 0, 587
Telemetry [baseline] (7.898 ms) : 0, 7898
Telemetry [candidate] (7.956 ms) : 0, 7956
IAST [baseline] (25.205 ms) : 0, 25205
IAST [candidate] (24.494 ms) : 0, 24494
section profiling
BytebuddyAgent [baseline] (714.992 ms) : 0, 714992
BytebuddyAgent [candidate] (718.059 ms) : 0, 718059
GlobalTracer [baseline] (376.451 ms) : 0, 376451
GlobalTracer [candidate] (377.984 ms) : 0, 377984
AppSec [baseline] (54.33 ms) : 0, 54330
AppSec [candidate] (53.316 ms) : 0, 53316
Debugger [baseline] (6.161 ms) : 0, 6161
Debugger [candidate] (6.202 ms) : 0, 6202
Remote Config [baseline] (665.463 µs) : 0, 665
Remote Config [candidate] (659.046 µs) : 0, 659
Telemetry [baseline] (8.082 ms) : 0, 8082
Telemetry [candidate] (8.101 ms) : 0, 8101
ProfilingAgent [baseline] (103.531 ms) : 0, 103531
ProfilingAgent [candidate] (103.982 ms) : 0, 103982
Profiling [baseline] (103.557 ms) : 0, 103557
Profiling [candidate] (104.008 ms) : 0, 104008
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-24T17:56:38 2025-04-24T18:04:22
git_branch master mcculls/fix-outline-test-environment
git_commit_date 1745495821 1745516160
git_commit_sha 44de9a3 77cb569
release_version 1.49.0-SNAPSHOT~44de9a3b51 1.49.0-SNAPSHOT~77cb569538
start_time 2025-04-24T17:56:23 2025-04-24T18:04:07
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1745518260 1745518260
ci_job_id 911493485 911493485
ci_pipeline_id 63244666 63244666
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-nsrtnna7-project-304-concurrent-1-nh7w75fq 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-nsrtnna7-project-304-concurrent-1-nh7w75fq 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.364 ms) : 1344, 1383
.   : milestone, 1364,
appsec (1.733 ms) : 1709, 1757
.   : milestone, 1733,
appsec_no_iast (1.732 ms) : 1708, 1756
.   : milestone, 1732,
code_origins (1.679 ms) : 1653, 1705
.   : milestone, 1679,
iast (1.518 ms) : 1494, 1543
.   : milestone, 1518,
profiling (1.507 ms) : 1483, 1530
.   : milestone, 1507,
tracing (1.49 ms) : 1466, 1514
.   : milestone, 1490,
section candidate
no_agent (1.355 ms) : 1336, 1375
.   : milestone, 1355,
appsec (1.733 ms) : 1710, 1757
.   : milestone, 1733,
appsec_no_iast (1.72 ms) : 1697, 1744
.   : milestone, 1720,
code_origins (1.689 ms) : 1661, 1716
.   : milestone, 1689,
iast (1.523 ms) : 1499, 1546
.   : milestone, 1523,
profiling (1.497 ms) : 1473, 1522
.   : milestone, 1497,
tracing (1.503 ms) : 1478, 1528
.   : milestone, 1503,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.364 ms [1.344 ms, 1.383 ms] -
appsec 1.733 ms [1.709 ms, 1.757 ms] 369.504 µs (27.1%)
appsec_no_iast 1.732 ms [1.708 ms, 1.756 ms] 368.478 µs (27.0%)
code_origins 1.679 ms [1.653 ms, 1.705 ms] 315.214 µs (23.1%)
iast 1.518 ms [1.494 ms, 1.543 ms] 154.463 µs (11.3%)
profiling 1.507 ms [1.483 ms, 1.53 ms] 142.73 µs (10.5%)
tracing 1.49 ms [1.466 ms, 1.514 ms] 126.487 µs (9.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.375 ms] -
appsec 1.733 ms [1.71 ms, 1.757 ms] 378.189 µs (27.9%)
appsec_no_iast 1.72 ms [1.697 ms, 1.744 ms] 365.153 µs (26.9%)
code_origins 1.689 ms [1.661 ms, 1.716 ms] 333.42 µs (24.6%)
iast 1.523 ms [1.499 ms, 1.546 ms] 167.407 µs (12.4%)
profiling 1.497 ms [1.473 ms, 1.522 ms] 142.068 µs (10.5%)
tracing 1.503 ms [1.478 ms, 1.528 ms] 147.944 µs (10.9%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.025 µs) : 360, 402
.   : milestone, 381,
iast (529.797 µs) : 507, 553
.   : milestone, 530,
iast_FULL (730.806 µs) : 708, 754
.   : milestone, 731,
iast_GLOBAL (571.365 µs) : 549, 594
.   : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (529.383 µs) : 508, 551
.   : milestone, 529,
iast_INACTIVE (467.352 µs) : 445, 490
.   : milestone, 467,
iast_TELEMETRY_OFF (513.23 µs) : 490, 536
.   : milestone, 513,
tracing (457.074 µs) : 436, 479
.   : milestone, 457,
section candidate
no_agent (385.578 µs) : 366, 405
.   : milestone, 386,
iast (515.951 µs) : 493, 539
.   : milestone, 516,
iast_FULL (732.141 µs) : 709, 755
.   : milestone, 732,
iast_GLOBAL (577.26 µs) : 552, 602
.   : milestone, 577,
iast_HARDCODED_SECRET_DISABLED (518.995 µs) : 497, 541
.   : milestone, 519,
iast_INACTIVE (473.483 µs) : 451, 496
.   : milestone, 473,
iast_TELEMETRY_OFF (517.485 µs) : 494, 541
.   : milestone, 517,
tracing (461.809 µs) : 440, 483
.   : milestone, 462,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.025 µs [359.881 µs, 402.17 µs] -
iast 529.797 µs [506.511 µs, 553.084 µs] 148.772 µs (39.0%)
iast_FULL 730.806 µs [708.091 µs, 753.52 µs] 349.781 µs (91.8%)
iast_GLOBAL 571.365 µs [549.016 µs, 593.714 µs] 190.34 µs (50.0%)
iast_HARDCODED_SECRET_DISABLED 529.383 µs [507.69 µs, 551.076 µs] 148.358 µs (38.9%)
iast_INACTIVE 467.352 µs [444.756 µs, 489.947 µs] 86.326 µs (22.7%)
iast_TELEMETRY_OFF 513.23 µs [490.156 µs, 536.304 µs] 132.205 µs (34.7%)
tracing 457.074 µs [435.579 µs, 478.569 µs] 76.049 µs (20.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.578 µs [365.97 µs, 405.187 µs] -
iast 515.951 µs [493.359 µs, 538.543 µs] 130.373 µs (33.8%)
iast_FULL 732.141 µs [709.205 µs, 755.077 µs] 346.563 µs (89.9%)
iast_GLOBAL 577.26 µs [552.486 µs, 602.035 µs] 191.682 µs (49.7%)
iast_HARDCODED_SECRET_DISABLED 518.995 µs [497.277 µs, 540.714 µs] 133.417 µs (34.6%)
iast_INACTIVE 473.483 µs [450.597 µs, 496.368 µs] 87.904 µs (22.8%)
iast_TELEMETRY_OFF 517.485 µs [494.26 µs, 540.709 µs] 131.906 µs (34.2%)
tracing 461.809 µs [440.455 µs, 483.164 µs] 76.231 µs (19.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/fix-outline-test-environment
git_commit_date 1745495821 1745516160
git_commit_sha 44de9a3 77cb569
release_version 1.49.0-SNAPSHOT~44de9a3b51 1.49.0-SNAPSHOT~77cb569538
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1745518744 1745518744
ci_job_id 911493486 911493486
ci_pipeline_id 63244666 63244666
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-u-qwygag-project-304-concurrent-0-m7yc7zyl 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-u-qwygag-project-304-concurrent-0-m7yc7zyl 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.967 s) : 14967000, 14967000
.   : milestone, 14967000,
appsec (15.017 s) : 15017000, 15017000
.   : milestone, 15017000,
iast (18.788 s) : 18788000, 18788000
.   : milestone, 18788000,
iast_GLOBAL (17.973 s) : 17973000, 17973000
.   : milestone, 17973000,
profiling (15.049 s) : 15049000, 15049000
.   : milestone, 15049000,
tracing (14.836 s) : 14836000, 14836000
.   : milestone, 14836000,
section candidate
no_agent (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
appsec (14.809 s) : 14809000, 14809000
.   : milestone, 14809000,
iast (18.442 s) : 18442000, 18442000
.   : milestone, 18442000,
iast_GLOBAL (17.819 s) : 17819000, 17819000
.   : milestone, 17819000,
profiling (14.976 s) : 14976000, 14976000
.   : milestone, 14976000,
tracing (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.967 s [14.967 s, 14.967 s] -
appsec 15.017 s [15.017 s, 15.017 s] 50.0 ms (0.3%)
iast 18.788 s [18.788 s, 18.788 s] 3.821 s (25.5%)
iast_GLOBAL 17.973 s [17.973 s, 17.973 s] 3.006 s (20.1%)
profiling 15.049 s [15.049 s, 15.049 s] 82.0 ms (0.5%)
tracing 14.836 s [14.836 s, 14.836 s] -131.0 ms (-0.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.922 s [14.922 s, 14.922 s] -
appsec 14.809 s [14.809 s, 14.809 s] -113.0 ms (-0.8%)
iast 18.442 s [18.442 s, 18.442 s] 3.52 s (23.6%)
iast_GLOBAL 17.819 s [17.819 s, 17.819 s] 2.897 s (19.4%)
profiling 14.976 s [14.976 s, 14.976 s] 54.0 ms (0.4%)
tracing 14.922 s [14.922 s, 14.922 s] 0.0 µs (0.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~77cb569538, baseline=1.49.0-SNAPSHOT~44de9a3b51
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1462, 1485
.   : milestone, 1474,
appsec (2.363 ms) : 2317, 2410
.   : milestone, 2363,
iast (2.151 ms) : 2092, 2210
.   : milestone, 2151,
iast_GLOBAL (2.188 ms) : 2129, 2248
.   : milestone, 2188,
profiling (2.009 ms) : 1961, 2057
.   : milestone, 2009,
tracing (1.974 ms) : 1928, 2019
.   : milestone, 1974,
section candidate
no_agent (1.47 ms) : 1458, 1481
.   : milestone, 1470,
appsec (2.367 ms) : 2320, 2413
.   : milestone, 2367,
iast (2.149 ms) : 2090, 2209
.   : milestone, 2149,
iast_GLOBAL (2.193 ms) : 2133, 2253
.   : milestone, 2193,
profiling (2.022 ms) : 1973, 2071
.   : milestone, 2022,
tracing (1.979 ms) : 1933, 2025
.   : milestone, 1979,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.462 ms, 1.485 ms] -
appsec 2.363 ms [2.317 ms, 2.41 ms] 889.339 µs (60.3%)
iast 2.151 ms [2.092 ms, 2.21 ms] 676.823 µs (45.9%)
iast_GLOBAL 2.188 ms [2.129 ms, 2.248 ms] 714.15 µs (48.5%)
profiling 2.009 ms [1.961 ms, 2.057 ms] 534.964 µs (36.3%)
tracing 1.974 ms [1.928 ms, 2.019 ms] 499.983 µs (33.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.458 ms, 1.481 ms] -
appsec 2.367 ms [2.32 ms, 2.413 ms] 896.846 µs (61.0%)
iast 2.149 ms [2.09 ms, 2.209 ms] 679.558 µs (46.2%)
iast_GLOBAL 2.193 ms [2.133 ms, 2.253 ms] 723.03 µs (49.2%)
profiling 2.022 ms [1.973 ms, 2.071 ms] 552.164 µs (37.6%)
tracing 1.979 ms [1.933 ms, 2.025 ms] 509.29 µs (34.6%)

@mcculls mcculls merged commit 511416f into master Apr 24, 2025
459 checks passed
@mcculls mcculls deleted the mcculls/fix-outline-test-environment branch April 24, 2025 18:33
@github-actions github-actions bot added this to the 1.49.0 milestone Apr 24, 2025
This was referenced Apr 24, 2025
Closed
Merged
@mcculls mcculls added the tag: no release notes Changes to exclude from release notes label Apr 24, 2025
@mcculls mcculls mentioned this pull request Apr 24, 2025
Merged
sezen-datadog pushed a commit that referenced this pull request Apr 25, 2025
@mcculls @sezen-datadog
Ensure net.bytebuddy.raw is set to true when ConfigTransformSpockExte…
c020225 
…nsion is installed (#8728)

This matches the same setting used by AgentInstaller which avoids spurious test failures
in agent-tooling, specifically TypeFactoryForkedTest. This requires raw-types enabled to
match the behaviour when installing the tracer with -javaagent. Without this byte-buddy
will use dufferent code paths involving generic checks, which can lead to test failures
using recent versions of byte-buddy (specifically going between full and outline types,
where outlines are always raw-types by their nature.)

These failures would not happen when using -javaagent because AgentInstaller forces the
use of the raw-types setting. ConfigTransformSpockExtension should therefore do the same.

Note we can't set this property in TypeFactoryForkedTest because by then it is too late,
byte-buddy is already confgured by ConfigTransformSpockExtension and the raw-types value
cannot be changed.
sezen-datadog pushed a commit that referenced this pull request Apr 25, 2025
@mcculls @sezen-datadog
Ensure net.bytebuddy.raw is set to true when ConfigTransformSpockExte…
264e86b 
…nsion is installed (#8728)

This matches the same setting used by AgentInstaller which avoids spurious test failures
in agent-tooling, specifically TypeFactoryForkedTest. This requires raw-types enabled to
match the behaviour when installing the tracer with -javaagent. Without this byte-buddy
will use dufferent code paths involving generic checks, which can lead to test failures
using recent versions of byte-buddy (specifically going between full and outline types,
where outlines are always raw-types by their nature.)

These failures would not happen when using -javaagent because AgentInstaller forces the
use of the raw-types setting. ConfigTransformSpockExtension should therefore do the same.

Note we can't set this property in TypeFactoryForkedTest because by then it is too late,
byte-buddy is already confgured by ConfigTransformSpockExtension and the raw-types value
cannot be changed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sarahchen6 sarahchen6 sarahchen6 approved these changes

@dougqh dougqh Awaiting requested review from dougqh dougqh is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement
Projects
None yet
Milestone
1.49.0
Development

Successfully merging this pull request may close these issues.
2 participants
@mcculls @sarahchen6