Skip to content

Conversation

@mcculls
Copy link
Contributor

@mcculls mcculls commented Mar 11, 2024

It's been a year since we switched to the new combining installer in #4578 so it should be safe to remove the old one.

@mcculls mcculls added tag: no release notes Changes to exclude from release notes type: refactoring labels Mar 11, 2024
@mcculls mcculls force-pushed the mcculls/remove-unused-agent-builder-path branch from 05b7ce5 to e394aab Compare March 11, 2024 10:48
@mcculls mcculls force-pushed the mcculls/remove-legacy-agent-installer branch from 6568ac3 to 5cc1bbb Compare March 11, 2024 10:49
@mcculls mcculls marked this pull request as ready for review March 11, 2024 10:50
@mcculls mcculls requested a review from a team as a code owner March 11, 2024 10:50
@mcculls mcculls requested review from am312 and dougqh March 11, 2024 10:50
@mcculls mcculls force-pushed the mcculls/remove-unused-agent-builder-path branch from e394aab to 81130bb Compare March 11, 2024 11:52
@mcculls mcculls force-pushed the mcculls/remove-legacy-agent-installer branch from 5cc1bbb to b172856 Compare March 11, 2024 11:56
@pr-commenter
Copy link

pr-commenter bot commented Mar 11, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/remove-legacy-agent-installer
git_commit_date 1710185575 1710185610
git_commit_sha 4fa5f84 a2e2d72
release_version 1.32.0-SNAPSHOT~4fa5f84da9 1.32.0-SNAPSHOT~a2e2d726f0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710188807 1710188807
ci_job_id 456923955 456923955
ci_pipeline_id 29919197 29919197
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 13 unstable metrics.

Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~a2e2d726f0, baseline=1.32.0-SNAPSHOT~4fa5f84da9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1080647
Total [baseline] (9.154 s) : 0, 9153946
Agent [candidate] (1.08 s) : 0, 1080121
Total [candidate] (9.158 s) : 0, 9157650
section appsec
Agent [baseline] (1.199 s) : 0, 1199113
Total [baseline] (9.353 s) : 0, 9352599
Agent [candidate] (1.199 s) : 0, 1199271
Total [candidate] (9.302 s) : 0, 9302215
section iast
Agent [baseline] (1.203 s) : 0, 1202895
Total [baseline] (9.317 s) : 0, 9317190
Agent [candidate] (1.215 s) : 0, 1214640
Total [candidate] (9.344 s) : 0, 9343961
section profiling
Agent [baseline] (1.279 s) : 0, 1278769
Total [baseline] (9.417 s) : 0, 9416958
Agent [candidate] (1.274 s) : 0, 1273676
Total [candidate] (9.365 s) : 0, 9364643
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.199 s 118.466 ms (11.0%)
Agent iast 1.203 s 122.248 ms (11.3%)
Agent profiling 1.279 s 198.122 ms (18.3%)
Total tracing 9.154 s -
Total appsec 9.353 s 198.653 ms (2.2%)
Total iast 9.317 s 163.244 ms (1.8%)
Total profiling 9.417 s 263.011 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.08 s -
Agent appsec 1.199 s 119.15 ms (11.0%)
Agent iast 1.215 s 134.52 ms (12.5%)
Agent profiling 1.274 s 193.556 ms (17.9%)
Total tracing 9.158 s -
Total appsec 9.302 s 144.565 ms (1.6%)
Total iast 9.344 s 186.311 ms (2.0%)
Total profiling 9.365 s 206.993 ms (2.3%)
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~a2e2d726f0, baseline=1.32.0-SNAPSHOT~4fa5f84da9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (697.074 ms) : 0, 697074
BytebuddyAgent [candidate] (696.154 ms) : 0, 696154
GlobalTracer [baseline] (291.773 ms) : 0, 291773
GlobalTracer [candidate] (292.218 ms) : 0, 292218
AppSec [baseline] (49.009 ms) : 0, 49009
AppSec [candidate] (49.146 ms) : 0, 49146
Remote Config [baseline] (718.467 µs) : 0, 718
Remote Config [candidate] (720.122 µs) : 0, 720
Telemetry [baseline] (7.647 ms) : 0, 7647
Telemetry [candidate] (7.58 ms) : 0, 7580
section appsec
BytebuddyAgent [baseline] (696.001 ms) : 0, 696001
BytebuddyAgent [candidate] (694.801 ms) : 0, 694801
GlobalTracer [baseline] (290.364 ms) : 0, 290364
GlobalTracer [candidate] (291.672 ms) : 0, 291672
AppSec [baseline] (153.26 ms) : 0, 153260
AppSec [candidate] (153.28 ms) : 0, 153280
Remote Config [baseline] (607.483 µs) : 0, 607
Remote Config [candidate] (606.348 µs) : 0, 606
Telemetry [baseline] (6.827 ms) : 0, 6827
Telemetry [candidate] (6.875 ms) : 0, 6875
IAST [baseline] (17.845 ms) : 0, 17845
IAST [candidate] (17.841 ms) : 0, 17841
section iast
BytebuddyAgent [baseline] (800.665 ms) : 0, 800665
BytebuddyAgent [candidate] (808.085 ms) : 0, 808085
GlobalTracer [baseline] (288.006 ms) : 0, 288006
GlobalTracer [candidate] (290.608 ms) : 0, 290608
AppSec [baseline] (49.222 ms) : 0, 49222
AppSec [candidate] (47.789 ms) : 0, 47789
Remote Config [baseline] (614.217 µs) : 0, 614
Remote Config [candidate] (617.354 µs) : 0, 617
Telemetry [baseline] (7.363 ms) : 0, 7363
Telemetry [candidate] (6.629 ms) : 0, 6629
IAST [baseline] (22.623 ms) : 0, 22623
IAST [candidate] (26.323 ms) : 0, 26323
section profiling
ProfilingAgent [baseline] (95.325 ms) : 0, 95325
ProfilingAgent [candidate] (94.031 ms) : 0, 94031
BytebuddyAgent [baseline] (692.452 ms) : 0, 692452
BytebuddyAgent [candidate] (688.504 ms) : 0, 688504
GlobalTracer [baseline] (375.971 ms) : 0, 375971
GlobalTracer [candidate] (376.979 ms) : 0, 376979
AppSec [baseline] (50.199 ms) : 0, 50199
AppSec [candidate] (49.975 ms) : 0, 49975
Remote Config [baseline] (822.929 µs) : 0, 823
Remote Config [candidate] (733.217 µs) : 0, 733
Telemetry [baseline] (7.347 ms) : 0, 7347
Telemetry [candidate] (7.43 ms) : 0, 7430
Profiling [baseline] (95.349 ms) : 0, 95349
Profiling [candidate] (94.054 ms) : 0, 94054
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~a2e2d726f0, baseline=1.32.0-SNAPSHOT~4fa5f84da9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1085327
Total [baseline] (8.562 s) : 0, 8561953
Agent [candidate] (1.088 s) : 0, 1088436
Total [candidate] (8.621 s) : 0, 8620748
section iast
Agent [baseline] (1.213 s) : 0, 1212961
Total [baseline] (9.066 s) : 0, 9066423
Agent [candidate] (1.205 s) : 0, 1205302
Total [candidate] (9.076 s) : 0, 9076414
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.204 s) : 0, 1203692
Total [baseline] (9.002 s) : 0, 9002334
Agent [candidate] (1.206 s) : 0, 1205783
Total [candidate] (9.065 s) : 0, 9065469
section iast_TELEMETRY_OFF
Agent [baseline] (1.197 s) : 0, 1196554
Total [baseline] (9.018 s) : 0, 9017976
Agent [candidate] (1.201 s) : 0, 1200772
Total [candidate] (9.04 s) : 0, 9040162
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent iast 1.213 s 127.633 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.204 s 118.365 ms (10.9%)
Agent iast_TELEMETRY_OFF 1.197 s 111.227 ms (10.2%)
Total tracing 8.562 s -
Total iast 9.066 s 504.47 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.002 s 440.381 ms (5.1%)
Total iast_TELEMETRY_OFF 9.018 s 456.023 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent iast 1.205 s 116.866 ms (10.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.206 s 117.347 ms (10.8%)
Agent iast_TELEMETRY_OFF 1.201 s 112.337 ms (10.3%)
Total tracing 8.621 s -
Total iast 9.076 s 455.666 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.065 s 444.722 ms (5.2%)
Total iast_TELEMETRY_OFF 9.04 s 419.414 ms (4.9%)
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~a2e2d726f0, baseline=1.32.0-SNAPSHOT~4fa5f84da9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (700.753 ms) : 0, 700753
BytebuddyAgent [candidate] (701.808 ms) : 0, 701808
GlobalTracer [baseline] (292.382 ms) : 0, 292382
GlobalTracer [candidate] (294.048 ms) : 0, 294048
AppSec [baseline] (49.138 ms) : 0, 49138
AppSec [candidate] (49.631 ms) : 0, 49631
Remote Config [baseline] (729.044 µs) : 0, 729
Remote Config [candidate] (730.277 µs) : 0, 730
Telemetry [baseline] (7.786 ms) : 0, 7786
Telemetry [candidate] (7.653 ms) : 0, 7653
section iast
BytebuddyAgent [baseline] (807.022 ms) : 0, 807022
BytebuddyAgent [candidate] (801.525 ms) : 0, 801525
GlobalTracer [baseline] (289.923 ms) : 0, 289923
GlobalTracer [candidate] (288.83 ms) : 0, 288830
AppSec [baseline] (49.604 ms) : 0, 49604
AppSec [candidate] (51.193 ms) : 0, 51193
IAST [baseline] (23.863 ms) : 0, 23863
IAST [candidate] (21.313 ms) : 0, 21313
Remote Config [baseline] (620.64 µs) : 0, 621
Remote Config [candidate] (610.277 µs) : 0, 610
Telemetry [baseline] (7.358 ms) : 0, 7358
Telemetry [candidate] (7.443 ms) : 0, 7443
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (799.889 ms) : 0, 799889
BytebuddyAgent [candidate] (800.773 ms) : 0, 800773
GlobalTracer [baseline] (288.398 ms) : 0, 288398
GlobalTracer [candidate] (289.121 ms) : 0, 289121
AppSec [baseline] (50.173 ms) : 0, 50173
AppSec [candidate] (49.508 ms) : 0, 49508
IAST [baseline] (21.418 ms) : 0, 21418
IAST [candidate] (24.003 ms) : 0, 24003
Remote Config [baseline] (612.46 µs) : 0, 612
Remote Config [candidate] (616.824 µs) : 0, 617
Telemetry [baseline] (8.913 ms) : 0, 8913
Telemetry [candidate] (7.422 ms) : 0, 7422
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.025 ms) : 0, 792025
BytebuddyAgent [candidate] (795.194 ms) : 0, 795194
GlobalTracer [baseline] (289.081 ms) : 0, 289081
GlobalTracer [candidate] (290.275 ms) : 0, 290275
AppSec [baseline] (48.657 ms) : 0, 48657
AppSec [candidate] (50.693 ms) : 0, 50693
IAST [baseline] (25.623 ms) : 0, 25623
IAST [candidate] (22.354 ms) : 0, 22354
Remote Config [baseline] (587.172 µs) : 0, 587
Remote Config [candidate] (579.497 µs) : 0, 579
Telemetry [baseline] (6.404 ms) : 0, 6404
Telemetry [candidate] (7.213 ms) : 0, 7213
Loading

Load

Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~a2e2d726f0, baseline=1.32.0-SNAPSHOT~4fa5f84da9
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.333 ms) : 1314, 1352
.   : milestone, 1333,
appsec (1.763 ms) : 1740, 1786
.   : milestone, 1763,
iast (1.531 ms) : 1507, 1554
.   : milestone, 1531,
profiling (1.575 ms) : 1551, 1600
.   : milestone, 1575,
tracing (1.499 ms) : 1476, 1523
.   : milestone, 1499,
section candidate
no_agent (1.334 ms) : 1315, 1353
.   : milestone, 1334,
appsec (1.77 ms) : 1747, 1794
.   : milestone, 1770,
iast (1.522 ms) : 1498, 1546
.   : milestone, 1522,
profiling (1.537 ms) : 1513, 1561
.   : milestone, 1537,
tracing (1.506 ms) : 1482, 1529
.   : milestone, 1506,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.333 ms [1.314 ms, 1.352 ms] -
appsec 1.763 ms [1.74 ms, 1.786 ms] 430.181 µs (32.3%)
iast 1.531 ms [1.507 ms, 1.554 ms] 197.82 µs (14.8%)
profiling 1.575 ms [1.551 ms, 1.6 ms] 242.365 µs (18.2%)
tracing 1.499 ms [1.476 ms, 1.523 ms] 166.603 µs (12.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.334 ms [1.315 ms, 1.353 ms] -
appsec 1.77 ms [1.747 ms, 1.794 ms] 436.458 µs (32.7%)
iast 1.522 ms [1.498 ms, 1.546 ms] 187.914 µs (14.1%)
profiling 1.537 ms [1.513 ms, 1.561 ms] 202.752 µs (15.2%)
tracing 1.506 ms [1.482 ms, 1.529 ms] 171.855 µs (12.9%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~a2e2d726f0, baseline=1.32.0-SNAPSHOT~4fa5f84da9
    dateFormat X
    axisFormat %s
section baseline
no_agent (361.969 µs) : 342, 382
.   : milestone, 362,
iast (472.252 µs) : 452, 493
.   : milestone, 472,
iast_FULL (532.805 µs) : 512, 553
.   : milestone, 533,
iast_GLOBAL (493.276 µs) : 472, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (477.81 µs) : 457, 498
.   : milestone, 478,
iast_INACTIVE (452.193 µs) : 431, 474
.   : milestone, 452,
iast_TELEMETRY_OFF (476.194 µs) : 455, 498
.   : milestone, 476,
tracing (440.411 µs) : 420, 461
.   : milestone, 440,
section candidate
no_agent (370.241 µs) : 350, 390
.   : milestone, 370,
iast (469.27 µs) : 449, 490
.   : milestone, 469,
iast_FULL (541.857 µs) : 521, 563
.   : milestone, 542,
iast_GLOBAL (493.094 µs) : 472, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (476.951 µs) : 456, 497
.   : milestone, 477,
iast_INACTIVE (447.905 µs) : 427, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (467.696 µs) : 447, 489
.   : milestone, 468,
tracing (447.077 µs) : 426, 468
.   : milestone, 447,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.969 µs [341.952 µs, 381.987 µs] -
iast 472.252 µs [451.862 µs, 492.643 µs] 110.283 µs (30.5%)
iast_FULL 532.805 µs [512.222 µs, 553.389 µs] 170.836 µs (47.2%)
iast_GLOBAL 493.276 µs [472.415 µs, 514.137 µs] 131.307 µs (36.3%)
iast_HARDCODED_SECRET_DISABLED 477.81 µs [457.223 µs, 498.397 µs] 115.84 µs (32.0%)
iast_INACTIVE 452.193 µs [430.738 µs, 473.648 µs] 90.224 µs (24.9%)
iast_TELEMETRY_OFF 476.194 µs [454.815 µs, 497.574 µs] 114.225 µs (31.6%)
tracing 440.411 µs [419.738 µs, 461.084 µs] 78.441 µs (21.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.241 µs [350.317 µs, 390.164 µs] -
iast 469.27 µs [448.753 µs, 489.787 µs] 99.029 µs (26.7%)
iast_FULL 541.857 µs [521.119 µs, 562.595 µs] 171.617 µs (46.4%)
iast_GLOBAL 493.094 µs [472.19 µs, 513.999 µs] 122.854 µs (33.2%)
iast_HARDCODED_SECRET_DISABLED 476.951 µs [456.445 µs, 497.457 µs] 106.71 µs (28.8%)
iast_INACTIVE 447.905 µs [427.21 µs, 468.6 µs] 77.664 µs (21.0%)
iast_TELEMETRY_OFF 467.696 µs [446.794 µs, 488.598 µs] 97.455 µs (26.3%)
tracing 447.077 µs [426.328 µs, 467.827 µs] 76.837 µs (20.8%)

Base automatically changed from mcculls/remove-unused-agent-builder-path to master March 11, 2024 19:32
@mcculls mcculls force-pushed the mcculls/remove-legacy-agent-installer branch from b172856 to a2e2d72 Compare March 11, 2024 19:34
@mcculls mcculls merged commit 39767bb into master Mar 11, 2024
@mcculls mcculls deleted the mcculls/remove-legacy-agent-installer branch March 11, 2024 22:09
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants