Skip to content

Conversation

@smola
Copy link
Member

@smola smola commented Mar 12, 2024

What Does This Do

Backport #6792

Motivation

Additional Notes

Jira ticket: APPSEC-52172

@smola smola added type: bug Bug report and fix tag: no release notes Changes to exclude from release notes comp: telemetry Telemetry labels Mar 12, 2024
@smola smola requested a review from a team as a code owner March 12, 2024 07:53
@smola smola enabled auto-merge (squash) March 12, 2024 08:30
@pr-commenter
Copy link

pr-commenter bot commented Mar 12, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/6792-backport
git_commit_date 1710229508 1710229321
git_commit_sha 0d49c12 61f87d8
release_version 1.32.0-SNAPSHOT~0d49c12f53 1.32.0-SNAPSHOT~61f87d811f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710232523 1710232523
ci_job_id 457187475 457187475
ci_pipeline_id 29940843 29940843
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:tracing:AppSec worse
[+1.721ms; +3.399ms] or [+3.489%; +6.890%]
51.898ms 49.338ms
scenario:startup:petclinic:iast:Remote Config better
[-57.545µs; -16.253µs] or [-9.407%; -2.657%]
574.844µs 611.742µs
scenario:startup:petclinic:profiling:AppSec worse
[+2.349ms; +3.946ms] or [+4.723%; +7.932%]
52.892ms 49.744ms
scenario:startup:petclinic:tracing:AppSec worse
[+2.073ms; +2.891ms] or [+4.216%; +5.880%]
51.645ms 49.163ms
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~61f87d811f, baseline=1.32.0-SNAPSHOT~0d49c12f53

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.078 s) : 0, 1078148
Total [baseline] (9.182 s) : 0, 9181641
Agent [candidate] (1.083 s) : 0, 1083434
Total [candidate] (9.17 s) : 0, 9169582
section appsec
Agent [baseline] (1.201 s) : 0, 1201324
Total [baseline] (9.244 s) : 0, 9244001
Agent [candidate] (1.202 s) : 0, 1202327
Total [candidate] (9.332 s) : 0, 9332258
section iast
Agent [baseline] (1.206 s) : 0, 1205575
Total [baseline] (9.323 s) : 0, 9322672
Agent [candidate] (1.21 s) : 0, 1209844
Total [candidate] (9.351 s) : 0, 9351150
section profiling
Agent [baseline] (1.282 s) : 0, 1281567
Total [baseline] (9.433 s) : 0, 9433303
Agent [candidate] (1.275 s) : 0, 1275414
Total [candidate] (9.422 s) : 0, 9422195
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.078 s -
Agent appsec 1.201 s 123.176 ms (11.4%)
Agent iast 1.206 s 127.427 ms (11.8%)
Agent profiling 1.282 s 203.42 ms (18.9%)
Total tracing 9.182 s -
Total appsec 9.244 s 62.361 ms (0.7%)
Total iast 9.323 s 141.032 ms (1.5%)
Total profiling 9.433 s 251.662 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.202 s 118.893 ms (11.0%)
Agent iast 1.21 s 126.41 ms (11.7%)
Agent profiling 1.275 s 191.979 ms (17.7%)
Total tracing 9.17 s -
Total appsec 9.332 s 162.676 ms (1.8%)
Total iast 9.351 s 181.567 ms (2.0%)
Total profiling 9.422 s 252.613 ms (2.8%)
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~61f87d811f, baseline=1.32.0-SNAPSHOT~0d49c12f53

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (694.944 ms) : 0, 694944
BytebuddyAgent [candidate] (696.844 ms) : 0, 696844
GlobalTracer [baseline] (291.365 ms) : 0, 291365
GlobalTracer [candidate] (292.055 ms) : 0, 292055
AppSec [baseline] (49.163 ms) : 0, 49163
AppSec [candidate] (51.645 ms) : 0, 51645
Remote Config [baseline] (716.312 µs) : 0, 716
Remote Config [candidate] (721.543 µs) : 0, 722
Telemetry [baseline] (7.68 ms) : 0, 7680
Telemetry [candidate] (7.761 ms) : 0, 7761
section appsec
BytebuddyAgent [baseline] (697.029 ms) : 0, 697029
BytebuddyAgent [candidate] (698.124 ms) : 0, 698124
GlobalTracer [baseline] (291.335 ms) : 0, 291335
GlobalTracer [candidate] (292.559 ms) : 0, 292559
AppSec [baseline] (153.187 ms) : 0, 153187
AppSec [candidate] (151.496 ms) : 0, 151496
IAST [baseline] (17.9 ms) : 0, 17900
IAST [candidate] (18.055 ms) : 0, 18055
Remote Config [baseline] (610.422 µs) : 0, 610
Remote Config [candidate] (616.047 µs) : 0, 616
Telemetry [baseline] (6.911 ms) : 0, 6911
Telemetry [candidate] (6.972 ms) : 0, 6972
section iast
BytebuddyAgent [baseline] (801.453 ms) : 0, 801453
BytebuddyAgent [candidate] (802.021 ms) : 0, 802021
GlobalTracer [baseline] (288.333 ms) : 0, 288333
GlobalTracer [candidate] (289.671 ms) : 0, 289671
AppSec [baseline] (49.339 ms) : 0, 49339
AppSec [candidate] (52.74 ms) : 0, 52740
IAST [baseline] (24.749 ms) : 0, 24749
IAST [candidate] (23.658 ms) : 0, 23658
Remote Config [baseline] (611.742 µs) : 0, 612
Remote Config [candidate] (574.844 µs) : 0, 575
Telemetry [baseline] (6.67 ms) : 0, 6670
Telemetry [candidate] (6.665 ms) : 0, 6665
section profiling
BytebuddyAgent [baseline] (693.723 ms) : 0, 693723
BytebuddyAgent [candidate] (688.926 ms) : 0, 688926
GlobalTracer [baseline] (377.853 ms) : 0, 377853
GlobalTracer [candidate] (375.003 ms) : 0, 375003
AppSec [baseline] (49.744 ms) : 0, 49744
AppSec [candidate] (52.892 ms) : 0, 52892
Remote Config [baseline] (732.893 µs) : 0, 733
Remote Config [candidate] (772.276 µs) : 0, 772
Telemetry [baseline] (7.495 ms) : 0, 7495
Telemetry [candidate] (7.339 ms) : 0, 7339
ProfilingAgent [baseline] (95.343 ms) : 0, 95343
ProfilingAgent [candidate] (94.465 ms) : 0, 94465
Profiling [baseline] (95.368 ms) : 0, 95368
Profiling [candidate] (94.489 ms) : 0, 94489
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~61f87d811f, baseline=1.32.0-SNAPSHOT~0d49c12f53

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1084906
Total [baseline] (8.611 s) : 0, 8611457
Agent [candidate] (1.091 s) : 0, 1091295
Total [candidate] (8.59 s) : 0, 8590235
section iast
Agent [baseline] (1.203 s) : 0, 1203500
Total [baseline] (9.037 s) : 0, 9037282
Agent [candidate] (1.21 s) : 0, 1210037
Total [candidate] (9.05 s) : 0, 9049852
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.203 s) : 0, 1202907
Total [baseline] (8.999 s) : 0, 8998511
Agent [candidate] (1.208 s) : 0, 1208235
Total [candidate] (9.084 s) : 0, 9083979
section iast_TELEMETRY_OFF
Agent [baseline] (1.206 s) : 0, 1206055
Total [baseline] (9.066 s) : 0, 9066335
Agent [candidate] (1.204 s) : 0, 1204123
Total [candidate] (9.07 s) : 0, 9069854
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent iast 1.203 s 118.594 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.203 s 118.001 ms (10.9%)
Agent iast_TELEMETRY_OFF 1.206 s 121.149 ms (11.2%)
Total tracing 8.611 s -
Total iast 9.037 s 425.825 ms (4.9%)
Total iast_HARDCODED_SECRET_DISABLED 8.999 s 387.054 ms (4.5%)
Total iast_TELEMETRY_OFF 9.066 s 454.878 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent iast 1.21 s 118.741 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.208 s 116.939 ms (10.7%)
Agent iast_TELEMETRY_OFF 1.204 s 112.828 ms (10.3%)
Total tracing 8.59 s -
Total iast 9.05 s 459.617 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.084 s 493.744 ms (5.7%)
Total iast_TELEMETRY_OFF 9.07 s 479.619 ms (5.6%)
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~61f87d811f, baseline=1.32.0-SNAPSHOT~0d49c12f53

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (699.904 ms) : 0, 699904
BytebuddyAgent [candidate] (702.262 ms) : 0, 702262
GlobalTracer [baseline] (292.621 ms) : 0, 292621
GlobalTracer [candidate] (293.867 ms) : 0, 293867
AppSec [baseline] (49.338 ms) : 0, 49338
AppSec [candidate] (51.898 ms) : 0, 51898
Remote Config [baseline] (749.623 µs) : 0, 750
Remote Config [candidate] (732.723 µs) : 0, 733
Telemetry [baseline] (7.769 ms) : 0, 7769
Telemetry [candidate] (7.858 ms) : 0, 7858
section iast
BytebuddyAgent [baseline] (799.959 ms) : 0, 799959
BytebuddyAgent [candidate] (802.212 ms) : 0, 802212
GlobalTracer [baseline] (288.381 ms) : 0, 288381
GlobalTracer [candidate] (289.175 ms) : 0, 289175
AppSec [baseline] (50.807 ms) : 0, 50807
AppSec [candidate] (51.357 ms) : 0, 51357
IAST [baseline] (22.736 ms) : 0, 22736
IAST [candidate] (25.48 ms) : 0, 25480
Remote Config [baseline] (618.284 µs) : 0, 618
Remote Config [candidate] (587.578 µs) : 0, 588
Telemetry [baseline] (6.597 ms) : 0, 6597
Telemetry [candidate] (6.653 ms) : 0, 6653
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.141 ms) : 0, 800141
BytebuddyAgent [candidate] (801.692 ms) : 0, 801692
GlobalTracer [baseline] (287.931 ms) : 0, 287931
GlobalTracer [candidate] (289.341 ms) : 0, 289341
AppSec [baseline] (50.337 ms) : 0, 50337
AppSec [candidate] (51.6 ms) : 0, 51600
IAST [baseline] (22.898 ms) : 0, 22898
IAST [candidate] (23.109 ms) : 0, 23109
Remote Config [baseline] (617.893 µs) : 0, 618
Remote Config [candidate] (600.655 µs) : 0, 601
Telemetry [baseline] (6.626 ms) : 0, 6626
Telemetry [candidate] (7.511 ms) : 0, 7511
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (798.716 ms) : 0, 798716
BytebuddyAgent [candidate] (795.026 ms) : 0, 795026
GlobalTracer [baseline] (290.512 ms) : 0, 290512
GlobalTracer [candidate] (290.966 ms) : 0, 290966
AppSec [baseline] (49.103 ms) : 0, 49103
AppSec [candidate] (54.835 ms) : 0, 54835
IAST [baseline] (25.244 ms) : 0, 25244
IAST [candidate] (21.673 ms) : 0, 21673
Remote Config [baseline] (593.912 µs) : 0, 594
Remote Config [candidate] (583.566 µs) : 0, 584
Telemetry [baseline] (7.19 ms) : 0, 7190
Telemetry [candidate] (6.546 ms) : 0, 6546
Loading

Load

Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~61f87d811f, baseline=1.32.0-SNAPSHOT~0d49c12f53
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.344 ms) : 1325, 1363
.   : milestone, 1344,
appsec (1.764 ms) : 1740, 1787
.   : milestone, 1764,
iast (1.528 ms) : 1505, 1551
.   : milestone, 1528,
profiling (1.509 ms) : 1485, 1533
.   : milestone, 1509,
tracing (1.507 ms) : 1482, 1531
.   : milestone, 1507,
section candidate
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.766 ms) : 1743, 1789
.   : milestone, 1766,
iast (1.529 ms) : 1506, 1552
.   : milestone, 1529,
profiling (1.521 ms) : 1498, 1545
.   : milestone, 1521,
tracing (1.492 ms) : 1469, 1515
.   : milestone, 1492,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.325 ms, 1.363 ms] -
appsec 1.764 ms [1.74 ms, 1.787 ms] 419.933 µs (31.3%)
iast 1.528 ms [1.505 ms, 1.551 ms] 184.83 µs (13.8%)
profiling 1.509 ms [1.485 ms, 1.533 ms] 165.464 µs (12.3%)
tracing 1.507 ms [1.482 ms, 1.531 ms] 163.238 µs (12.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.322 ms, 1.36 ms] -
appsec 1.766 ms [1.743 ms, 1.789 ms] 424.991 µs (31.7%)
iast 1.529 ms [1.506 ms, 1.552 ms] 187.966 µs (14.0%)
profiling 1.521 ms [1.498 ms, 1.545 ms] 180.35 µs (13.4%)
tracing 1.492 ms [1.469 ms, 1.515 ms] 151.046 µs (11.3%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~61f87d811f, baseline=1.32.0-SNAPSHOT~0d49c12f53
    dateFormat X
    axisFormat %s
section baseline
no_agent (368.548 µs) : 348, 389
.   : milestone, 369,
iast (463.932 µs) : 444, 484
.   : milestone, 464,
iast_FULL (538.476 µs) : 518, 559
.   : milestone, 538,
iast_GLOBAL (494.263 µs) : 473, 516
.   : milestone, 494,
iast_HARDCODED_SECRET_DISABLED (464.764 µs) : 445, 485
.   : milestone, 465,
iast_INACTIVE (448.429 µs) : 427, 470
.   : milestone, 448,
iast_TELEMETRY_OFF (467.314 µs) : 447, 488
.   : milestone, 467,
tracing (445.069 µs) : 424, 466
.   : milestone, 445,
section candidate
no_agent (359.306 µs) : 339, 380
.   : milestone, 359,
iast (465.279 µs) : 445, 485
.   : milestone, 465,
iast_FULL (535.564 µs) : 515, 556
.   : milestone, 536,
iast_GLOBAL (497.693 µs) : 478, 518
.   : milestone, 498,
iast_HARDCODED_SECRET_DISABLED (472.371 µs) : 452, 493
.   : milestone, 472,
iast_INACTIVE (450.299 µs) : 429, 472
.   : milestone, 450,
iast_TELEMETRY_OFF (470.232 µs) : 450, 491
.   : milestone, 470,
tracing (445.406 µs) : 424, 467
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 368.548 µs [348.271 µs, 388.825 µs] -
iast 463.932 µs [443.585 µs, 484.279 µs] 95.384 µs (25.9%)
iast_FULL 538.476 µs [517.944 µs, 559.009 µs] 169.928 µs (46.1%)
iast_GLOBAL 494.263 µs [472.633 µs, 515.892 µs] 125.714 µs (34.1%)
iast_HARDCODED_SECRET_DISABLED 464.764 µs [444.748 µs, 484.779 µs] 96.216 µs (26.1%)
iast_INACTIVE 448.429 µs [426.925 µs, 469.933 µs] 79.881 µs (21.7%)
iast_TELEMETRY_OFF 467.314 µs [446.646 µs, 487.982 µs] 98.766 µs (26.8%)
tracing 445.069 µs [424.48 µs, 465.658 µs] 76.521 µs (20.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 359.306 µs [338.683 µs, 379.928 µs] -
iast 465.279 µs [445.095 µs, 485.464 µs] 105.974 µs (29.5%)
iast_FULL 535.564 µs [515.032 µs, 556.096 µs] 176.259 µs (49.1%)
iast_GLOBAL 497.693 µs [477.61 µs, 517.776 µs] 138.388 µs (38.5%)
iast_HARDCODED_SECRET_DISABLED 472.371 µs [451.774 µs, 492.969 µs] 113.066 µs (31.5%)
iast_INACTIVE 450.299 µs [429.017 µs, 471.582 µs] 90.994 µs (25.3%)
iast_TELEMETRY_OFF 470.232 µs [449.524 µs, 490.94 µs] 110.926 µs (30.9%)
tracing 445.406 µs [423.746 µs, 467.065 µs] 86.1 µs (24.0%)

@smola smola merged commit d2fd917 into release/v1.31.x Mar 12, 2024
@smola smola deleted the smola/6792-backport branch March 12, 2024 08:47
@smola smola changed the title Fix logged exception for dependency URIs representing directories (#6792) (1.30.x backport) Fix logged exception for dependency URIs representing directories (#6792) (1.31.x backport) Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: telemetry Telemetry tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants