Skip to content

Fix Exception Replay in Lambda #8849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Fix Exception Replay in Lambda #8849

wants to merge 6 commits into from

Conversation

nhulston
Copy link
Contributor

@nhulston nhulston commented May 19, 2025 •
edited
Loading

What Does This Do

In Lambda, the inner stack trace is truncated -- stack frames from fileName='AWSLambda.java' are removed. Therefore, in the original code, we would end up with a negative currentIdx, resulting in an error occurring in sanityCheckSnapshotAssignment.

This change adds a simple fallback:

      int currentIdx = innerTrace.length - snapshot.getStack().size();

      if (currentIdx < 0) {
        // This means the innerTrace was truncated by the underlying environment.
        // This is known to happen in AWS Lambda, but may also happen elsewhere.
        currentIdx = i;
      }

From manual testing in Lambda, this correctly gets the index in Lambda for all snapshots.
Screenshot 2025-05-19 at 1 21 16 PM

Motivation

https://datadoghq.atlassian.net/browse/SVLS-6855

Additional Notes

  • Exception Replay requires the exception to occur twice in Java: the first occurrence transforms the class, and the second occurrence captures the local variables
  • In Java, currently method parameters are captured by Exception Replay but other local variables are not captured yet

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@nhulston nhulston added type: enhancement comp: debugger Dynamic Instrumentation labels May 19, 2025
@pr-commenter
Copy link

pr-commenter bot commented May 19, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1747845985 1747846355
end_time 2025-05-21T16:47:46 2025-05-21T16:53:56
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_sha 58089f3 1f12c25
start_time 2025-05-21T16:46:26 2025-05-21T16:52:36
See matching parameters
Baseline Candidate
ci_job_id 948550027 948550027
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1747845400 1747845400

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+579.873µs; +614.558µs] or [+5.767%; +6.112%]		 worse
[+594.154µs; +651.668µs] or [+5.824%; +6.387%]		 worse
[+553.132µs; +651.974µs] or [+5.373%; +6.333%]		 worse
[+500.061µs; +843.173µs] or [+4.727%; +7.971%]		 worse
[-6.440op/s; -4.149op/s] or [-6.665%; -4.294%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-30.568µs; +43.765µs] or [-10.925%; +15.642%]		 unstable
[-44.257µs; +58.112µs] or [-13.640%; +17.910%]		 unstable
[-54.772µs; +68.325µs] or [-16.113%; +20.100%]		 unstable
[-140.565µs; +409.950µs] or [-14.919%; +43.509%]		 same
scenario:basic same same same unstable
[-38.673µs; +141.429µs] or [-4.807%; +17.581%]		 unstable
[-231.458op/s; +115.314op/s] or [-9.490%; +4.728%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (324.475 µs) : 284, 365
.   : milestone, 324,
basic (315.173 µs) : 305, 325
.   : milestone, 315,
loop (10.203 ms) : 10172, 10233
.   : milestone, 10203,
section candidate
noprobe (331.402 µs) : 276, 387
.   : milestone, 331,
basic (316.213 µs) : 306, 326
.   : milestone, 316,
loop (10.826 ms) : 10804, 10848
.   : milestone, 10826,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 324.475 µs [284.051 µs, 364.899 µs]
basic 315.173 µs [305.194 µs, 325.151 µs]
loop 10.203 ms [10.172 ms, 10.233 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 331.402 µs [275.595 µs, 387.21 µs]
basic 316.213 µs [306.048 µs, 326.377 µs]
loop 10.826 ms [10.804 ms, 10.848 ms]

@pr-commenter
Copy link

pr-commenter bot commented May 19, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_date 1747843428 1747845400
git_commit_sha 58089f3 1f12c25
release_version 1.50.0-SNAPSHOT~58089f324f 1.50.0-SNAPSHOT~1f12c25da2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747847720 1747847720
ci_job_id 948550021 948550021
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-pw3vcngw-project-304-concurrent-2-mr4c9knq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-pw3vcngw-project-304-concurrent-2-mr4c9knq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 56 metrics, 14 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:Remote Config worse
[+14.368µs; +59.830µs] or [+2.487%; +10.356%]		 614.835µs 577.736µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1026092
Total [baseline] (10.455 s) : 0, 10455475
Agent [candidate] (1.022 s) : 0, 1021593
Total [candidate] (10.455 s) : 0, 10455242
section appsec
Agent [baseline] (1.16 s) : 0, 1160093
Total [baseline] (10.639 s) : 0, 10638653
Agent [candidate] (1.16 s) : 0, 1159737
Total [candidate] (10.628 s) : 0, 10628407
section iast
Agent [baseline] (1.165 s) : 0, 1164850
Total [baseline] (10.925 s) : 0, 10924915
Agent [candidate] (1.147 s) : 0, 1147400
Total [candidate] (10.93 s) : 0, 10930161
section profiling
Agent [baseline] (1.266 s) : 0, 1265807
Total [baseline] (10.816 s) : 0, 10816311
Agent [candidate] (1.273 s) : 0, 1272961
Total [candidate] (10.86 s) : 0, 10860176
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.026 s -
Agent appsec 1.16 s 134.001 ms (13.1%)
Agent iast 1.165 s 138.757 ms (13.5%)
Agent profiling 1.266 s 239.715 ms (23.4%)
Total tracing 10.455 s -
Total appsec 10.639 s 183.178 ms (1.8%)
Total iast 10.925 s 469.44 ms (4.5%)
Total profiling 10.816 s 360.836 ms (3.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.022 s -
Agent appsec 1.16 s 138.145 ms (13.5%)
Agent iast 1.147 s 125.808 ms (12.3%)
Agent profiling 1.273 s 251.369 ms (24.6%)
Total tracing 10.455 s -
Total appsec 10.628 s 173.165 ms (1.7%)
Total iast 10.93 s 474.919 ms (4.5%)
Total profiling 10.86 s 404.934 ms (3.9%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.611 ms) : 0, 684611
BytebuddyAgent [candidate] (682.427 ms) : 0, 682427
GlobalTracer [baseline] (242.585 ms) : 0, 242585
GlobalTracer [candidate] (240.721 ms) : 0, 240721
AppSec [baseline] (55.616 ms) : 0, 55616
AppSec [candidate] (55.173 ms) : 0, 55173
Debugger [baseline] (7.611 ms) : 0, 7611
Debugger [candidate] (6.857 ms) : 0, 6857
Remote Config [baseline] (697.175 µs) : 0, 697
Remote Config [candidate] (702.882 µs) : 0, 703
Telemetry [baseline] (11.34 ms) : 0, 11340
Telemetry [candidate] (12.116 ms) : 0, 12116
section appsec
BytebuddyAgent [baseline] (699.866 ms) : 0, 699866
BytebuddyAgent [candidate] (699.86 ms) : 0, 699860
GlobalTracer [baseline] (236.36 ms) : 0, 236360
GlobalTracer [candidate] (236.605 ms) : 0, 236605
AppSec [baseline] (175.114 ms) : 0, 175114
AppSec [candidate] (175.007 ms) : 0, 175007
Debugger [baseline] (5.938 ms) : 0, 5938
Debugger [candidate] (5.899 ms) : 0, 5899
Remote Config [baseline] (630.746 µs) : 0, 631
Remote Config [candidate] (634.404 µs) : 0, 634
Telemetry [baseline] (7.779 ms) : 0, 7779
Telemetry [candidate] (7.372 ms) : 0, 7372
IAST [baseline] (21.73 ms) : 0, 21730
IAST [candidate] (21.753 ms) : 0, 21753
section iast
BytebuddyAgent [baseline] (814.416 ms) : 0, 814416
BytebuddyAgent [candidate] (801.041 ms) : 0, 801041
GlobalTracer [baseline] (233.097 ms) : 0, 233097
GlobalTracer [candidate] (230.372 ms) : 0, 230372
AppSec [baseline] (53.004 ms) : 0, 53004
AppSec [candidate] (51.114 ms) : 0, 51114
Debugger [baseline] (5.918 ms) : 0, 5918
Debugger [candidate] (5.779 ms) : 0, 5779
Remote Config [baseline] (593.746 µs) : 0, 594
Remote Config [candidate] (577.072 µs) : 0, 577
Telemetry [baseline] (7.861 ms) : 0, 7861
Telemetry [candidate] (7.799 ms) : 0, 7799
IAST [baseline] (26.197 ms) : 0, 26197
IAST [candidate] (26.493 ms) : 0, 26493
section profiling
BytebuddyAgent [baseline] (675.572 ms) : 0, 675572
BytebuddyAgent [candidate] (678.634 ms) : 0, 678634
GlobalTracer [baseline] (359.634 ms) : 0, 359634
GlobalTracer [candidate] (362.027 ms) : 0, 362027
AppSec [baseline] (61.801 ms) : 0, 61801
AppSec [candidate] (62.264 ms) : 0, 62264
Debugger [baseline] (6.251 ms) : 0, 6251
Debugger [candidate] (6.36 ms) : 0, 6360
Remote Config [baseline] (659.852 µs) : 0, 660
Remote Config [candidate] (657.077 µs) : 0, 657
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (8.313 ms) : 0, 8313
ProfilingAgent [baseline] (102.814 ms) : 0, 102814
ProfilingAgent [candidate] (103.589 ms) : 0, 103589
Profiling [baseline] (102.838 ms) : 0, 102838
Profiling [candidate] (103.613 ms) : 0, 103613
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1020732
Total [baseline] (8.652 s) : 0, 8652168
Agent [candidate] (1.017 s) : 0, 1017118
Total [candidate] (8.607 s) : 0, 8607095
section iast
Agent [baseline] (1.15 s) : 0, 1149518
Total [baseline] (9.228 s) : 0, 9228251
Agent [candidate] (1.148 s) : 0, 1147968
Total [candidate] (9.228 s) : 0, 9228182
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.151 s) : 0, 1151419
Total [baseline] (9.221 s) : 0, 9220792
Agent [candidate] (1.15 s) : 0, 1150253
Total [candidate] (9.199 s) : 0, 9199083
section iast_TELEMETRY_OFF
Agent [baseline] (1.145 s) : 0, 1145382
Total [baseline] (9.193 s) : 0, 9192773
Agent [candidate] (1.155 s) : 0, 1154517
Total [candidate] (9.232 s) : 0, 9232212
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.021 s -
Agent iast 1.15 s 128.786 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.151 s 130.687 ms (12.8%)
Agent iast_TELEMETRY_OFF 1.145 s 124.65 ms (12.2%)
Total tracing 8.652 s -
Total iast 9.228 s 576.083 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.221 s 568.624 ms (6.6%)
Total iast_TELEMETRY_OFF 9.193 s 540.605 ms (6.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.017 s -
Agent iast 1.148 s 130.85 ms (12.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.15 s 133.136 ms (13.1%)
Agent iast_TELEMETRY_OFF 1.155 s 137.4 ms (13.5%)
Total tracing 8.607 s -
Total iast 9.228 s 621.086 ms (7.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.199 s 591.988 ms (6.9%)
Total iast_TELEMETRY_OFF 9.232 s 625.117 ms (7.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.628 ms) : 0, 682628
BytebuddyAgent [candidate] (680.3 ms) : 0, 680300
GlobalTracer [baseline] (240.517 ms) : 0, 240517
GlobalTracer [candidate] (239.421 ms) : 0, 239421
AppSec [baseline] (54.458 ms) : 0, 54458
AppSec [candidate] (54.329 ms) : 0, 54329
Debugger [baseline] (9.079 ms) : 0, 9079
Debugger [candidate] (8.968 ms) : 0, 8968
Remote Config [baseline] (700.994 µs) : 0, 701
Remote Config [candidate] (683.736 µs) : 0, 684
Telemetry [baseline] (9.807 ms) : 0, 9807
Telemetry [candidate] (9.797 ms) : 0, 9797
section iast
BytebuddyAgent [baseline] (802.082 ms) : 0, 802082
BytebuddyAgent [candidate] (800.67 ms) : 0, 800670
GlobalTracer [baseline] (230.996 ms) : 0, 230996
GlobalTracer [candidate] (230.939 ms) : 0, 230939
AppSec [baseline] (50.269 ms) : 0, 50269
AppSec [candidate] (49.5 ms) : 0, 49500
Debugger [baseline] (5.907 ms) : 0, 5907
Debugger [candidate] (5.919 ms) : 0, 5919
Remote Config [baseline] (577.736 µs) : 0, 578
Remote Config [candidate] (614.835 µs) : 0, 615
Telemetry [baseline] (7.883 ms) : 0, 7883
Telemetry [candidate] (7.859 ms) : 0, 7859
IAST [baseline] (27.544 ms) : 0, 27544
IAST [candidate] (28.999 ms) : 0, 28999
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.813 ms) : 0, 801813
BytebuddyAgent [candidate] (802.5 ms) : 0, 802500
GlobalTracer [baseline] (231.424 ms) : 0, 231424
GlobalTracer [candidate] (231.166 ms) : 0, 231166
AppSec [baseline] (52.889 ms) : 0, 52889
AppSec [candidate] (51.149 ms) : 0, 51149
Debugger [baseline] (5.995 ms) : 0, 5995
Debugger [candidate] (5.959 ms) : 0, 5959
Remote Config [baseline] (628.716 µs) : 0, 629
Remote Config [candidate] (595.533 µs) : 0, 596
Telemetry [baseline] (7.972 ms) : 0, 7972
Telemetry [candidate] (7.965 ms) : 0, 7965
IAST [baseline] (27.275 ms) : 0, 27275
IAST [candidate] (27.387 ms) : 0, 27387
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.26 ms) : 0, 799260
BytebuddyAgent [candidate] (805.686 ms) : 0, 805686
GlobalTracer [baseline] (230.174 ms) : 0, 230174
GlobalTracer [candidate] (231.77 ms) : 0, 231770
AppSec [baseline] (54.287 ms) : 0, 54287
AppSec [candidate] (55.681 ms) : 0, 55681
Debugger [baseline] (5.957 ms) : 0, 5957
Debugger [candidate] (6.005 ms) : 0, 6005
Remote Config [baseline] (587.194 µs) : 0, 587
Remote Config [candidate] (623.31 µs) : 0, 623
Telemetry [baseline] (7.755 ms) : 0, 7755
Telemetry [candidate] (7.804 ms) : 0, 7804
IAST [baseline] (23.809 ms) : 0, 23809
IAST [candidate] (23.242 ms) : 0, 23242
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-21T16:47:30 2025-05-21T16:55:15
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_date 1747843428 1747845400
git_commit_sha 58089f3 1f12c25
release_version 1.50.0-SNAPSHOT~58089f324f 1.50.0-SNAPSHOT~1f12c25da2
start_time 2025-05-21T16:47:16 2025-05-21T16:55:00
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1747846912 1747846912
ci_job_id 948550022 948550022
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-3qkfhwhr-project-304-concurrent-0-ex7sp2gz 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-3qkfhwhr-project-304-concurrent-0-ex7sp2gz 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.366 ms) : 1345, 1386
.   : milestone, 1366,
appsec (1.731 ms) : 1709, 1754
.   : milestone, 1731,
appsec_no_iast (1.735 ms) : 1712, 1758
.   : milestone, 1735,
code_origins (1.677 ms) : 1649, 1704
.   : milestone, 1677,
iast (1.505 ms) : 1480, 1529
.   : milestone, 1505,
profiling (1.516 ms) : 1492, 1540
.   : milestone, 1516,
tracing (1.498 ms) : 1473, 1523
.   : milestone, 1498,
section candidate
no_agent (1.345 ms) : 1326, 1364
.   : milestone, 1345,
appsec (1.729 ms) : 1705, 1752
.   : milestone, 1729,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
code_origins (1.661 ms) : 1634, 1687
.   : milestone, 1661,
iast (1.519 ms) : 1495, 1542
.   : milestone, 1519,
profiling (1.526 ms) : 1501, 1551
.   : milestone, 1526,
tracing (1.496 ms) : 1474, 1519
.   : milestone, 1496,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.366 ms [1.345 ms, 1.386 ms] -
appsec 1.731 ms [1.709 ms, 1.754 ms] 365.75 µs (26.8%)
appsec_no_iast 1.735 ms [1.712 ms, 1.758 ms] 369.749 µs (27.1%)
code_origins 1.677 ms [1.649 ms, 1.704 ms] 310.987 µs (22.8%)
iast 1.505 ms [1.48 ms, 1.529 ms] 139.004 µs (10.2%)
profiling 1.516 ms [1.492 ms, 1.54 ms] 150.313 µs (11.0%)
tracing 1.498 ms [1.473 ms, 1.523 ms] 132.232 µs (9.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.345 ms [1.326 ms, 1.364 ms] -
appsec 1.729 ms [1.705 ms, 1.752 ms] 383.554 µs (28.5%)
appsec_no_iast 1.722 ms [1.698 ms, 1.746 ms] 376.675 µs (28.0%)
code_origins 1.661 ms [1.634 ms, 1.687 ms] 315.461 µs (23.5%)
iast 1.519 ms [1.495 ms, 1.542 ms] 173.606 µs (12.9%)
profiling 1.526 ms [1.501 ms, 1.551 ms] 180.613 µs (13.4%)
tracing 1.496 ms [1.474 ms, 1.519 ms] 151.12 µs (11.2%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (380.8 µs) : 361, 401
.   : milestone, 381,
iast (522.769 µs) : 501, 545
.   : milestone, 523,
iast_FULL (738.32 µs) : 715, 762
.   : milestone, 738,
iast_GLOBAL (569.882 µs) : 547, 592
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (521.483 µs) : 499, 544
.   : milestone, 521,
iast_INACTIVE (460.825 µs) : 440, 482
.   : milestone, 461,
iast_TELEMETRY_OFF (515.777 µs) : 493, 539
.   : milestone, 516,
tracing (464.261 µs) : 442, 487
.   : milestone, 464,
section candidate
no_agent (378.1 µs) : 358, 398
.   : milestone, 378,
iast (523.531 µs) : 500, 547
.   : milestone, 524,
iast_FULL (730.646 µs) : 709, 753
.   : milestone, 731,
iast_GLOBAL (573.889 µs) : 551, 597
.   : milestone, 574,
iast_HARDCODED_SECRET_DISABLED (518.315 µs) : 495, 542
.   : milestone, 518,
iast_INACTIVE (463.142 µs) : 441, 485
.   : milestone, 463,
iast_TELEMETRY_OFF (517.545 µs) : 494, 541
.   : milestone, 518,
tracing (459.539 µs) : 437, 482
.   : milestone, 460,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.8 µs [360.861 µs, 400.739 µs] -
iast 522.769 µs [501.016 µs, 544.523 µs] 141.969 µs (37.3%)
iast_FULL 738.32 µs [714.793 µs, 761.847 µs] 357.52 µs (93.9%)
iast_GLOBAL 569.882 µs [547.362 µs, 592.401 µs] 189.082 µs (49.7%)
iast_HARDCODED_SECRET_DISABLED 521.483 µs [498.989 µs, 543.977 µs] 140.682 µs (36.9%)
iast_INACTIVE 460.825 µs [439.525 µs, 482.124 µs] 80.024 µs (21.0%)
iast_TELEMETRY_OFF 515.777 µs [492.548 µs, 539.006 µs] 134.977 µs (35.4%)
tracing 464.261 µs [441.753 µs, 486.769 µs] 83.461 µs (21.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.1 µs [358.202 µs, 397.999 µs] -
iast 523.531 µs [500.093 µs, 546.969 µs] 145.431 µs (38.5%)
iast_FULL 730.646 µs [708.757 µs, 752.536 µs] 352.546 µs (93.2%)
iast_GLOBAL 573.889 µs [550.687 µs, 597.09 µs] 195.788 µs (51.8%)
iast_HARDCODED_SECRET_DISABLED 518.315 µs [495.1 µs, 541.53 µs] 140.214 µs (37.1%)
iast_INACTIVE 463.142 µs [440.916 µs, 485.368 µs] 85.041 µs (22.5%)
iast_TELEMETRY_OFF 517.545 µs [493.977 µs, 541.114 µs] 139.445 µs (36.9%)
tracing 459.539 µs [436.908 µs, 482.169 µs] 81.438 µs (21.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nicholas.hulston/fix-exception-replay-in-lambda
git_commit_date 1747843428 1747845400
git_commit_sha 58089f3 1f12c25
release_version 1.50.0-SNAPSHOT~58089f324f 1.50.0-SNAPSHOT~1f12c25da2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1747847409 1747847409
ci_job_id 948550023 948550023
ci_pipeline_id 65761122 65761122
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-pw3vcngw-project-304-concurrent-3-huawmceg 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-pw3vcngw-project-304-concurrent-3-huawmceg 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (2.394 ms) : 2345, 2442
.   : milestone, 2394,
iast (2.174 ms) : 2113, 2235
.   : milestone, 2174,
iast_GLOBAL (2.211 ms) : 2150, 2272
.   : milestone, 2211,
profiling (2.032 ms) : 1982, 2082
.   : milestone, 2032,
tracing (2.004 ms) : 1956, 2051
.   : milestone, 2004,
section candidate
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (2.388 ms) : 2340, 2437
.   : milestone, 2388,
iast (2.175 ms) : 2114, 2237
.   : milestone, 2175,
iast_GLOBAL (2.212 ms) : 2151, 2273
.   : milestone, 2212,
profiling (2.028 ms) : 1979, 2077
.   : milestone, 2028,
tracing (1.998 ms) : 1951, 2046
.   : milestone, 1998,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 2.394 ms [2.345 ms, 2.442 ms] 920.421 µs (62.5%)
iast 2.174 ms [2.113 ms, 2.235 ms] 700.915 µs (47.6%)
iast_GLOBAL 2.211 ms [2.15 ms, 2.272 ms] 737.525 µs (50.1%)
profiling 2.032 ms [1.982 ms, 2.082 ms] 558.637 µs (37.9%)
tracing 2.004 ms [1.956 ms, 2.051 ms] 530.355 µs (36.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.467 ms, 1.49 ms] -
appsec 2.388 ms [2.34 ms, 2.437 ms] 910.161 µs (61.6%)
iast 2.175 ms [2.114 ms, 2.237 ms] 697.035 µs (47.2%)
iast_GLOBAL 2.212 ms [2.151 ms, 2.273 ms] 733.614 µs (49.6%)
profiling 2.028 ms [1.979 ms, 2.077 ms] 549.884 µs (37.2%)
tracing 1.998 ms [1.951 ms, 2.046 ms] 520.198 µs (35.2%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~1f12c25da2, baseline=1.50.0-SNAPSHOT~58089f324f
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.961 s) : 14961000, 14961000
.   : milestone, 14961000,
appsec (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,
iast (18.796 s) : 18796000, 18796000
.   : milestone, 18796000,
iast_GLOBAL (18.328 s) : 18328000, 18328000
.   : milestone, 18328000,
profiling (15.29 s) : 15290000, 15290000
.   : milestone, 15290000,
tracing (14.993 s) : 14993000, 14993000
.   : milestone, 14993000,
section candidate
no_agent (15.291 s) : 15291000, 15291000
.   : milestone, 15291000,
appsec (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
iast (18.725 s) : 18725000, 18725000
.   : milestone, 18725000,
iast_GLOBAL (17.728 s) : 17728000, 17728000
.   : milestone, 17728000,
profiling (15.106 s) : 15106000, 15106000
.   : milestone, 15106000,
tracing (14.71 s) : 14710000, 14710000
.   : milestone, 14710000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.961 s [14.961 s, 14.961 s] -
appsec 14.851 s [14.851 s, 14.851 s] -110.0 ms (-0.7%)
iast 18.796 s [18.796 s, 18.796 s] 3.835 s (25.6%)
iast_GLOBAL 18.328 s [18.328 s, 18.328 s] 3.367 s (22.5%)
profiling 15.29 s [15.29 s, 15.29 s] 329.0 ms (2.2%)
tracing 14.993 s [14.993 s, 14.993 s] 32.0 ms (0.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.291 s [15.291 s, 15.291 s] -
appsec 14.893 s [14.893 s, 14.893 s] -398.0 ms (-2.6%)
iast 18.725 s [18.725 s, 18.725 s] 3.434 s (22.5%)
iast_GLOBAL 17.728 s [17.728 s, 17.728 s] 2.437 s (15.9%)
profiling 15.106 s [15.106 s, 15.106 s] -185.0 ms (-1.2%)
tracing 14.71 s [14.71 s, 14.71 s] -581.0 ms (-3.8%)

@nhulston nhulston marked this pull request as ready for review May 19, 2025 18:32
@nhulston nhulston requested a review from a team as a code owner May 19, 2025 18:32
@nhulston nhulston requested review from jpbempel and removed request for a team May 19, 2025 18:32
jpbempel
jpbempel reviewed May 20, 2025
View reviewed changes
...gent-debugger/src/test/java/com/datadog/debugger/exception/DefaultExceptionDebuggerTest.java Outdated
Comment on lines 301 to 306
List<Snapshot> snapshots = state.getSnapshots();
StackTraceElement ste = exception.getStackTrace()[0];
CapturedStackFrame dummyFrame = CapturedStackFrame.from(ste);
for (Snapshot snap : snapshots) {
snap.getStack().add(0, dummyFrame);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we mock instead the exception that is provided to handleException instead to return a truncated stack like what it is done by AWS lambda?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LMK what you think -- i ended up having to mock the snapshot, probe, exception state, and exception probe manager

Copy link
Member

@jpbempel jpbempel May 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i ended up having to mock the snapshot, probe, exception state, and exception probe manager

I am not sure it is necessary to do all of that, collecting real exception in snapshot is what happen anyway.
What matters is the truncated stacktrace for main exception

@nhulston nhulston requested review from jpbempel and hghotra and removed request for hghotra May 20, 2025 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hghotra hghotra hghotra approved these changes

@jpbempel jpbempel Awaiting requested review from jpbempel

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation type: enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nhulston @jpbempel @hghotra