Skip to content

mark flaky Exception Replay integration test #9970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

mark flaky Exception Replay integration test #9970

wants to merge 2 commits into from
+2 −0

Conversation

@jpbempel
Copy link
Member

@jpbempel jpbempel commented Nov 14, 2025

ExceptionDebuggerIntegrationTest::test3CapturedRecursiveFrames

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jpbempel jpbempel requested a review from a team as a code owner November 14, 2025 13:40
@jpbempel jpbempel added type: bug Bug report and fix tag: no release notes Changes to exclude from release notes labels Nov 14, 2025
@jpbempel jpbempel requested review from shatzi and removed request for a team November 14, 2025 13:40
@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Nov 14, 2025
@jpbempel
mark flaky Exception Replay integration test
e3c7eda 
ExceptionDebuggerIntegrationTest::test3CapturedRecursiveFrames
@jpbempel jpbempel force-pushed the jpbempel/mark-flaky branch from 967d961 to e3c7eda Compare November 14, 2025 13:45
@datadog-datadog-prod-us1

This comment has been minimized.

Sign in to view
@pr-commenter
Copy link

pr-commenter bot commented Nov 17, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/mark-flaky
git_commit_date 1763407991 1763408818
git_commit_sha b6d4e4f 56898bf
release_version 1.56.0-SNAPSHOT~b6d4e4fc4a 1.56.0-SNAPSHOT~56898bf128
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1763410867 1763410867
ci_job_id 1240187315 1240187315
ci_pipeline_id 82790005 82790005
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-5-0i64ojqa 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-5-0i64ojqa 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 5 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.105 s) : 0, 1104737
Total [baseline] (8.932 s) : 0, 8931555
Agent [candidate] (1.106 s) : 0, 1105890
Total [candidate] (8.853 s) : 0, 8853308
section iast
Agent [baseline] (1.242 s) : 0, 1241696
Total [baseline] (9.551 s) : 0, 9551277
Agent [candidate] (1.238 s) : 0, 1237574
Total [candidate] (9.549 s) : 0, 9549302
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.105 s -
Agent iast 1.242 s 136.96 ms (12.4%)
Total tracing 8.932 s -
Total iast 9.551 s 619.722 ms (6.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.106 s -
Agent iast 1.238 s 131.684 ms (11.9%)
Total tracing 8.853 s -
Total iast 9.549 s 695.993 ms (7.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (708.56 ms) : 0, 708560
BytebuddyAgent [candidate] (710.678 ms) : 0, 710678
GlobalTracer [baseline] (249.976 ms) : 0, 249976
GlobalTracer [candidate] (249.898 ms) : 0, 249898
AppSec [baseline] (32.711 ms) : 0, 32711
AppSec [candidate] (32.519 ms) : 0, 32519
Debugger [baseline] (64.339 ms) : 0, 64339
Debugger [candidate] (63.979 ms) : 0, 63979
Remote Config [baseline] (650.071 µs) : 0, 650
Remote Config [candidate] (623.466 µs) : 0, 623
Telemetry [baseline] (8.401 ms) : 0, 8401
Telemetry [candidate] (8.065 ms) : 0, 8065
Flare Poller [baseline] (3.756 ms) : 0, 3756
Flare Poller [candidate] (3.65 ms) : 0, 3650
section iast
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (833.558 ms) : 0, 833558
BytebuddyAgent [candidate] (829.988 ms) : 0, 829988
GlobalTracer [baseline] (238.194 ms) : 0, 238194
GlobalTracer [candidate] (238.316 ms) : 0, 238316
AppSec [baseline] (32.223 ms) : 0, 32223
AppSec [candidate] (34.917 ms) : 0, 34917
Debugger [baseline] (60.15 ms) : 0, 60150
Debugger [candidate] (59.802 ms) : 0, 59802
Remote Config [baseline] (549.869 µs) : 0, 550
Remote Config [candidate] (542.721 µs) : 0, 543
Telemetry [baseline] (7.708 ms) : 0, 7708
Telemetry [candidate] (7.595 ms) : 0, 7595
Flare Poller [baseline] (3.474 ms) : 0, 3474
Flare Poller [candidate] (3.431 ms) : 0, 3431
IAST [baseline] (29.29 ms) : 0, 29290
IAST [candidate] (26.578 ms) : 0, 26578
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.096 s) : 0, 1096322
Total [baseline] (10.766 s) : 0, 10765669
Agent [candidate] (1.099 s) : 0, 1098546
Total [candidate] (10.77 s) : 0, 10769789
section appsec
Agent [baseline] (1.288 s) : 0, 1288342
Total [baseline] (11.141 s) : 0, 11141080
Agent [candidate] (1.279 s) : 0, 1278710
Total [candidate] (11.152 s) : 0, 11152485
section iast
Agent [baseline] (1.24 s) : 0, 1239513
Total [baseline] (11.218 s) : 0, 11218167
Agent [candidate] (1.236 s) : 0, 1236199
Total [candidate] (11.273 s) : 0, 11272622
section profiling
Agent [baseline] (1.243 s) : 0, 1242513
Total [baseline] (10.983 s) : 0, 10983336
Agent [candidate] (1.227 s) : 0, 1226724
Total [candidate] (11.017 s) : 0, 11016948
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.096 s -
Agent appsec 1.288 s 192.021 ms (17.5%)
Agent iast 1.24 s 143.191 ms (13.1%)
Agent profiling 1.243 s 146.192 ms (13.3%)
Total tracing 10.766 s -
Total appsec 11.141 s 375.411 ms (3.5%)
Total iast 11.218 s 452.497 ms (4.2%)
Total profiling 10.983 s 217.667 ms (2.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.099 s -
Agent appsec 1.279 s 180.163 ms (16.4%)
Agent iast 1.236 s 137.653 ms (12.5%)
Agent profiling 1.227 s 128.177 ms (11.7%)
Total tracing 10.77 s -
Total appsec 11.152 s 382.696 ms (3.6%)
Total iast 11.273 s 502.833 ms (4.7%)
Total profiling 11.017 s 247.16 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.449 ms) : 0, 1449
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (704.399 ms) : 0, 704399
BytebuddyAgent [candidate] (705.541 ms) : 0, 705541
GlobalTracer [baseline] (247.446 ms) : 0, 247446
GlobalTracer [candidate] (248.167 ms) : 0, 248167
AppSec [baseline] (32.074 ms) : 0, 32074
AppSec [candidate] (32.169 ms) : 0, 32169
Debugger [baseline] (63.802 ms) : 0, 63802
Debugger [candidate] (63.856 ms) : 0, 63856
Remote Config [baseline] (617.11 µs) : 0, 617
Remote Config [candidate] (625.482 µs) : 0, 625
Telemetry [baseline] (8.145 ms) : 0, 8145
Telemetry [candidate] (8.224 ms) : 0, 8224
Flare Poller [baseline] (3.633 ms) : 0, 3633
Flare Poller [candidate] (3.649 ms) : 0, 3649
section appsec
crashtracking [baseline] (1.481 ms) : 0, 1481
crashtracking [candidate] (1.448 ms) : 0, 1448
BytebuddyAgent [baseline] (735.355 ms) : 0, 735355
BytebuddyAgent [candidate] (729.659 ms) : 0, 729659
GlobalTracer [baseline] (241.803 ms) : 0, 241803
GlobalTracer [candidate] (240.245 ms) : 0, 240245
IAST [baseline] (25.215 ms) : 0, 25215
IAST [candidate] (24.903 ms) : 0, 24903
AppSec [baseline] (174.79 ms) : 0, 174790
AppSec [candidate] (173.862 ms) : 0, 173862
Debugger [baseline] (61.41 ms) : 0, 61410
Debugger [candidate] (60.695 ms) : 0, 60695
Remote Config [baseline] (699.861 µs) : 0, 700
Remote Config [candidate] (675.801 µs) : 0, 676
Telemetry [baseline] (8.369 ms) : 0, 8369
Telemetry [candidate] (8.366 ms) : 0, 8366
Flare Poller [baseline] (3.981 ms) : 0, 3981
Flare Poller [candidate] (3.918 ms) : 0, 3918
section iast
crashtracking [baseline] (1.448 ms) : 0, 1448
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (830.477 ms) : 0, 830477
BytebuddyAgent [candidate] (827.859 ms) : 0, 827859
GlobalTracer [baseline] (238.226 ms) : 0, 238226
GlobalTracer [candidate] (237.764 ms) : 0, 237764
IAST [baseline] (28.946 ms) : 0, 28946
IAST [candidate] (28.284 ms) : 0, 28284
AppSec [baseline] (32.376 ms) : 0, 32376
AppSec [candidate] (33.021 ms) : 0, 33021
Debugger [baseline] (61.331 ms) : 0, 61331
Debugger [candidate] (61.166 ms) : 0, 61166
Remote Config [baseline] (551.311 µs) : 0, 551
Remote Config [candidate] (551.521 µs) : 0, 552
Telemetry [baseline] (7.763 ms) : 0, 7763
Telemetry [candidate] (7.649 ms) : 0, 7649
Flare Poller [baseline] (3.532 ms) : 0, 3532
Flare Poller [candidate] (3.484 ms) : 0, 3484
section profiling
crashtracking [baseline] (1.452 ms) : 0, 1452
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (740.671 ms) : 0, 740671
BytebuddyAgent [candidate] (729.651 ms) : 0, 729651
GlobalTracer [baseline] (224.018 ms) : 0, 224018
GlobalTracer [candidate] (221.656 ms) : 0, 221656
AppSec [baseline] (32.634 ms) : 0, 32634
AppSec [candidate] (32.173 ms) : 0, 32173
Debugger [baseline] (63.697 ms) : 0, 63697
Debugger [candidate] (62.52 ms) : 0, 62520
Remote Config [baseline] (656.223 µs) : 0, 656
Remote Config [candidate] (646.808 µs) : 0, 647
Telemetry [baseline] (8.016 ms) : 0, 8016
Telemetry [candidate] (8.03 ms) : 0, 8030
Flare Poller [baseline] (3.853 ms) : 0, 3853
Flare Poller [candidate] (3.749 ms) : 0, 3749
ProfilingAgent [baseline] (96.993 ms) : 0, 96993
ProfilingAgent [candidate] (97.143 ms) : 0, 97143
Profiling [baseline] (97.585 ms) : 0, 97585
Profiling [candidate] (97.722 ms) : 0, 97722
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/mark-flaky
git_commit_date 1763407991 1763408818
git_commit_sha b6d4e4f 56898bf
release_version 1.56.0-SNAPSHOT~b6d4e4fc4a 1.56.0-SNAPSHOT~56898bf128
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1763411310 1763411310
ci_job_id 1240187316 1240187316
ci_pipeline_id 82790005 82790005
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-isk4r4c9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-isk4r4c9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 17 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load worse
[+144.886µs; +278.330µs] or [+5.347%; +10.272%]		 worse
[+238.760µs; +635.018µs] or [+3.075%; +8.179%]		 unstable
[-231.083op/s; +61.395op/s] or [-17.344%; +4.608%]		 2.921ms 8.201ms 1247.531op/s 2.709ms 7.764ms 1332.375op/s
scenario:load:petclinic:appsec:high_load better
[-1.486ms; -0.730ms] or [-7.706%; -3.786%]		 unsure
[-1.949ms; -0.568ms] or [-6.276%; -1.830%]		 unstable
[-12.364op/s; +38.801op/s] or [-5.182%; +16.262%]		 18.174ms 29.797ms 251.812op/s 19.282ms 31.056ms 238.594op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.203 ms) : 1191, 1214
.   : milestone, 1203,
iast (3.288 ms) : 3188, 3387
.   : milestone, 3288,
iast_FULL (5.82 ms) : 5761, 5879
.   : milestone, 5820,
iast_GLOBAL (3.439 ms) : 3393, 3485
.   : milestone, 3439,
profiling (2.003 ms) : 1986, 2021
.   : milestone, 2003,
tracing (1.805 ms) : 1790, 1819
.   : milestone, 1805,
section candidate
no_agent (1.182 ms) : 1170, 1194
.   : milestone, 1182,
iast (3.187 ms) : 3151, 3224
.   : milestone, 3187,
iast_FULL (5.662 ms) : 5606, 5717
.   : milestone, 5662,
iast_GLOBAL (3.677 ms) : 3627, 3727
.   : milestone, 3677,
profiling (1.99 ms) : 1974, 2006
.   : milestone, 1990,
tracing (1.811 ms) : 1796, 1826
.   : milestone, 1811,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.203 ms [1.191 ms, 1.214 ms] -
iast 3.288 ms [3.188 ms, 3.387 ms] 2.085 ms (173.4%)
iast_FULL 5.82 ms [5.761 ms, 5.879 ms] 4.618 ms (384.0%)
iast_GLOBAL 3.439 ms [3.393 ms, 3.485 ms] 2.237 ms (186.0%)
profiling 2.003 ms [1.986 ms, 2.021 ms] 800.774 µs (66.6%)
tracing 1.805 ms [1.79 ms, 1.819 ms] 601.978 µs (50.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.182 ms [1.17 ms, 1.194 ms] -
iast 3.187 ms [3.151 ms, 3.224 ms] 2.005 ms (169.6%)
iast_FULL 5.662 ms [5.606 ms, 5.717 ms] 4.479 ms (378.8%)
iast_GLOBAL 3.677 ms [3.627 ms, 3.727 ms] 2.495 ms (211.0%)
profiling 1.99 ms [1.974 ms, 2.006 ms] 807.504 µs (68.3%)
tracing 1.811 ms [1.796 ms, 1.826 ms] 628.209 µs (53.1%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.4 ms) : 18213, 18586
.   : milestone, 18400,
appsec (19.561 ms) : 19361, 19760
.   : milestone, 19561,
code_origins (17.638 ms) : 17462, 17814
.   : milestone, 17638,
iast (17.646 ms) : 17473, 17818
.   : milestone, 17646,
profiling (18.495 ms) : 18307, 18683
.   : milestone, 18495,
tracing (17.718 ms) : 17542, 17893
.   : milestone, 17718,
section candidate
no_agent (19.454 ms) : 19254, 19654
.   : milestone, 19454,
appsec (18.538 ms) : 18352, 18724
.   : milestone, 18538,
code_origins (18.083 ms) : 17902, 18263
.   : milestone, 18083,
iast (17.786 ms) : 17611, 17962
.   : milestone, 17786,
profiling (18.614 ms) : 18423, 18804
.   : milestone, 18614,
tracing (17.301 ms) : 17131, 17470
.   : milestone, 17301,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.4 ms [18.213 ms, 18.586 ms] -
appsec 19.561 ms [19.361 ms, 19.76 ms] 1.161 ms (6.3%)
code_origins 17.638 ms [17.462 ms, 17.814 ms] -761.422 µs (-4.1%)
iast 17.646 ms [17.473 ms, 17.818 ms] -754.146 µs (-4.1%)
profiling 18.495 ms [18.307 ms, 18.683 ms] 95.174 µs (0.5%)
tracing 17.718 ms [17.542 ms, 17.893 ms] -682.123 µs (-3.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.454 ms [19.254 ms, 19.654 ms] -
appsec 18.538 ms [18.352 ms, 18.724 ms] -916.469 µs (-4.7%)
code_origins 18.083 ms [17.902 ms, 18.263 ms] -1.372 ms (-7.1%)
iast 17.786 ms [17.611 ms, 17.962 ms] -1.668 ms (-8.6%)
profiling 18.614 ms [18.423 ms, 18.804 ms] -840.716 µs (-4.3%)
tracing 17.301 ms [17.131 ms, 17.47 ms] -2.154 ms (-11.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/mark-flaky
git_commit_date 1763407991 1763408818
git_commit_sha b6d4e4f 56898bf
release_version 1.56.0-SNAPSHOT~b6d4e4fc4a 1.56.0-SNAPSHOT~56898bf128
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1763410999 1763410999
ci_job_id 1240187317 1240187317
ci_pipeline_id 82790005 82790005
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-6-8a6n9uot 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-6-8a6n9uot 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1468, 1491
.   : milestone, 1480,
appsec (3.76 ms) : 3540, 3979
.   : milestone, 3760,
iast (2.223 ms) : 2159, 2286
.   : milestone, 2223,
iast_GLOBAL (2.267 ms) : 2203, 2330
.   : milestone, 2267,
profiling (2.511 ms) : 2348, 2674
.   : milestone, 2511,
tracing (2.036 ms) : 1986, 2085
.   : milestone, 2036,
section candidate
no_agent (1.487 ms) : 1475, 1499
.   : milestone, 1487,
appsec (3.711 ms) : 3491, 3930
.   : milestone, 3711,
iast (2.23 ms) : 2167, 2294
.   : milestone, 2230,
iast_GLOBAL (2.27 ms) : 2206, 2334
.   : milestone, 2270,
profiling (2.085 ms) : 2032, 2138
.   : milestone, 2085,
tracing (2.051 ms) : 2001, 2100
.   : milestone, 2051,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.468 ms, 1.491 ms] -
appsec 3.76 ms [3.54 ms, 3.979 ms] 2.28 ms (154.0%)
iast 2.223 ms [2.159 ms, 2.286 ms] 742.93 µs (50.2%)
iast_GLOBAL 2.267 ms [2.203 ms, 2.33 ms] 786.709 µs (53.2%)
profiling 2.511 ms [2.348 ms, 2.674 ms] 1.031 ms (69.7%)
tracing 2.036 ms [1.986 ms, 2.085 ms] 555.554 µs (37.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.487 ms [1.475 ms, 1.499 ms] -
appsec 3.711 ms [3.491 ms, 3.93 ms] 2.224 ms (149.5%)
iast 2.23 ms [2.167 ms, 2.294 ms] 743.225 µs (50.0%)
iast_GLOBAL 2.27 ms [2.206 ms, 2.334 ms] 783.071 µs (52.7%)
profiling 2.085 ms [2.032 ms, 2.138 ms] 598.351 µs (40.2%)
tracing 2.051 ms [2.001 ms, 2.1 ms] 563.499 µs (37.9%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~56898bf128, baseline=1.56.0-SNAPSHOT~b6d4e4fc4a
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.008 s) : 15008000, 15008000
.   : milestone, 15008000,
appsec (15.205 s) : 15205000, 15205000
.   : milestone, 15205000,
iast (18.645 s) : 18645000, 18645000
.   : milestone, 18645000,
iast_GLOBAL (17.96 s) : 17960000, 17960000
.   : milestone, 17960000,
profiling (14.95 s) : 14950000, 14950000
.   : milestone, 14950000,
tracing (14.596 s) : 14596000, 14596000
.   : milestone, 14596000,
section candidate
no_agent (15.478 s) : 15478000, 15478000
.   : milestone, 15478000,
appsec (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
iast (18.651 s) : 18651000, 18651000
.   : milestone, 18651000,
iast_GLOBAL (17.779 s) : 17779000, 17779000
.   : milestone, 17779000,
profiling (14.934 s) : 14934000, 14934000
.   : milestone, 14934000,
tracing (14.755 s) : 14755000, 14755000
.   : milestone, 14755000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.008 s [15.008 s, 15.008 s] -
appsec 15.205 s [15.205 s, 15.205 s] 197.0 ms (1.3%)
iast 18.645 s [18.645 s, 18.645 s] 3.637 s (24.2%)
iast_GLOBAL 17.96 s [17.96 s, 17.96 s] 2.952 s (19.7%)
profiling 14.95 s [14.95 s, 14.95 s] -58.0 ms (-0.4%)
tracing 14.596 s [14.596 s, 14.596 s] -412.0 ms (-2.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.478 s [15.478 s, 15.478 s] -
appsec 14.922 s [14.922 s, 14.922 s] -556.0 ms (-3.6%)
iast 18.651 s [18.651 s, 18.651 s] 3.173 s (20.5%)
iast_GLOBAL 17.779 s [17.779 s, 17.779 s] 2.301 s (14.9%)
profiling 14.934 s [14.934 s, 14.934 s] -544.0 ms (-3.5%)
tracing 14.755 s [14.755 s, 14.755 s] -723.0 ms (-4.7%)

AlexeyKuznetsov-DD
AlexeyKuznetsov-DD reviewed Nov 17, 2025
View reviewed changes
...gger-integration-tests/src/test/java/datadog/smoketest/ExceptionDebuggerIntegrationTest.java
});
}

@Flaky
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like more tests are flaky:

  • test3CapturedFrames,
  • test3CapturedRecursiveFrames,
  • test5CapturedFrames,
  • testLambdaHiddenFrames
  • testNoSubsequentCaptureAfterFirst

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shatzi shatzi Awaiting requested review from shatzi shatzi is a code owner automatically assigned from DataDog/debugger-java

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD Awaiting requested review from AlexeyKuznetsov-DD

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jpbempel @AlexeyKuznetsov-DD