Skip to content

Mark test as flaky. #9268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 29, 2025
Merged

Mark test as flaky. #9268

merged 2 commits into from
Jul 29, 2025

Conversation

AlexeyKuznetsov-DD
Copy link
Contributor

What Does This Do

Mark test as flaky.

Motivation

Gren CI.

Additional Notes

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD self-assigned this Jul 29, 2025
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Jul 29, 2025
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD marked this pull request as ready for review July 29, 2025 13:23
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD requested a review from a team as a code owner July 29, 2025 13:23
@pr-commenter
Copy link

pr-commenter bot commented Jul 29, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/flaky-test
git_commit_date 1753791092 1753800666
git_commit_sha dc91a2c 4510874
release_version 1.51.1-SNAPSHOT~dc91a2c678 1.51.1-SNAPSHOT~4510874069
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753802469 1753802469
ci_job_id 1053113266 1053113266
ci_pipeline_id 72038244 72038244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-pcf5iahb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-pcf5iahb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1045568
Total [baseline] (8.593 s) : 0, 8592613
Agent [candidate] (1.047 s) : 0, 1046845
Total [candidate] (8.584 s) : 0, 8584348
section iast
Agent [baseline] (1.174 s) : 0, 1173630
Total [baseline] (9.312 s) : 0, 9311551
Agent [candidate] (1.179 s) : 0, 1179435
Total [candidate] (9.336 s) : 0, 9335710
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent iast 1.174 s 128.062 ms (12.2%)
Total tracing 8.593 s -
Total iast 9.312 s 718.938 ms (8.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.179 s 132.59 ms (12.7%)
Total tracing 8.584 s -
Total iast 9.336 s 751.361 ms (8.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.441 ms) : 0, 1441
crashtracking [candidate] (1.434 ms) : 0, 1434
BytebuddyAgent [baseline] (731.061 ms) : 0, 731061
BytebuddyAgent [candidate] (732.497 ms) : 0, 732497
GlobalTracer [baseline] (241.656 ms) : 0, 241656
GlobalTracer [candidate] (242.005 ms) : 0, 242005
AppSec [baseline] (30.313 ms) : 0, 30313
AppSec [candidate] (30.506 ms) : 0, 30506
Debugger [baseline] (6.057 ms) : 0, 6057
Debugger [candidate] (6.054 ms) : 0, 6054
Remote Config [baseline] (660.877 µs) : 0, 661
Remote Config [candidate] (655.075 µs) : 0, 655
Telemetry [baseline] (13.382 ms) : 0, 13382
Telemetry [candidate] (12.663 ms) : 0, 12663
section iast
crashtracking [baseline] (1.436 ms) : 0, 1436
crashtracking [candidate] (1.437 ms) : 0, 1437
BytebuddyAgent [baseline] (847.229 ms) : 0, 847229
BytebuddyAgent [candidate] (851.853 ms) : 0, 851853
GlobalTracer [baseline] (231.21 ms) : 0, 231210
GlobalTracer [candidate] (231.958 ms) : 0, 231958
IAST [baseline] (28.255 ms) : 0, 28255
IAST [candidate] (30.191 ms) : 0, 30191
AppSec [baseline] (29.159 ms) : 0, 29159
AppSec [candidate] (26.638 ms) : 0, 26638
Debugger [baseline] (6.687 ms) : 0, 6687
Debugger [candidate] (7.575 ms) : 0, 7575
Remote Config [baseline] (593.007 µs) : 0, 593
Remote Config [candidate] (597.49 µs) : 0, 597
Telemetry [baseline] (8.111 ms) : 0, 8111
Telemetry [candidate] (8.156 ms) : 0, 8156
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1044125
Total [baseline] (10.68 s) : 0, 10679873
Agent [candidate] (1.042 s) : 0, 1042398
Total [candidate] (10.696 s) : 0, 10696100
section appsec
Agent [baseline] (1.227 s) : 0, 1226878
Total [baseline] (10.814 s) : 0, 10813678
Agent [candidate] (1.219 s) : 0, 1218600
Total [candidate] (10.774 s) : 0, 10773867
section iast
Agent [baseline] (1.18 s) : 0, 1180452
Total [baseline] (10.897 s) : 0, 10896919
Agent [candidate] (1.182 s) : 0, 1182434
Total [candidate] (10.954 s) : 0, 10954263
section profiling
Agent [baseline] (1.194 s) : 0, 1194208
Total [baseline] (10.962 s) : 0, 10962336
Agent [candidate] (1.201 s) : 0, 1200547
Total [candidate] (10.854 s) : 0, 10854364
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.044 s -
Agent appsec 1.227 s 182.752 ms (17.5%)
Agent iast 1.18 s 136.326 ms (13.1%)
Agent profiling 1.194 s 150.082 ms (14.4%)
Total tracing 10.68 s -
Total appsec 10.814 s 133.806 ms (1.3%)
Total iast 10.897 s 217.046 ms (2.0%)
Total profiling 10.962 s 282.464 ms (2.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.042 s -
Agent appsec 1.219 s 176.203 ms (16.9%)
Agent iast 1.182 s 140.037 ms (13.4%)
Agent profiling 1.201 s 158.149 ms (15.2%)
Total tracing 10.696 s -
Total appsec 10.774 s 77.767 ms (0.7%)
Total iast 10.954 s 258.163 ms (2.4%)
Total profiling 10.854 s 158.264 ms (1.5%) 
gantt
    title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.436 ms) : 0, 1436
crashtracking [candidate] (1.433 ms) : 0, 1433
BytebuddyAgent [baseline] (730.597 ms) : 0, 730597
BytebuddyAgent [candidate] (730.648 ms) : 0, 730648
GlobalTracer [baseline] (241.993 ms) : 0, 241993
GlobalTracer [candidate] (241.842 ms) : 0, 241842
AppSec [baseline] (30.348 ms) : 0, 30348
AppSec [candidate] (30.19 ms) : 0, 30190
Debugger [baseline] (6.025 ms) : 0, 6025
Debugger [candidate] (6.042 ms) : 0, 6042
Remote Config [baseline] (660.258 µs) : 0, 660
Remote Config [candidate] (651.523 µs) : 0, 652
Telemetry [baseline] (11.34 ms) : 0, 11340
Telemetry [candidate] (10.559 ms) : 0, 10559
section appsec
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.435 ms) : 0, 1435
BytebuddyAgent [baseline] (758.312 ms) : 0, 758312
BytebuddyAgent [candidate] (752.595 ms) : 0, 752595
GlobalTracer [baseline] (235.654 ms) : 0, 235654
GlobalTracer [candidate] (234.401 ms) : 0, 234401
AppSec [baseline] (169.625 ms) : 0, 169625
AppSec [candidate] (170.08 ms) : 0, 170080
Debugger [baseline] (7.316 ms) : 0, 7316
Debugger [candidate] (5.71 ms) : 0, 5710
Remote Config [baseline] (620.044 µs) : 0, 620
Remote Config [candidate] (624.934 µs) : 0, 625
Telemetry [baseline] (9.04 ms) : 0, 9040
Telemetry [candidate] (8.969 ms) : 0, 8969
IAST [baseline] (23.703 ms) : 0, 23703
IAST [candidate] (23.713 ms) : 0, 23713
section iast
crashtracking [baseline] (1.439 ms) : 0, 1439
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (853.406 ms) : 0, 853406
BytebuddyAgent [candidate] (854.302 ms) : 0, 854302
GlobalTracer [baseline] (231.724 ms) : 0, 231724
GlobalTracer [candidate] (232.152 ms) : 0, 232152
AppSec [baseline] (30.52 ms) : 0, 30520
AppSec [candidate] (30.674 ms) : 0, 30674
Debugger [baseline] (5.837 ms) : 0, 5837
Debugger [candidate] (6.71 ms) : 0, 6710
Remote Config [baseline] (590.565 µs) : 0, 591
Remote Config [candidate] (616.028 µs) : 0, 616
Telemetry [baseline] (8.18 ms) : 0, 8180
Telemetry [candidate] (8.248 ms) : 0, 8248
IAST [baseline] (27.526 ms) : 0, 27526
IAST [candidate] (27.083 ms) : 0, 27083
section profiling
ProfilingAgent [baseline] (108.214 ms) : 0, 108214
ProfilingAgent [candidate] (109.377 ms) : 0, 109377
crashtracking [baseline] (1.411 ms) : 0, 1411
crashtracking [candidate] (1.418 ms) : 0, 1418
BytebuddyAgent [baseline] (761.508 ms) : 0, 761508
BytebuddyAgent [candidate] (765.201 ms) : 0, 765201
GlobalTracer [baseline] (220.547 ms) : 0, 220547
GlobalTracer [candidate] (221.805 ms) : 0, 221805
AppSec [baseline] (30.221 ms) : 0, 30221
AppSec [candidate] (30.471 ms) : 0, 30471
Debugger [baseline] (6.291 ms) : 0, 6291
Debugger [candidate] (6.368 ms) : 0, 6368
Remote Config [baseline] (697.667 µs) : 0, 698
Remote Config [candidate] (692.028 µs) : 0, 692
Telemetry [baseline] (16.053 ms) : 0, 16053
Telemetry [candidate] (15.489 ms) : 0, 15489
Profiling [baseline] (108.875 ms) : 0, 108875
Profiling [candidate] (110.014 ms) : 0, 110014
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/flaky-test
git_commit_date 1753791092 1753800666
git_commit_sha dc91a2c 4510874
release_version 1.51.1-SNAPSHOT~dc91a2c678 1.51.1-SNAPSHOT~4510874069
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1753802070 1753802070
ci_job_id 1053113267 1053113267
ci_pipeline_id 72038244 72038244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-z4s90ofj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-z4s90ofj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 2 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:profiling:high_load worse
[+186.808µs; +487.218µs] or [+2.123%; +5.536%]		 unstable
[-83.235op/s; +44.798op/s] or [-15.795%; +8.501%]		 9.138ms 507.750op/s 8.801ms 526.969op/s
scenario:load:insecure-bank:no_agent:high_load better
[-328.188µs; -215.138µs] or [-7.160%; -4.693%]		 unstable
[-50.576op/s; +174.326op/s] or [-5.048%; +17.398%]		 4.312ms 1063.875op/s 4.584ms 1002.000op/s
scenario:load:petclinic:iast:high_load better
[-2.315ms; -1.495ms] or [-5.165%; -3.335%]		 unstable
[-2.949op/s; +12.174op/s] or [-2.825%; +11.664%]		 42.914ms 108.987op/s 44.819ms 104.375op/s
scenario:load:petclinic:appsec:high_load worse
[+1.819ms; +2.738ms] or [+3.890%; +5.853%]		 unstable
[-11.419op/s; +2.144op/s] or [-11.413%; +2.143%]		 49.051ms 95.412op/s 46.773ms 100.050op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.292 ms) : 35999, 36584
.   : milestone, 36292,
appsec (46.773 ms) : 46357, 47188
.   : milestone, 46773,
code_origins (46.211 ms) : 45809, 46613
.   : milestone, 46211,
iast (44.819 ms) : 44434, 45203
.   : milestone, 44819,
profiling (47.796 ms) : 47335, 48256
.   : milestone, 47796,
tracing (42.481 ms) : 42135, 42828
.   : milestone, 42481,
section candidate
no_agent (36.716 ms) : 36433, 36999
.   : milestone, 36716,
appsec (49.051 ms) : 48614, 49489
.   : milestone, 49051,
code_origins (45.278 ms) : 44890, 45665
.   : milestone, 45278,
iast (42.914 ms) : 42536, 43291
.   : milestone, 42914,
profiling (47.043 ms) : 46558, 47528
.   : milestone, 47043,
tracing (43.362 ms) : 42996, 43728
.   : milestone, 43362,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.292 ms [35.999 ms, 36.584 ms] -
appsec 46.773 ms [46.357 ms, 47.188 ms] 10.481 ms (28.9%)
code_origins 46.211 ms [45.809 ms, 46.613 ms] 9.919 ms (27.3%)
iast 44.819 ms [44.434 ms, 45.203 ms] 8.527 ms (23.5%)
profiling 47.796 ms [47.335 ms, 48.256 ms] 11.504 ms (31.7%)
tracing 42.481 ms [42.135 ms, 42.828 ms] 6.19 ms (17.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.716 ms [36.433 ms, 36.999 ms] -
appsec 49.051 ms [48.614 ms, 49.489 ms] 12.335 ms (33.6%)
code_origins 45.278 ms [44.89 ms, 45.665 ms] 8.562 ms (23.3%)
iast 42.914 ms [42.536 ms, 43.291 ms] 6.198 ms (16.9%)
profiling 47.043 ms [46.558 ms, 47.528 ms] 10.327 ms (28.1%)
tracing 43.362 ms [42.996 ms, 43.728 ms] 6.646 ms (18.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.584 ms) : 4533, 4635
.   : milestone, 4584,
iast (9.48 ms) : 9326, 9634
.   : milestone, 9480,
iast_FULL (14.0 ms) : 13729, 14271
.   : milestone, 14000,
iast_GLOBAL (10.687 ms) : 10491, 10882
.   : milestone, 10687,
profiling (8.801 ms) : 8661, 8942
.   : milestone, 8801,
tracing (7.531 ms) : 7426, 7636
.   : milestone, 7531,
section candidate
no_agent (4.312 ms) : 4258, 4366
.   : milestone, 4312,
iast (9.428 ms) : 9277, 9580
.   : milestone, 9428,
iast_FULL (13.869 ms) : 13597, 14140
.   : milestone, 13869,
iast_GLOBAL (10.403 ms) : 10218, 10587
.   : milestone, 10403,
profiling (9.138 ms) : 8999, 9277
.   : milestone, 9138,
tracing (7.399 ms) : 7296, 7502
.   : milestone, 7399,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.584 ms [4.533 ms, 4.635 ms] -
iast 9.48 ms [9.326 ms, 9.634 ms] 4.896 ms (106.8%)
iast_FULL 14.0 ms [13.729 ms, 14.271 ms] 9.416 ms (205.4%)
iast_GLOBAL 10.687 ms [10.491 ms, 10.882 ms] 6.103 ms (133.1%)
profiling 8.801 ms [8.661 ms, 8.942 ms] 4.217 ms (92.0%)
tracing 7.531 ms [7.426 ms, 7.636 ms] 2.947 ms (64.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.312 ms [4.258 ms, 4.366 ms] -
iast 9.428 ms [9.277 ms, 9.58 ms] 5.116 ms (118.6%)
iast_FULL 13.869 ms [13.597 ms, 14.14 ms] 9.556 ms (221.6%)
iast_GLOBAL 10.403 ms [10.218 ms, 10.587 ms] 6.09 ms (141.2%)
profiling 9.138 ms [8.999 ms, 9.277 ms] 4.826 ms (111.9%)
tracing 7.399 ms [7.296 ms, 7.502 ms] 3.087 ms (71.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/flaky-test
git_commit_date 1753791092 1753800666
git_commit_sha dc91a2c 4510874
release_version 1.51.1-SNAPSHOT~dc91a2c678 1.51.1-SNAPSHOT~4510874069
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1753802624 1753802624
ci_job_id 1053113268 1053113268
ci_pipeline_id 72038244 72038244
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-c0uoywit 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-c0uoywit 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.956 s) : 14956000, 14956000
.   : milestone, 14956000,
appsec (14.81 s) : 14810000, 14810000
.   : milestone, 14810000,
iast (18.083 s) : 18083000, 18083000
.   : milestone, 18083000,
iast_GLOBAL (18.203 s) : 18203000, 18203000
.   : milestone, 18203000,
profiling (15.341 s) : 15341000, 15341000
.   : milestone, 15341000,
tracing (15.041 s) : 15041000, 15041000
.   : milestone, 15041000,
section candidate
no_agent (15.679 s) : 15679000, 15679000
.   : milestone, 15679000,
appsec (14.829 s) : 14829000, 14829000
.   : milestone, 14829000,
iast (18.593 s) : 18593000, 18593000
.   : milestone, 18593000,
iast_GLOBAL (18.028 s) : 18028000, 18028000
.   : milestone, 18028000,
profiling (15.748 s) : 15748000, 15748000
.   : milestone, 15748000,
tracing (15.084 s) : 15084000, 15084000
.   : milestone, 15084000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.956 s [14.956 s, 14.956 s] -
appsec 14.81 s [14.81 s, 14.81 s] -146.0 ms (-1.0%)
iast 18.083 s [18.083 s, 18.083 s] 3.127 s (20.9%)
iast_GLOBAL 18.203 s [18.203 s, 18.203 s] 3.247 s (21.7%)
profiling 15.341 s [15.341 s, 15.341 s] 385.0 ms (2.6%)
tracing 15.041 s [15.041 s, 15.041 s] 85.0 ms (0.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.679 s [15.679 s, 15.679 s] -
appsec 14.829 s [14.829 s, 14.829 s] -850.0 ms (-5.4%)
iast 18.593 s [18.593 s, 18.593 s] 2.914 s (18.6%)
iast_GLOBAL 18.028 s [18.028 s, 18.028 s] 2.349 s (15.0%)
profiling 15.748 s [15.748 s, 15.748 s] 69.0 ms (0.4%)
tracing 15.084 s [15.084 s, 15.084 s] -595.0 ms (-3.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~4510874069, baseline=1.51.1-SNAPSHOT~dc91a2c678
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1485
.   : milestone, 1473,
appsec (3.633 ms) : 3416, 3850
.   : milestone, 3633,
iast (2.184 ms) : 2122, 2247
.   : milestone, 2184,
iast_GLOBAL (2.226 ms) : 2164, 2289
.   : milestone, 2226,
profiling (2.036 ms) : 1985, 2086
.   : milestone, 2036,
tracing (2.01 ms) : 1961, 2058
.   : milestone, 2010,
section candidate
no_agent (1.473 ms) : 1461, 1485
.   : milestone, 1473,
appsec (3.576 ms) : 3364, 3788
.   : milestone, 3576,
iast (2.195 ms) : 2133, 2258
.   : milestone, 2195,
iast_GLOBAL (2.225 ms) : 2163, 2288
.   : milestone, 2225,
profiling (2.047 ms) : 1996, 2098
.   : milestone, 2047,
tracing (2.01 ms) : 1961, 2058
.   : milestone, 2010,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.461 ms, 1.485 ms] -
appsec 3.633 ms [3.416 ms, 3.85 ms] 2.16 ms (146.7%)
iast 2.184 ms [2.122 ms, 2.247 ms] 711.52 µs (48.3%)
iast_GLOBAL 2.226 ms [2.164 ms, 2.289 ms] 753.419 µs (51.1%)
profiling 2.036 ms [1.985 ms, 2.086 ms] 562.754 µs (38.2%)
tracing 2.01 ms [1.961 ms, 2.058 ms] 536.693 µs (36.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.461 ms, 1.485 ms] -
appsec 3.576 ms [3.364 ms, 3.788 ms] 2.103 ms (142.8%)
iast 2.195 ms [2.133 ms, 2.258 ms] 722.181 µs (49.0%)
iast_GLOBAL 2.225 ms [2.163 ms, 2.288 ms] 752.452 µs (51.1%)
profiling 2.047 ms [1.996 ms, 2.098 ms] 573.633 µs (38.9%)
tracing 2.01 ms [1.961 ms, 2.058 ms] 536.771 µs (36.4%)

sarahchen6
sarahchen6 approved these changes Jul 29, 2025
View reviewed changes
Copy link
Contributor

@sarahchen6 sarahchen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! If you know the reason for the flake or if it's waiting on a team to address, it may help to include that too in the @Flaky("description").

@AlexeyKuznetsov-DD
Copy link
Contributor Author

Thanks! If you know the reason for the flake or if it's waiting on a team to address, it may help to include that too in the @Flaky("description").

Good catch! Will update!

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD merged commit 032aad2 into master Jul 29, 2025
504 checks passed
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD deleted the alexeyk/flaky-test branch July 29, 2025 15:29
@github-actions github-actions bot added this to the 1.52.0 milestone Jul 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sarahchen6 sarahchen6 sarahchen6 approved these changes

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer is a code owner automatically assigned from DataDog/apm-java

@smola smola Awaiting requested review from smola

Assignees

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD

Labels
comp: testing Testing tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.52.0
Development

Successfully merging this pull request may close these issues.
2 participants
@AlexeyKuznetsov-DD @sarahchen6