add recipe for handling flaky tests #9020

emmettbutler · 2025-06-20T19:29:49Z

What Does This Do

This change adds some documentation explaining how to handle flaky tests when they occur on pull requests. It also removes a few outdated references to CircleCI.

Motivation

The motivation is to make it easier for new contributors to get up to speed on the project.

Jira ticket: LANGPLAT-352

github-actions · 2025-06-20T19:29:59Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Remove the tag from the pull request title

If you need help, please check our contributing guidelines.

pr-commenter · 2025-06-20T20:03:19Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	emmett.butler/flaky-guidance
git_commit_date	1750427857	1750447681
git_commit_sha	`60c30c2`	`f8ec06c`
release_version	1.51.0-SNAPSHOT~60c30c276e	1.51.0-SNAPSHOT~f8ec06c202

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750449582	1750449582
ci_job_id	992078701	992078701
ci_pipeline_id	68361068	68361068
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-clrnfvyp-project-304-concurrent-0-4979j6gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-clrnfvyp-project-304-concurrent-0-4979j6gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 43 metrics, 10 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1025897
Total [baseline] (10.504 s) : 0, 10503768
Agent [candidate] (1.025 s) : 0, 1025436
Total [candidate] (10.643 s) : 0, 10642590
section appsec
Agent [baseline] (1.188 s) : 0, 1187607
Total [baseline] (10.833 s) : 0, 10832560
Agent [candidate] (1.185 s) : 0, 1185209
Total [candidate] (10.773 s) : 0, 10772905
section iast
Agent [baseline] (1.151 s) : 0, 1151015
Total [baseline] (10.848 s) : 0, 10847997
Agent [candidate] (1.161 s) : 0, 1160964
Total [candidate] (10.864 s) : 0, 10863621
section profiling
Agent [baseline] (1.268 s) : 0, 1268441
Total [baseline] (10.952 s) : 0, 10951651
Agent [candidate] (1.273 s) : 0, 1272572
Total [candidate] (10.991 s) : 0, 10990852

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.026 s	-
Agent	appsec	1.188 s	161.71 ms (15.8%)
Agent	iast	1.151 s	125.118 ms (12.2%)
Agent	profiling	1.268 s	242.543 ms (23.6%)
Total	tracing	10.504 s	-
Total	appsec	10.833 s	328.792 ms (3.1%)
Total	iast	10.848 s	344.229 ms (3.3%)
Total	profiling	10.952 s	447.883 ms (4.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.025 s	-
Agent	appsec	1.185 s	159.773 ms (15.6%)
Agent	iast	1.161 s	135.528 ms (13.2%)
Agent	profiling	1.273 s	247.136 ms (24.1%)
Total	tracing	10.643 s	-
Total	appsec	10.773 s	130.315 ms (1.2%)
Total	iast	10.864 s	221.031 ms (2.1%)
Total	profiling	10.991 s	348.263 ms (3.3%)

gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.68 ms) : 0, 686680
BytebuddyAgent [candidate] (685.804 ms) : 0, 685804
GlobalTracer [baseline] (242.344 ms) : 0, 242344
GlobalTracer [candidate] (242.107 ms) : 0, 242107
AppSec [baseline] (61.238 ms) : 0, 61238
AppSec [candidate] (58.359 ms) : 0, 58359
Debugger [baseline] (6.097 ms) : 0, 6097
Debugger [candidate] (7.007 ms) : 0, 7007
Remote Config [baseline] (645.145 µs) : 0, 645
Remote Config [candidate] (669.307 µs) : 0, 669
Telemetry [baseline] (8.003 ms) : 0, 8003
Telemetry [candidate] (10.641 ms) : 0, 10641
section appsec
BytebuddyAgent [baseline] (714.341 ms) : 0, 714341
BytebuddyAgent [candidate] (713.031 ms) : 0, 713031
GlobalTracer [baseline] (236.799 ms) : 0, 236799
GlobalTracer [candidate] (236.308 ms) : 0, 236308
IAST [baseline] (22.015 ms) : 0, 22015
IAST [candidate] (22.129 ms) : 0, 22129
AppSec [baseline] (179.877 ms) : 0, 179877
AppSec [candidate] (179.283 ms) : 0, 179283
Debugger [baseline] (5.809 ms) : 0, 5809
Debugger [candidate] (5.77 ms) : 0, 5770
Remote Config [baseline] (604.759 µs) : 0, 605
Remote Config [candidate] (606.005 µs) : 0, 606
Telemetry [baseline] (7.265 ms) : 0, 7265
Telemetry [candidate] (7.216 ms) : 0, 7216
section iast
BytebuddyAgent [baseline] (804.712 ms) : 0, 804712
BytebuddyAgent [candidate] (812.268 ms) : 0, 812268
GlobalTracer [baseline] (231.61 ms) : 0, 231610
GlobalTracer [candidate] (233.392 ms) : 0, 233392
IAST [baseline] (25.285 ms) : 0, 25285
IAST [candidate] (27.029 ms) : 0, 27029
AppSec [baseline] (54.034 ms) : 0, 54034
AppSec [candidate] (52.633 ms) : 0, 52633
Debugger [baseline] (5.927 ms) : 0, 5927
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (601.135 µs) : 0, 601
Remote Config [candidate] (629.193 µs) : 0, 629
Telemetry [baseline] (7.972 ms) : 0, 7972
Telemetry [candidate] (8.044 ms) : 0, 8044
section profiling
BytebuddyAgent [baseline] (676.044 ms) : 0, 676044
BytebuddyAgent [candidate] (677.589 ms) : 0, 677589
GlobalTracer [baseline] (360.592 ms) : 0, 360592
GlobalTracer [candidate] (361.396 ms) : 0, 361396
AppSec [baseline] (62.258 ms) : 0, 62258
AppSec [candidate] (62.303 ms) : 0, 62303
Debugger [baseline] (6.09 ms) : 0, 6090
Debugger [candidate] (6.097 ms) : 0, 6097
Remote Config [baseline] (705.341 µs) : 0, 705
Remote Config [candidate] (706.053 µs) : 0, 706
Telemetry [baseline] (8.06 ms) : 0, 8060
Telemetry [candidate] (8.135 ms) : 0, 8135
ProfilingAgent [baseline] (106.325 ms) : 0, 106325
ProfilingAgent [candidate] (107.586 ms) : 0, 107586
Profiling [baseline] (106.35 ms) : 0, 106350
Profiling [candidate] (107.612 ms) : 0, 107612

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1030421
Total [baseline] (8.552 s) : 0, 8551769
Agent [candidate] (1.025 s) : 0, 1025058
Total [candidate] (8.568 s) : 0, 8567647
section iast
Agent [baseline] (1.148 s) : 0, 1147624
Total [baseline] (9.204 s) : 0, 9203944
Agent [candidate] (1.152 s) : 0, 1152074
Total [candidate] (9.219 s) : 0, 9219466

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	iast	1.148 s	117.203 ms (11.4%)
Total	tracing	8.552 s	-
Total	iast	9.204 s	652.176 ms (7.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.025 s	-
Agent	iast	1.152 s	127.015 ms (12.4%)
Total	tracing	8.568 s	-
Total	iast	9.219 s	651.819 ms (7.6%)

gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.091 ms) : 0, 691091
BytebuddyAgent [candidate] (685.648 ms) : 0, 685648
GlobalTracer [baseline] (242.007 ms) : 0, 242007
GlobalTracer [candidate] (242.091 ms) : 0, 242091
AppSec [baseline] (55.663 ms) : 0, 55663
AppSec [candidate] (58.542 ms) : 0, 58542
Debugger [baseline] (7.053 ms) : 0, 7053
Debugger [candidate] (7.611 ms) : 0, 7611
Remote Config [baseline] (697.43 µs) : 0, 697
Remote Config [candidate] (657.787 µs) : 0, 658
Telemetry [baseline] (12.886 ms) : 0, 12886
Telemetry [candidate] (9.628 ms) : 0, 9628
section iast
BytebuddyAgent [baseline] (802.759 ms) : 0, 802759
BytebuddyAgent [candidate] (805.568 ms) : 0, 805568
GlobalTracer [baseline] (231.144 ms) : 0, 231144
GlobalTracer [candidate] (232.076 ms) : 0, 232076
IAST [baseline] (27.331 ms) : 0, 27331
IAST [candidate] (26.119 ms) : 0, 26119
AppSec [baseline] (51.178 ms) : 0, 51178
AppSec [candidate] (53.023 ms) : 0, 53023
Debugger [baseline] (5.835 ms) : 0, 5835
Debugger [candidate] (5.932 ms) : 0, 5932
Remote Config [baseline] (602.001 µs) : 0, 602
Remote Config [candidate] (597.377 µs) : 0, 597
Telemetry [baseline] (7.973 ms) : 0, 7973
Telemetry [candidate] (7.96 ms) : 0, 7960

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	emmett.butler/flaky-guidance
git_commit_date	1750427857	1750447681
git_commit_sha	`60c30c2`	`f8ec06c`
release_version	1.51.0-SNAPSHOT~60c30c276e	1.51.0-SNAPSHOT~f8ec06c202

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750449252	1750449252
ci_job_id	992078702	992078702
ci_pipeline_id	68361068	68361068
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-bziamzy-project-304-concurrent-0-kviyhrps 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-bziamzy-project-304-concurrent-0-kviyhrps 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	better [-560.820µs; -235.932µs] or [-5.950%; -2.503%]	unstable [-48.791op/s; +91.354op/s] or [-9.906%; +18.547%]	9.027ms	513.844op/s	9.425ms	492.562op/s
scenario:load:insecure-bank:tracing:high_load	better [-605.283µs; -349.509µs] or [-7.449%; -4.301%]	unstable [-46.962op/s; +116.524op/s] or [-8.232%; +20.426%]	7.648ms	605.250op/s	8.125ms	570.469op/s
scenario:load:petclinic:appsec:high_load	worse [+1.484ms; +2.357ms] or [+3.231%; +5.131%]	unstable [-10.528op/s; +2.353op/s] or [-10.339%; +2.311%]	47.869ms	97.737op/s	45.948ms	101.825op/s
scenario:load:petclinic:iast:high_load	better [-2.365ms; -1.548ms] or [-5.258%; -3.443%]	unstable [-2.544op/s; +11.969op/s] or [-2.446%; +11.509%]	43.020ms	108.713op/s	44.977ms	104.000op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.529 ms) : 37223, 37835
.   : milestone, 37529,
appsec (45.948 ms) : 45558, 46338
.   : milestone, 45948,
code_origins (44.91 ms) : 44527, 45293
.   : milestone, 44910,
iast (44.977 ms) : 44581, 45373
.   : milestone, 44977,
profiling (48.746 ms) : 48309, 49182
.   : milestone, 48746,
tracing (42.558 ms) : 42226, 42891
.   : milestone, 42558,
section candidate
no_agent (36.583 ms) : 36293, 36873
.   : milestone, 36583,
appsec (47.869 ms) : 47448, 48290
.   : milestone, 47869,
code_origins (44.016 ms) : 43649, 44383
.   : milestone, 44016,
iast (43.02 ms) : 42658, 43382
.   : milestone, 43020,
profiling (48.052 ms) : 47584, 48521
.   : milestone, 48052,
tracing (42.79 ms) : 42450, 43130
.   : milestone, 42790,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.529 ms [37.223 ms, 37.835 ms]	-
appsec	45.948 ms [45.558 ms, 46.338 ms]	8.419 ms (22.4%)
code_origins	44.91 ms [44.527 ms, 45.293 ms]	7.381 ms (19.7%)
iast	44.977 ms [44.581 ms, 45.373 ms]	7.448 ms (19.8%)
profiling	48.746 ms [48.309 ms, 49.182 ms]	11.217 ms (29.9%)
tracing	42.558 ms [42.226 ms, 42.891 ms]	5.03 ms (13.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.583 ms [36.293 ms, 36.873 ms]	-
appsec	47.869 ms [47.448 ms, 48.29 ms]	11.286 ms (30.9%)
code_origins	44.016 ms [43.649 ms, 44.383 ms]	7.433 ms (20.3%)
iast	43.02 ms [42.658 ms, 43.382 ms]	6.437 ms (17.6%)
profiling	48.052 ms [47.584 ms, 48.521 ms]	11.469 ms (31.4%)
tracing	42.79 ms [42.45 ms, 43.13 ms]	6.207 ms (17.0%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.387 ms) : 4337, 4436
.   : milestone, 4387,
iast (9.425 ms) : 9269, 9581
.   : milestone, 9425,
iast_FULL (13.818 ms) : 13544, 14091
.   : milestone, 13818,
iast_GLOBAL (9.898 ms) : 9726, 10070
.   : milestone, 9898,
profiling (8.476 ms) : 8336, 8615
.   : milestone, 8476,
tracing (8.125 ms) : 8000, 8251
.   : milestone, 8125,
section candidate
no_agent (4.468 ms) : 4414, 4523
.   : milestone, 4468,
iast (9.027 ms) : 8880, 9173
.   : milestone, 9027,
iast_FULL (14.096 ms) : 13812, 14379
.   : milestone, 14096,
iast_GLOBAL (10.207 ms) : 10025, 10389
.   : milestone, 10207,
profiling (8.772 ms) : 8632, 8911
.   : milestone, 8772,
tracing (7.648 ms) : 7536, 7759
.   : milestone, 7648,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.387 ms [4.337 ms, 4.436 ms]	-
iast	9.425 ms [9.269 ms, 9.581 ms]	5.038 ms (114.9%)
iast_FULL	13.818 ms [13.544 ms, 14.091 ms]	9.431 ms (215.0%)
iast_GLOBAL	9.898 ms [9.726 ms, 10.07 ms]	5.511 ms (125.6%)
profiling	8.476 ms [8.336 ms, 8.615 ms]	4.089 ms (93.2%)
tracing	8.125 ms [8.0 ms, 8.251 ms]	3.739 ms (85.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.468 ms [4.414 ms, 4.523 ms]	-
iast	9.027 ms [8.88 ms, 9.173 ms]	4.558 ms (102.0%)
iast_FULL	14.096 ms [13.812 ms, 14.379 ms]	9.628 ms (215.5%)
iast_GLOBAL	10.207 ms [10.025 ms, 10.389 ms]	5.739 ms (128.4%)
profiling	8.772 ms [8.632 ms, 8.911 ms]	4.304 ms (96.3%)
tracing	7.648 ms [7.536 ms, 7.759 ms]	3.18 ms (71.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	emmett.butler/flaky-guidance
git_commit_date	1750427857	1750447681
git_commit_sha	`60c30c2`	`f8ec06c`
release_version	1.51.0-SNAPSHOT~60c30c276e	1.51.0-SNAPSHOT~f8ec06c202

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1750449763	1750449763
ci_job_id	992078703	992078703
ci_pipeline_id	68361068	68361068
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-bziamzy-project-304-concurrent-1-fv58j1gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-bziamzy-project-304-concurrent-1-fv58j1gk 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.59 s) : 15590000, 15590000
.   : milestone, 15590000,
appsec (15.101 s) : 15101000, 15101000
.   : milestone, 15101000,
iast (18.597 s) : 18597000, 18597000
.   : milestone, 18597000,
iast_GLOBAL (18.075 s) : 18075000, 18075000
.   : milestone, 18075000,
profiling (15.369 s) : 15369000, 15369000
.   : milestone, 15369000,
tracing (15.156 s) : 15156000, 15156000
.   : milestone, 15156000,
section candidate
no_agent (14.768 s) : 14768000, 14768000
.   : milestone, 14768000,
appsec (14.961 s) : 14961000, 14961000
.   : milestone, 14961000,
iast (18.938 s) : 18938000, 18938000
.   : milestone, 18938000,
iast_GLOBAL (17.98 s) : 17980000, 17980000
.   : milestone, 17980000,
profiling (15.309 s) : 15309000, 15309000
.   : milestone, 15309000,
tracing (14.932 s) : 14932000, 14932000
.   : milestone, 14932000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.59 s [15.59 s, 15.59 s]	-
appsec	15.101 s [15.101 s, 15.101 s]	-489.0 ms (-3.1%)
iast	18.597 s [18.597 s, 18.597 s]	3.007 s (19.3%)
iast_GLOBAL	18.075 s [18.075 s, 18.075 s]	2.485 s (15.9%)
profiling	15.369 s [15.369 s, 15.369 s]	-221.0 ms (-1.4%)
tracing	15.156 s [15.156 s, 15.156 s]	-434.0 ms (-2.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.768 s [14.768 s, 14.768 s]	-
appsec	14.961 s [14.961 s, 14.961 s]	193.0 ms (1.3%)
iast	18.938 s [18.938 s, 18.938 s]	4.17 s (28.2%)
iast_GLOBAL	17.98 s [17.98 s, 17.98 s]	3.212 s (21.7%)
profiling	15.309 s [15.309 s, 15.309 s]	541.0 ms (3.7%)
tracing	14.932 s [14.932 s, 14.932 s]	164.0 ms (1.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~f8ec06c202, baseline=1.51.0-SNAPSHOT~60c30c276e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (2.396 ms) : 2347, 2444
.   : milestone, 2396,
iast (2.194 ms) : 2132, 2255
.   : milestone, 2194,
iast_GLOBAL (2.222 ms) : 2160, 2283
.   : milestone, 2222,
profiling (2.048 ms) : 1998, 2099
.   : milestone, 2048,
tracing (2.009 ms) : 1961, 2056
.   : milestone, 2009,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.393 ms) : 2345, 2442
.   : milestone, 2393,
iast (2.192 ms) : 2130, 2253
.   : milestone, 2192,
iast_GLOBAL (2.227 ms) : 2165, 2289
.   : milestone, 2227,
profiling (2.035 ms) : 1986, 2085
.   : milestone, 2035,
tracing (1.996 ms) : 1949, 2043
.   : milestone, 1996,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	2.396 ms [2.347 ms, 2.444 ms]	921.382 µs (62.5%)
iast	2.194 ms [2.132 ms, 2.255 ms]	719.104 µs (48.8%)
iast_GLOBAL	2.222 ms [2.16 ms, 2.283 ms]	747.106 µs (50.7%)
profiling	2.048 ms [1.998 ms, 2.099 ms]	573.892 µs (38.9%)
tracing	2.009 ms [1.961 ms, 2.056 ms]	534.218 µs (36.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.464 ms, 1.487 ms]	-
appsec	2.393 ms [2.345 ms, 2.442 ms]	918.427 µs (62.3%)
iast	2.192 ms [2.13 ms, 2.253 ms]	716.564 µs (48.6%)
iast_GLOBAL	2.227 ms [2.165 ms, 2.289 ms]	751.884 µs (51.0%)
profiling	2.035 ms [1.986 ms, 2.085 ms]	560.378 µs (38.0%)
tracing	1.996 ms [1.949 ms, 2.043 ms]	521.097 µs (35.3%)

add recipe for handling flaky tests

f8ec06c

emmettbutler requested a review from a team as a code owner June 20, 2025 19:29

emmettbutler requested a review from PerfectSlayer June 20, 2025 19:29

mcculls added type: documentation comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes labels Jun 20, 2025

mcculls changed the title ~~add recipe for handling flaky tests [LANGPLAT-352]~~ add recipe for handling flaky tests Jun 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add recipe for handling flaky tests #9020

add recipe for handling flaky tests #9020

Uh oh!

emmettbutler commented Jun 20, 2025 •

edited by jira bot

Loading

Uh oh!

github-actions bot commented Jun 20, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jun 20, 2025

Uh oh!

Uh oh!

add recipe for handling flaky tests #9020

Are you sure you want to change the base?

add recipe for handling flaky tests #9020

Uh oh!

Conversation

emmettbutler commented Jun 20, 2025 • edited by jira bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Uh oh!

github-actions bot commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Jun 20, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

emmettbutler commented Jun 20, 2025 •

edited by jira bot

Loading

github-actions bot commented Jun 20, 2025 •

edited

Loading