Skip to content

Fix Maven smoke test timeout in CI #8993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 16, 2025
Merged

Fix Maven smoke test timeout in CI #8993

merged 1 commit into from
Jun 16, 2025

Conversation

bric3
Copy link
Contributor

@bric3 bric3 commented Jun 16, 2025

What Does This Do

Increases the maven smoke test timeout for CI. Numerous maven smoke tests are failing in CI, du to process timeout. Locally the tests are passing, let's increase a tad the timeouts.

java.util.concurrent.TimeoutException: Instrumented process failed to exit
	at datadog.smoketest.MavenSmokeTest.runProcess(MavenSmokeTest.groovy:336)
	at datadog.smoketest.MavenSmokeTest.retryUntilSuccessfulOrNoAttemptsLeft(MavenSmokeTest.groovy:312)
	at datadog.smoketest.MavenSmokeTest.givenMavenDependenciesAreLoaded(MavenSmokeTest.groovy:294)
	at datadog.smoketest.MavenSmokeTest.test #projectName, v#mavenVersion(MavenSmokeTest.groovy:65)

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@bric3 bric3 added the tag: no release notes Changes to exclude from release notes label Jun 16, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jun 16, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master bdu/increasemaven-smoke-test-timeout
git_commit_date 1750076840 1750083912
git_commit_sha 9db71d4 9efa172
release_version 1.50.0-SNAPSHOT~9db71d4352 1.50.0-SNAPSHOT~9efa17209f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1750086037 1750086037
ci_job_id 983615637 983615637
ci_pipeline_id 67851301 67851301
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-z87jatgw-project-304-concurrent-0-dg6a855z 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-z87jatgw-project-304-concurrent-0-dg6a855z 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1030228
Total [baseline] (8.537 s) : 0, 8536842
Agent [candidate] (1.027 s) : 0, 1027087
Total [candidate] (8.527 s) : 0, 8526713
section iast
Agent [baseline] (1.156 s) : 0, 1156352
Total [baseline] (9.215 s) : 0, 9215031
Agent [candidate] (1.162 s) : 0, 1162034
Total [candidate] (9.268 s) : 0, 9267711
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.164 s) : 0, 1163926
Total [baseline] (9.182 s) : 0, 9181700
Agent [candidate] (1.157 s) : 0, 1156742
Total [candidate] (9.214 s) : 0, 9214170
section iast_TELEMETRY_OFF
Agent [baseline] (1.15 s) : 0, 1150062
Total [baseline] (9.208 s) : 0, 9207710
Agent [candidate] (1.155 s) : 0, 1154535
Total [candidate] (9.199 s) : 0, 9198904
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.03 s -
Agent iast 1.156 s 126.124 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.164 s 133.698 ms (13.0%)
Agent iast_TELEMETRY_OFF 1.15 s 119.834 ms (11.6%)
Total tracing 8.537 s -
Total iast 9.215 s 678.189 ms (7.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.182 s 644.858 ms (7.6%)
Total iast_TELEMETRY_OFF 9.208 s 670.868 ms (7.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent iast 1.162 s 134.947 ms (13.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.157 s 129.655 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.155 s 127.448 ms (12.4%)
Total tracing 8.527 s -
Total iast 9.268 s 740.998 ms (8.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.214 s 687.458 ms (8.1%)
Total iast_TELEMETRY_OFF 9.199 s 672.191 ms (7.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.22 ms) : 0, 685220
BytebuddyAgent [candidate] (683.884 ms) : 0, 683884
GlobalTracer [baseline] (242.53 ms) : 0, 242530
GlobalTracer [candidate] (241.287 ms) : 0, 241287
AppSec [baseline] (59.705 ms) : 0, 59705
AppSec [candidate] (59.337 ms) : 0, 59337
Debugger [baseline] (6.228 ms) : 0, 6228
Debugger [candidate] (6.228 ms) : 0, 6228
Remote Config [baseline] (758.284 µs) : 0, 758
Remote Config [candidate] (756.494 µs) : 0, 756
Telemetry [baseline] (12.28 ms) : 0, 12280
Telemetry [candidate] (12.098 ms) : 0, 12098
section iast
BytebuddyAgent [baseline] (804.753 ms) : 0, 804753
BytebuddyAgent [candidate] (809.187 ms) : 0, 809187
GlobalTracer [baseline] (231.403 ms) : 0, 231403
GlobalTracer [candidate] (232.313 ms) : 0, 232313
IAST [baseline] (26.164 ms) : 0, 26164
IAST [candidate] (25.423 ms) : 0, 25423
AppSec [baseline] (56.085 ms) : 0, 56085
AppSec [candidate] (56.884 ms) : 0, 56884
Debugger [baseline] (5.985 ms) : 0, 5985
Debugger [candidate] (5.999 ms) : 0, 5999
Remote Config [baseline] (604.557 µs) : 0, 605
Remote Config [candidate] (597.818 µs) : 0, 598
Telemetry [baseline] (7.917 ms) : 0, 7917
Telemetry [candidate] (7.989 ms) : 0, 7989
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (810.149 ms) : 0, 810149
BytebuddyAgent [candidate] (804.591 ms) : 0, 804591
GlobalTracer [baseline] (233.104 ms) : 0, 233104
GlobalTracer [candidate] (232.227 ms) : 0, 232227
IAST [baseline] (27.381 ms) : 0, 27381
IAST [candidate] (28.818 ms) : 0, 28818
AppSec [baseline] (55.119 ms) : 0, 55119
AppSec [candidate] (53.217 ms) : 0, 53217
Debugger [baseline] (6.019 ms) : 0, 6019
Debugger [candidate] (5.955 ms) : 0, 5955
Remote Config [baseline] (588.91 µs) : 0, 589
Remote Config [candidate] (589.856 µs) : 0, 590
Telemetry [baseline] (7.89 ms) : 0, 7890
Telemetry [candidate] (7.927 ms) : 0, 7927
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.151 ms) : 0, 800151
BytebuddyAgent [candidate] (803.419 ms) : 0, 803419
GlobalTracer [baseline] (231.198 ms) : 0, 231198
GlobalTracer [candidate] (232.164 ms) : 0, 232164
IAST [baseline] (29.324 ms) : 0, 29324
IAST [candidate] (29.483 ms) : 0, 29483
AppSec [baseline] (51.483 ms) : 0, 51483
AppSec [candidate] (51.446 ms) : 0, 51446
Debugger [baseline] (6.051 ms) : 0, 6051
Debugger [candidate] (6.038 ms) : 0, 6038
Remote Config [baseline] (593.296 µs) : 0, 593
Remote Config [candidate] (597.714 µs) : 0, 598
Telemetry [baseline] (7.814 ms) : 0, 7814
Telemetry [candidate] (7.812 ms) : 0, 7812
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.034 s) : 0, 1034087
Total [baseline] (10.597 s) : 0, 10597162
Agent [candidate] (1.027 s) : 0, 1026700
Total [candidate] (10.63 s) : 0, 10630389
section appsec
Agent [baseline] (1.191 s) : 0, 1191468
Total [baseline] (10.883 s) : 0, 10882764
Agent [candidate] (1.185 s) : 0, 1185109
Total [candidate] (10.758 s) : 0, 10758494
section iast
Agent [baseline] (1.157 s) : 0, 1157475
Total [baseline] (10.887 s) : 0, 10887092
Agent [candidate] (1.16 s) : 0, 1159666
Total [candidate] (10.937 s) : 0, 10937420
section profiling
Agent [baseline] (1.279 s) : 0, 1279394
Total [baseline] (10.954 s) : 0, 10954163
Agent [candidate] (1.271 s) : 0, 1270596
Total [candidate] (10.978 s) : 0, 10978201
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.034 s -
Agent appsec 1.191 s 157.381 ms (15.2%)
Agent iast 1.157 s 123.389 ms (11.9%)
Agent profiling 1.279 s 245.308 ms (23.7%)
Total tracing 10.597 s -
Total appsec 10.883 s 285.603 ms (2.7%)
Total iast 10.887 s 289.93 ms (2.7%)
Total profiling 10.954 s 357.001 ms (3.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.027 s -
Agent appsec 1.185 s 158.408 ms (15.4%)
Agent iast 1.16 s 132.966 ms (13.0%)
Agent profiling 1.271 s 243.896 ms (23.8%)
Total tracing 10.63 s -
Total appsec 10.758 s 128.106 ms (1.2%)
Total iast 10.937 s 307.032 ms (2.9%)
Total profiling 10.978 s 347.812 ms (3.3%) 
gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.527 ms) : 0, 689527
BytebuddyAgent [candidate] (683.594 ms) : 0, 683594
GlobalTracer [baseline] (242.973 ms) : 0, 242973
GlobalTracer [candidate] (240.847 ms) : 0, 240847
AppSec [baseline] (58.246 ms) : 0, 58246
AppSec [candidate] (59.139 ms) : 0, 59139
Debugger [baseline] (7.646 ms) : 0, 7646
Debugger [candidate] (7.645 ms) : 0, 7645
Remote Config [baseline] (755.364 µs) : 0, 755
Remote Config [candidate] (751.087 µs) : 0, 751
Telemetry [baseline] (11.343 ms) : 0, 11343
Telemetry [candidate] (11.285 ms) : 0, 11285
section appsec
BytebuddyAgent [baseline] (712.95 ms) : 0, 712950
BytebuddyAgent [candidate] (712.084 ms) : 0, 712084
GlobalTracer [baseline] (237.95 ms) : 0, 237950
GlobalTracer [candidate] (234.987 ms) : 0, 234987
AppSec [baseline] (180.731 ms) : 0, 180731
AppSec [candidate] (179.153 ms) : 0, 179153
Debugger [baseline] (5.949 ms) : 0, 5949
Debugger [candidate] (5.789 ms) : 0, 5789
Remote Config [baseline] (621.77 µs) : 0, 622
Remote Config [candidate] (604.938 µs) : 0, 605
Telemetry [baseline] (7.319 ms) : 0, 7319
Telemetry [candidate] (7.182 ms) : 0, 7182
IAST [baseline] (22.212 ms) : 0, 22212
IAST [candidate] (21.704 ms) : 0, 21704
section iast
BytebuddyAgent [baseline] (805.685 ms) : 0, 805685
BytebuddyAgent [candidate] (807.008 ms) : 0, 807008
GlobalTracer [baseline] (231.372 ms) : 0, 231372
GlobalTracer [candidate] (232.679 ms) : 0, 232679
AppSec [baseline] (55.24 ms) : 0, 55240
AppSec [candidate] (53.95 ms) : 0, 53950
Debugger [baseline] (6.061 ms) : 0, 6061
Debugger [candidate] (6.047 ms) : 0, 6047
Remote Config [baseline] (594.002 µs) : 0, 594
Remote Config [candidate] (591.309 µs) : 0, 591
Telemetry [baseline] (7.99 ms) : 0, 7990
Telemetry [candidate] (7.935 ms) : 0, 7935
IAST [baseline] (26.953 ms) : 0, 26953
IAST [candidate] (27.777 ms) : 0, 27777
section profiling
ProfilingAgent [baseline] (107.979 ms) : 0, 107979
ProfilingAgent [candidate] (105.86 ms) : 0, 105860
BytebuddyAgent [baseline] (679.472 ms) : 0, 679472
BytebuddyAgent [candidate] (673.923 ms) : 0, 673923
GlobalTracer [baseline] (361.072 ms) : 0, 361072
GlobalTracer [candidate] (360.339 ms) : 0, 360339
AppSec [baseline] (64.598 ms) : 0, 64598
AppSec [candidate] (64.723 ms) : 0, 64723
Debugger [baseline] (6.197 ms) : 0, 6197
Debugger [candidate] (6.152 ms) : 0, 6152
Remote Config [baseline] (658.963 µs) : 0, 659
Remote Config [candidate] (648.918 µs) : 0, 649
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (8.24 ms) : 0, 8240
Profiling [baseline] (108.003 ms) : 0, 108003
Profiling [candidate] (105.886 ms) : 0, 105886
Loading

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~9efa17209f, baseline=1.50.0-SNAPSHOT~9db71d4352
    dateFormat X
    axisFormat %s
section baseline
no_agent (386.833 µs) : 367, 407
.   : milestone, 387,
iast (535.844 µs) : 512, 559
.   : milestone, 536,
iast_FULL (755.737 µs) : 732, 780
.   : milestone, 756,
iast_GLOBAL (585.09 µs) : 562, 608
.   : milestone, 585,
iast_HARDCODED_SECRET_DISABLED (530.703 µs) : 507, 554
.   : milestone, 531,
iast_INACTIVE (473.184 µs) : 451, 496
.   : milestone, 473,
iast_TELEMETRY_OFF (521.501 µs) : 500, 543
.   : milestone, 522,
tracing (469.46 µs) : 447, 492
.   : milestone, 469,
section candidate
no_agent (390.173 µs) : 370, 411
.   : milestone, 390,
iast (542.631 µs) : 519, 566
.   : milestone, 543,
iast_FULL (754.693 µs) : 731, 778
.   : milestone, 755,
iast_GLOBAL (582.01 µs) : 559, 605
.   : milestone, 582,
iast_HARDCODED_SECRET_DISABLED (541.59 µs) : 518, 565
.   : milestone, 542,
iast_INACTIVE (480.898 µs) : 458, 504
.   : milestone, 481,
iast_TELEMETRY_OFF (525.941 µs) : 504, 548
.   : milestone, 526,
tracing (465.445 µs) : 443, 487
.   : milestone, 465,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 386.833 µs [366.554 µs, 407.111 µs] -
iast 535.844 µs [512.458 µs, 559.229 µs] 149.011 µs (38.5%)
iast_FULL 755.737 µs [731.803 µs, 779.672 µs] 368.904 µs (95.4%)
iast_GLOBAL 585.09 µs [561.724 µs, 608.457 µs] 198.258 µs (51.3%)
iast_HARDCODED_SECRET_DISABLED 530.703 µs [507.453 µs, 553.954 µs] 143.87 µs (37.2%)
iast_INACTIVE 473.184 µs [450.565 µs, 495.803 µs] 86.351 µs (22.3%)
iast_TELEMETRY_OFF 521.501 µs [499.857 µs, 543.144 µs] 134.668 µs (34.8%)
tracing 469.46 µs [447.231 µs, 491.688 µs] 82.627 µs (21.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 390.173 µs [369.669 µs, 410.678 µs] -
iast 542.631 µs [519.307 µs, 565.955 µs] 152.458 µs (39.1%)
iast_FULL 754.693 µs [731.002 µs, 778.385 µs] 364.52 µs (93.4%)
iast_GLOBAL 582.01 µs [558.842 µs, 605.177 µs] 191.836 µs (49.2%)
iast_HARDCODED_SECRET_DISABLED 541.59 µs [518.422 µs, 564.758 µs] 151.416 µs (38.8%)
iast_INACTIVE 480.898 µs [458.086 µs, 503.71 µs] 90.725 µs (23.3%)
iast_TELEMETRY_OFF 525.941 µs [504.065 µs, 547.817 µs] 135.768 µs (34.8%)
tracing 465.445 µs [443.479 µs, 487.411 µs] 75.271 µs (19.3%)

Dacapo

@bric3 bric3 marked this pull request as ready for review June 16, 2025 15:23
@bric3 bric3 requested a review from a team as a code owner June 16, 2025 15:23
@bric3 bric3 requested a review from daniel-mohedano June 16, 2025 15:23
@smola smola added the comp: ci visibility Continuous Integration Visibility label Jun 16, 2025
@PerfectSlayer PerfectSlayer changed the title fix: Maven smoke test are timed out in CI Fix Maven smoke test timeout in CI Jun 16, 2025
@bric3 bric3 merged commit 4fa48aa into master Jun 16, 2025
490 of 491 checks passed
@bric3 bric3 deleted the bdu/increasemaven-smoke-test-timeout branch June 16, 2025 16:04
@github-actions github-actions bot added this to the 1.50.0 milestone Jun 16, 2025
@bric3 bric3 mentioned this pull request Jun 16, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@sarahchen6 sarahchen6 sarahchen6 approved these changes

@daniel-mohedano daniel-mohedano daniel-mohedano approved these changes

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD AlexeyKuznetsov-DD approved these changes

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.50.0
Development

Successfully merging this pull request may close these issues.
5 participants
@bric3 @smola @sarahchen6 @daniel-mohedano @AlexeyKuznetsov-DD