Skip to content

Increase default timeout for requests made to CI Visibility backend through EVP proxy #6884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 4, 2024
Merged

Increase default timeout for requests made to CI Visibility backend through EVP proxy #6884

merged 1 commit into from
Apr 4, 2024

Conversation

@nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Apr 4, 2024 •
edited
Loading

What Does This Do

When setting the timeout for the tracer's default HTTP client, an increased value is used if CI Visibility is enabled (30 seconds vs 10 seconds).

Motivation

CI Visibility has a "get skippable tests" endpoint that might take more than 15 seconds to respond in some cases.

Jira ticket: CIVIS-9601

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Apr 4, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review April 4, 2024 09:36
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner April 4, 2024 09:36
@pr-commenter
Copy link

pr-commenter bot commented Apr 4, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/ci-vis-backend-timeout-increase
git_commit_date 1712214564 1712220406
git_commit_sha c1ae1bc 40add45
release_version 1.32.0-SNAPSHOT~c1ae1bcc31 1.32.0-SNAPSHOT~40add45397
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1712223734 1712223734
ci_job_id 477712848 477712848
ci_pipeline_id 31431718 31431718
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~40add45397, baseline=1.32.0-SNAPSHOT~c1ae1bcc31

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.075 s) : 0, 1074953
Total [baseline] (10.363 s) : 0, 10363188
Agent [candidate] (1.078 s) : 0, 1077738
Total [candidate] (10.446 s) : 0, 10445834
section appsec
Agent [baseline] (1.199 s) : 0, 1198799
Total [baseline] (10.429 s) : 0, 10429026
Agent [candidate] (1.198 s) : 0, 1198002
Total [candidate] (10.49 s) : 0, 10490039
section iast
Agent [baseline] (1.208 s) : 0, 1208305
Total [baseline] (10.807 s) : 0, 10806525
Agent [candidate] (1.208 s) : 0, 1208037
Total [candidate] (10.787 s) : 0, 10786739
section profiling
Agent [baseline] (1.268 s) : 0, 1267696
Total [baseline] (10.613 s) : 0, 10612776
Agent [candidate] (1.27 s) : 0, 1270116
Total [candidate] (10.623 s) : 0, 10623281
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent appsec 1.199 s 123.847 ms (11.5%)
Agent iast 1.208 s 133.352 ms (12.4%)
Agent profiling 1.268 s 192.743 ms (17.9%)
Total tracing 10.363 s -
Total appsec 10.429 s 65.838 ms (0.6%)
Total iast 10.807 s 443.337 ms (4.3%)
Total profiling 10.613 s 249.588 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.078 s -
Agent appsec 1.198 s 120.263 ms (11.2%)
Agent iast 1.208 s 130.299 ms (12.1%)
Agent profiling 1.27 s 192.378 ms (17.9%)
Total tracing 10.446 s -
Total appsec 10.49 s 44.205 ms (0.4%)
Total iast 10.787 s 340.905 ms (3.3%)
Total profiling 10.623 s 177.447 ms (1.7%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~40add45397, baseline=1.32.0-SNAPSHOT~c1ae1bcc31

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.266 ms) : 0, 673266
BytebuddyAgent [candidate] (674.874 ms) : 0, 674874
GlobalTracer [baseline] (309.791 ms) : 0, 309791
GlobalTracer [candidate] (310.581 ms) : 0, 310581
AppSec [baseline] (49.336 ms) : 0, 49336
AppSec [candidate] (49.59 ms) : 0, 49590
Remote Config [baseline] (659.629 µs) : 0, 660
Remote Config [candidate] (658.878 µs) : 0, 659
Telemetry [baseline] (7.559 ms) : 0, 7559
Telemetry [candidate] (7.607 ms) : 0, 7607
section appsec
BytebuddyAgent [baseline] (696.229 ms) : 0, 696229
BytebuddyAgent [candidate] (694.496 ms) : 0, 694496
GlobalTracer [baseline] (291.324 ms) : 0, 291324
GlobalTracer [candidate] (291.947 ms) : 0, 291947
AppSec [baseline] (149.962 ms) : 0, 149962
AppSec [candidate] (150.192 ms) : 0, 150192
IAST [baseline] (18.86 ms) : 0, 18860
IAST [candidate] (18.893 ms) : 0, 18893
Remote Config [baseline] (603.178 µs) : 0, 603
Remote Config [candidate] (605.028 µs) : 0, 605
Telemetry [baseline] (7.409 ms) : 0, 7409
Telemetry [candidate] (7.438 ms) : 0, 7438
section iast
BytebuddyAgent [baseline] (799.332 ms) : 0, 799332
BytebuddyAgent [candidate] (800.978 ms) : 0, 800978
GlobalTracer [baseline] (291.393 ms) : 0, 291393
GlobalTracer [candidate] (290.52 ms) : 0, 290520
AppSec [baseline] (50.765 ms) : 0, 50765
AppSec [candidate] (49.099 ms) : 0, 49099
IAST [baseline] (24.813 ms) : 0, 24813
IAST [candidate] (23.339 ms) : 0, 23339
Remote Config [baseline] (609.743 µs) : 0, 610
Remote Config [candidate] (574.273 µs) : 0, 574
Telemetry [baseline] (6.631 ms) : 0, 6631
Telemetry [candidate] (8.904 ms) : 0, 8904
section profiling
BytebuddyAgent [baseline] (677.218 ms) : 0, 677218
BytebuddyAgent [candidate] (677.431 ms) : 0, 677431
GlobalTracer [baseline] (380.245 ms) : 0, 380245
GlobalTracer [candidate] (382.129 ms) : 0, 382129
AppSec [baseline] (50.109 ms) : 0, 50109
AppSec [candidate] (50.36 ms) : 0, 50360
Remote Config [baseline] (706.524 µs) : 0, 707
Remote Config [candidate] (698.443 µs) : 0, 698
Telemetry [baseline] (7.454 ms) : 0, 7454
Telemetry [candidate] (7.484 ms) : 0, 7484
ProfilingAgent [baseline] (95.623 ms) : 0, 95623
ProfilingAgent [candidate] (95.667 ms) : 0, 95667
Profiling [baseline] (95.647 ms) : 0, 95647
Profiling [candidate] (95.691 ms) : 0, 95691
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~40add45397, baseline=1.32.0-SNAPSHOT~c1ae1bcc31

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.074 s) : 0, 1073827
Total [baseline] (8.559 s) : 0, 8558838
Agent [candidate] (1.093 s) : 0, 1092576
Total [candidate] (8.63 s) : 0, 8630149
section iast
Agent [baseline] (1.199 s) : 0, 1198533
Total [baseline] (9.034 s) : 0, 9034260
Agent [candidate] (1.2 s) : 0, 1199639
Total [candidate] (9.019 s) : 0, 9019174
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.199 s) : 0, 1198985
Total [baseline] (8.959 s) : 0, 8959050
Agent [candidate] (1.197 s) : 0, 1196980
Total [candidate] (8.98 s) : 0, 8979727
section iast_TELEMETRY_OFF
Agent [baseline] (1.192 s) : 0, 1192328
Total [baseline] (9.0 s) : 0, 9000143
Agent [candidate] (1.205 s) : 0, 1205163
Total [candidate] (9.03 s) : 0, 9029709
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent iast 1.199 s 124.706 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.199 s 125.159 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.192 s 118.501 ms (11.0%)
Total tracing 8.559 s -
Total iast 9.034 s 475.422 ms (5.6%)
Total iast_HARDCODED_SECRET_DISABLED 8.959 s 400.212 ms (4.7%)
Total iast_TELEMETRY_OFF 9.0 s 441.305 ms (5.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.093 s -
Agent iast 1.2 s 107.063 ms (9.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.197 s 104.404 ms (9.6%)
Agent iast_TELEMETRY_OFF 1.205 s 112.587 ms (10.3%)
Total tracing 8.63 s -
Total iast 9.019 s 389.025 ms (4.5%)
Total iast_HARDCODED_SECRET_DISABLED 8.98 s 349.578 ms (4.1%)
Total iast_TELEMETRY_OFF 9.03 s 399.56 ms (4.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~40add45397, baseline=1.32.0-SNAPSHOT~c1ae1bcc31

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.004 ms) : 0, 673004
BytebuddyAgent [candidate] (683.848 ms) : 0, 683848
GlobalTracer [baseline] (308.962 ms) : 0, 308962
GlobalTracer [candidate] (314.991 ms) : 0, 314991
AppSec [baseline] (49.371 ms) : 0, 49371
AppSec [candidate] (50.454 ms) : 0, 50454
Remote Config [baseline] (654.496 µs) : 0, 654
Remote Config [candidate] (676.816 µs) : 0, 677
Telemetry [baseline] (7.571 ms) : 0, 7571
Telemetry [candidate] (7.72 ms) : 0, 7720
section iast
BytebuddyAgent [baseline] (794.521 ms) : 0, 794521
BytebuddyAgent [candidate] (794.338 ms) : 0, 794338
GlobalTracer [baseline] (287.715 ms) : 0, 287715
GlobalTracer [candidate] (289.021 ms) : 0, 289021
AppSec [baseline] (50.191 ms) : 0, 50191
AppSec [candidate] (48.825 ms) : 0, 48825
IAST [baseline] (24.419 ms) : 0, 24419
IAST [candidate] (25.126 ms) : 0, 25126
Remote Config [baseline] (572.874 µs) : 0, 573
Remote Config [candidate] (565.82 µs) : 0, 566
Telemetry [baseline] (6.638 ms) : 0, 6638
Telemetry [candidate] (7.333 ms) : 0, 7333
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (794.669 ms) : 0, 794669
BytebuddyAgent [candidate] (793.118 ms) : 0, 793118
GlobalTracer [baseline] (288.405 ms) : 0, 288405
GlobalTracer [candidate] (288.089 ms) : 0, 288089
AppSec [baseline] (50.71 ms) : 0, 50710
AppSec [candidate] (50.493 ms) : 0, 50493
IAST [baseline] (22.815 ms) : 0, 22815
IAST [candidate] (22.845 ms) : 0, 22845
Remote Config [baseline] (591.123 µs) : 0, 591
Remote Config [candidate] (585.52 µs) : 0, 586
Telemetry [baseline] (7.383 ms) : 0, 7383
Telemetry [candidate] (7.459 ms) : 0, 7459
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (790.856 ms) : 0, 790856
BytebuddyAgent [candidate] (798.374 ms) : 0, 798374
GlobalTracer [baseline] (287.573 ms) : 0, 287573
GlobalTracer [candidate] (290.729 ms) : 0, 290729
AppSec [baseline] (49.225 ms) : 0, 49225
AppSec [candidate] (50.355 ms) : 0, 50355
IAST [baseline] (23.116 ms) : 0, 23116
IAST [candidate] (23.137 ms) : 0, 23137
Remote Config [baseline] (599.553 µs) : 0, 600
Remote Config [candidate] (581.897 µs) : 0, 582
Telemetry [baseline] (6.585 ms) : 0, 6585
Telemetry [candidate] (7.32 ms) : 0, 7320
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-04-04T09:14:34 2024-04-04T09:36:31
git_branch master nikita-tkachenko/ci-vis-backend-timeout-increase
git_commit_date 1712214564 1712220406
git_commit_sha c1ae1bc 40add45
release_version 1.32.0-SNAPSHOT~c1ae1bcc31 1.32.0-SNAPSHOT~40add45397
start_time 2024-04-04T09:14:21 2024-04-04T09:36:18
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1712223734 1712223734
ci_job_id 477712848 477712848
ci_pipeline_id 31431718 31431718
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~40add45397, baseline=1.32.0-SNAPSHOT~c1ae1bcc31
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.722 ms) : 1698, 1746
.   : milestone, 1722,
appsec_no_iast (1.721 ms) : 1698, 1745
.   : milestone, 1721,
iast (1.502 ms) : 1480, 1524
.   : milestone, 1502,
profiling (1.487 ms) : 1462, 1513
.   : milestone, 1487,
tracing (1.489 ms) : 1465, 1513
.   : milestone, 1489,
section candidate
no_agent (1.349 ms) : 1329, 1368
.   : milestone, 1349,
appsec (1.706 ms) : 1682, 1731
.   : milestone, 1706,
appsec_no_iast (1.721 ms) : 1696, 1745
.   : milestone, 1721,
iast (1.499 ms) : 1477, 1522
.   : milestone, 1499,
profiling (1.488 ms) : 1464, 1511
.   : milestone, 1488,
tracing (1.478 ms) : 1454, 1502
.   : milestone, 1478,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.322 ms, 1.36 ms] -
appsec 1.722 ms [1.698 ms, 1.746 ms] 380.821 µs (28.4%)
appsec_no_iast 1.721 ms [1.698 ms, 1.745 ms] 380.265 µs (28.4%)
iast 1.502 ms [1.48 ms, 1.524 ms] 161.053 µs (12.0%)
profiling 1.487 ms [1.462 ms, 1.513 ms] 146.25 µs (10.9%)
tracing 1.489 ms [1.465 ms, 1.513 ms] 147.968 µs (11.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.329 ms, 1.368 ms] -
appsec 1.706 ms [1.682 ms, 1.731 ms] 357.843 µs (26.5%)
appsec_no_iast 1.721 ms [1.696 ms, 1.745 ms] 371.978 µs (27.6%)
iast 1.499 ms [1.477 ms, 1.522 ms] 150.826 µs (11.2%)
profiling 1.488 ms [1.464 ms, 1.511 ms] 139.017 µs (10.3%)
tracing 1.478 ms [1.454 ms, 1.502 ms] 129.516 µs (9.6%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~40add45397, baseline=1.32.0-SNAPSHOT~c1ae1bcc31
    dateFormat X
    axisFormat %s
section baseline
no_agent (366.135 µs) : 346, 386
.   : milestone, 366,
iast (464.834 µs) : 444, 486
.   : milestone, 465,
iast_FULL (533.029 µs) : 511, 555
.   : milestone, 533,
iast_GLOBAL (487.954 µs) : 467, 509
.   : milestone, 488,
iast_HARDCODED_SECRET_DISABLED (472.452 µs) : 451, 494
.   : milestone, 472,
iast_INACTIVE (435.707 µs) : 415, 456
.   : milestone, 436,
iast_TELEMETRY_OFF (465.609 µs) : 444, 487
.   : milestone, 466,
tracing (439.904 µs) : 419, 461
.   : milestone, 440,
section candidate
no_agent (357.347 µs) : 338, 377
.   : milestone, 357,
iast (465.699 µs) : 445, 486
.   : milestone, 466,
iast_FULL (536.832 µs) : 516, 557
.   : milestone, 537,
iast_GLOBAL (504.226 µs) : 481, 527
.   : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (471.598 µs) : 450, 493
.   : milestone, 472,
iast_INACTIVE (446.74 µs) : 426, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (468.338 µs) : 447, 490
.   : milestone, 468,
tracing (436.374 µs) : 416, 457
.   : milestone, 436,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 366.135 µs [346.385 µs, 385.886 µs] -
iast 464.834 µs [443.94 µs, 485.728 µs] 98.698 µs (27.0%)
iast_FULL 533.029 µs [511.244 µs, 554.814 µs] 166.894 µs (45.6%)
iast_GLOBAL 487.954 µs [466.873 µs, 509.034 µs] 121.818 µs (33.3%)
iast_HARDCODED_SECRET_DISABLED 472.452 µs [450.621 µs, 494.283 µs] 106.317 µs (29.0%)
iast_INACTIVE 435.707 µs [415.313 µs, 456.102 µs] 69.572 µs (19.0%)
iast_TELEMETRY_OFF 465.609 µs [444.043 µs, 487.175 µs] 99.474 µs (27.2%)
tracing 439.904 µs [418.947 µs, 460.86 µs] 73.768 µs (20.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 357.347 µs [337.745 µs, 376.949 µs] -
iast 465.699 µs [445.029 µs, 486.369 µs] 108.352 µs (30.3%)
iast_FULL 536.832 µs [516.229 µs, 557.436 µs] 179.485 µs (50.2%)
iast_GLOBAL 504.226 µs [481.422 µs, 527.031 µs] 146.879 µs (41.1%)
iast_HARDCODED_SECRET_DISABLED 471.598 µs [450.405 µs, 492.791 µs] 114.251 µs (32.0%)
iast_INACTIVE 446.74 µs [425.933 µs, 467.548 µs] 89.393 µs (25.0%)
iast_TELEMETRY_OFF 468.338 µs [447.129 µs, 489.547 µs] 110.991 µs (31.1%)
tracing 436.374 µs [415.926 µs, 456.823 µs] 79.027 µs (22.1%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 386364a into master Apr 4, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/ci-vis-backend-timeout-increase branch April 4, 2024 11:43
@github-actions github-actions bot added this to the 1.33.0 milestone Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

1 more reviewer

@romainkomorn-exdatadog romainkomorn-exdatadog romainkomorn-exdatadog approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp: ci visibility Continuous Integration Visibility

Projects

None yet

Milestone

1.33.0

Development

Successfully merging this pull request may close these issues.

3 participants

@nikita-tkachenko-datadog @romainkomorn-exdatadog