Update impacted tests logic #8923

daniel-mohedano · 2025-06-04T12:29:13Z

What Does This Do

Updates the logic behind the impacted tests feature:

Removes the backend request for changed files
Adds default_branch to the expected response from the Settings endpoint
Implements a best effort algorithm for calculating the base commit SHA (commit where the feature/source branch originally diverted from the base branch)

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

…dors" This reverts commit a7f738d.

This reverts commit e401549.

This reverts commit 63e7236.

pr-commenter · 2025-06-04T13:00:07Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/impacted-tests-update
git_commit_date	1750453785	1750513056
git_commit_sha	`aa9dd27`	`67cdbe7`
release_version	1.51.0-SNAPSHOT~aa9dd27535	1.51.0-SNAPSHOT~67cdbe7862

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750514876	1750514876
ci_job_id	992433655	992433655
ci_pipeline_id	68388472	68388472
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-bziamzy-project-304-concurrent-0-robv6ttf 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-bziamzy-project-304-concurrent-0-robv6ttf 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 43 metrics, 10 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.026 s) : 0, 1026336
Total [baseline] (10.611 s) : 0, 10610928
Agent [candidate] (1.042 s) : 0, 1041635
Total [candidate] (10.678 s) : 0, 10677603
section appsec
Agent [baseline] (1.181 s) : 0, 1180754
Total [baseline] (10.767 s) : 0, 10766655
Agent [candidate] (1.182 s) : 0, 1182166
Total [candidate] (10.754 s) : 0, 10753795
section iast
Agent [baseline] (1.159 s) : 0, 1158896
Total [baseline] (10.941 s) : 0, 10940590
Agent [candidate] (1.155 s) : 0, 1154857
Total [candidate] (10.858 s) : 0, 10858189
section profiling
Agent [baseline] (1.272 s) : 0, 1271575
Total [baseline] (11.028 s) : 0, 11027822
Agent [candidate] (1.269 s) : 0, 1269021
Total [candidate] (10.971 s) : 0, 10971331

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.026 s	-
Agent	appsec	1.181 s	154.418 ms (15.0%)
Agent	iast	1.159 s	132.56 ms (12.9%)
Agent	profiling	1.272 s	245.239 ms (23.9%)
Total	tracing	10.611 s	-
Total	appsec	10.767 s	155.727 ms (1.5%)
Total	iast	10.941 s	329.661 ms (3.1%)
Total	profiling	11.028 s	416.894 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	appsec	1.182 s	140.53 ms (13.5%)
Agent	iast	1.155 s	113.222 ms (10.9%)
Agent	profiling	1.269 s	227.386 ms (21.8%)
Total	tracing	10.678 s	-
Total	appsec	10.754 s	76.192 ms (0.7%)
Total	iast	10.858 s	180.586 ms (1.7%)
Total	profiling	10.971 s	293.729 ms (2.8%)

gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.278 ms) : 0, 687278
BytebuddyAgent [candidate] (698.327 ms) : 0, 698327
GlobalTracer [baseline] (242.597 ms) : 0, 242597
GlobalTracer [candidate] (244.676 ms) : 0, 244676
AppSec [baseline] (58.113 ms) : 0, 58113
AppSec [candidate] (58.41 ms) : 0, 58410
Debugger [baseline] (6.216 ms) : 0, 6216
Debugger [candidate] (6.373 ms) : 0, 6373
Remote Config [baseline] (667.19 µs) : 0, 667
Remote Config [candidate] (690.938 µs) : 0, 691
Telemetry [baseline] (10.543 ms) : 0, 10543
Telemetry [candidate] (11.905 ms) : 0, 11905
section appsec
BytebuddyAgent [baseline] (710.129 ms) : 0, 710129
BytebuddyAgent [candidate] (709.654 ms) : 0, 709654
GlobalTracer [baseline] (235.036 ms) : 0, 235036
GlobalTracer [candidate] (235.601 ms) : 0, 235601
AppSec [baseline] (179.326 ms) : 0, 179326
AppSec [candidate] (180.451 ms) : 0, 180451
Debugger [baseline] (5.764 ms) : 0, 5764
Debugger [candidate] (5.826 ms) : 0, 5826
Remote Config [baseline] (621.421 µs) : 0, 621
Remote Config [candidate] (612.281 µs) : 0, 612
Telemetry [baseline] (7.224 ms) : 0, 7224
Telemetry [candidate] (7.269 ms) : 0, 7269
IAST [baseline] (21.827 ms) : 0, 21827
IAST [candidate] (21.902 ms) : 0, 21902
section iast
BytebuddyAgent [baseline] (810.975 ms) : 0, 810975
BytebuddyAgent [candidate] (807.89 ms) : 0, 807890
GlobalTracer [baseline] (233.328 ms) : 0, 233328
GlobalTracer [candidate] (232.424 ms) : 0, 232424
AppSec [baseline] (52.148 ms) : 0, 52148
AppSec [candidate] (50.664 ms) : 0, 50664
Debugger [baseline] (5.924 ms) : 0, 5924
Debugger [candidate] (5.991 ms) : 0, 5991
Remote Config [baseline] (604.634 µs) : 0, 605
Remote Config [candidate] (610.468 µs) : 0, 610
Telemetry [baseline] (8.042 ms) : 0, 8042
Telemetry [candidate] (7.999 ms) : 0, 7999
IAST [baseline] (26.934 ms) : 0, 26934
IAST [candidate] (28.379 ms) : 0, 28379
section profiling
ProfilingAgent [baseline] (106.994 ms) : 0, 106994
ProfilingAgent [candidate] (106.029 ms) : 0, 106029
BytebuddyAgent [baseline] (677.427 ms) : 0, 677427
BytebuddyAgent [candidate] (676.376 ms) : 0, 676376
GlobalTracer [baseline] (361.174 ms) : 0, 361174
GlobalTracer [candidate] (361.137 ms) : 0, 361137
AppSec [baseline] (62.27 ms) : 0, 62270
AppSec [candidate] (62.04 ms) : 0, 62040
Debugger [baseline] (6.158 ms) : 0, 6158
Debugger [candidate] (6.088 ms) : 0, 6088
Remote Config [baseline] (694.196 µs) : 0, 694
Remote Config [candidate] (692.399 µs) : 0, 692
Telemetry [baseline] (8.165 ms) : 0, 8165
Telemetry [candidate] (8.04 ms) : 0, 8040
Profiling [baseline] (107.02 ms) : 0, 107020
Profiling [candidate] (106.054 ms) : 0, 106054

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1029716
Total [baseline] (8.573 s) : 0, 8573392
Agent [candidate] (1.046 s) : 0, 1045534
Total [candidate] (8.617 s) : 0, 8616609
section iast
Agent [baseline] (1.159 s) : 0, 1159231
Total [baseline] (9.254 s) : 0, 9254440
Agent [candidate] (1.153 s) : 0, 1152740
Total [candidate] (9.258 s) : 0, 9257731

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	iast	1.159 s	129.516 ms (12.6%)
Total	tracing	8.573 s	-
Total	iast	9.254 s	681.047 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	iast	1.153 s	107.205 ms (10.3%)
Total	tracing	8.617 s	-
Total	iast	9.258 s	641.122 ms (7.4%)

gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.112 ms) : 0, 688112
BytebuddyAgent [candidate] (701.348 ms) : 0, 701348
GlobalTracer [baseline] (242.839 ms) : 0, 242839
GlobalTracer [candidate] (246.59 ms) : 0, 246590
AppSec [baseline] (60.512 ms) : 0, 60512
AppSec [candidate] (59.471 ms) : 0, 59471
Debugger [baseline] (6.215 ms) : 0, 6215
Debugger [candidate] (6.331 ms) : 0, 6331
Remote Config [baseline] (664.255 µs) : 0, 664
Remote Config [candidate] (695.167 µs) : 0, 695
Telemetry [baseline] (10.455 ms) : 0, 10455
Telemetry [candidate] (9.943 ms) : 0, 9943
section iast
BytebuddyAgent [baseline] (811.869 ms) : 0, 811869
BytebuddyAgent [candidate] (806.407 ms) : 0, 806407
GlobalTracer [baseline] (232.424 ms) : 0, 232424
GlobalTracer [candidate] (231.805 ms) : 0, 231805
IAST [baseline] (27.689 ms) : 0, 27689
IAST [candidate] (28.477 ms) : 0, 28477
AppSec [baseline] (51.696 ms) : 0, 51696
AppSec [candidate] (50.688 ms) : 0, 50688
Debugger [baseline] (5.973 ms) : 0, 5973
Debugger [candidate] (5.917 ms) : 0, 5917
Remote Config [baseline] (621.75 µs) : 0, 622
Remote Config [candidate] (603.822 µs) : 0, 604
Telemetry [baseline] (8.041 ms) : 0, 8041
Telemetry [candidate] (8.023 ms) : 0, 8023

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/impacted-tests-update
git_commit_date	1750453785	1750513056
git_commit_sha	`aa9dd27`	`67cdbe7`
release_version	1.51.0-SNAPSHOT~aa9dd27535	1.51.0-SNAPSHOT~67cdbe7862

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750514553	1750514553
ci_job_id	992433656	992433656
ci_pipeline_id	68388472	68388472
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-clrnfvyp-project-304-concurrent-0-zfjzyjf0 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-clrnfvyp-project-304-concurrent-0-zfjzyjf0 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:tracing:high_load	worse [+268.810µs; +542.493µs] or [+3.425%; +6.913%]	unstable [-94.626op/s; +36.876op/s] or [-16.024%; +6.245%]	8.253ms	561.656op/s	7.848ms	590.531op/s
scenario:load:petclinic:iast:high_load	better [-2.343ms; -1.523ms] or [-5.208%; -3.385%]	unstable [-2.927op/s; +12.202op/s] or [-2.814%; +11.730%]	43.050ms	108.662op/s	44.983ms	104.025op/s
scenario:load:petclinic:profiling:high_load	worse [+0.985ms; +1.865ms] or [+2.185%; +4.138%]	unstable [-10.622op/s; +4.222op/s] or [-10.227%; +4.065%]	46.495ms	100.662op/s	45.070ms	103.862op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.06 ms) : 36757, 37363
.   : milestone, 37060,
appsec (48.288 ms) : 47859, 48718
.   : milestone, 48288,
code_origins (45.141 ms) : 44766, 45515
.   : milestone, 45141,
iast (44.983 ms) : 44595, 45371
.   : milestone, 44983,
profiling (45.07 ms) : 44655, 45485
.   : milestone, 45070,
tracing (43.168 ms) : 42803, 43532
.   : milestone, 43168,
section candidate
no_agent (37.873 ms) : 37568, 38178
.   : milestone, 37873,
appsec (48.132 ms) : 47704, 48560
.   : milestone, 48132,
code_origins (45.587 ms) : 45214, 45961
.   : milestone, 45587,
iast (43.05 ms) : 42676, 43424
.   : milestone, 43050,
profiling (46.495 ms) : 46091, 46898
.   : milestone, 46495,
tracing (43.115 ms) : 42760, 43470
.   : milestone, 43115,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.06 ms [36.757 ms, 37.363 ms]	-
appsec	48.288 ms [47.859 ms, 48.718 ms]	11.228 ms (30.3%)
code_origins	45.141 ms [44.766 ms, 45.515 ms]	8.081 ms (21.8%)
iast	44.983 ms [44.595 ms, 45.371 ms]	7.923 ms (21.4%)
profiling	45.07 ms [44.655 ms, 45.485 ms]	8.01 ms (21.6%)
tracing	43.168 ms [42.803 ms, 43.532 ms]	6.108 ms (16.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.873 ms [37.568 ms, 38.178 ms]	-
appsec	48.132 ms [47.704 ms, 48.56 ms]	10.259 ms (27.1%)
code_origins	45.587 ms [45.214 ms, 45.961 ms]	7.715 ms (20.4%)
iast	43.05 ms [42.676 ms, 43.424 ms]	5.177 ms (13.7%)
profiling	46.495 ms [46.091 ms, 46.898 ms]	8.622 ms (22.8%)
tracing	43.115 ms [42.76 ms, 43.47 ms]	5.242 ms (13.8%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.473 ms) : 4422, 4524
.   : milestone, 4473,
iast (9.461 ms) : 9297, 9625
.   : milestone, 9461,
iast_FULL (13.932 ms) : 13654, 14209
.   : milestone, 13932,
iast_GLOBAL (9.708 ms) : 9537, 9879
.   : milestone, 9708,
profiling (8.719 ms) : 8576, 8862
.   : milestone, 8719,
tracing (7.848 ms) : 7720, 7975
.   : milestone, 7848,
section candidate
no_agent (4.564 ms) : 4511, 4617
.   : milestone, 4564,
iast (9.509 ms) : 9348, 9670
.   : milestone, 9509,
iast_FULL (13.599 ms) : 13327, 13870
.   : milestone, 13599,
iast_GLOBAL (9.998 ms) : 9822, 10173
.   : milestone, 9998,
profiling (8.488 ms) : 8350, 8626
.   : milestone, 8488,
tracing (8.253 ms) : 8127, 8380
.   : milestone, 8253,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.473 ms [4.422 ms, 4.524 ms]	-
iast	9.461 ms [9.297 ms, 9.625 ms]	4.988 ms (111.5%)
iast_FULL	13.932 ms [13.654 ms, 14.209 ms]	9.459 ms (211.5%)
iast_GLOBAL	9.708 ms [9.537 ms, 9.879 ms]	5.235 ms (117.0%)
profiling	8.719 ms [8.576 ms, 8.862 ms]	4.246 ms (94.9%)
tracing	7.848 ms [7.72 ms, 7.975 ms]	3.375 ms (75.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.564 ms [4.511 ms, 4.617 ms]	-
iast	9.509 ms [9.348 ms, 9.67 ms]	4.945 ms (108.3%)
iast_FULL	13.599 ms [13.327 ms, 13.87 ms]	9.034 ms (197.9%)
iast_GLOBAL	9.998 ms [9.822 ms, 10.173 ms]	5.433 ms (119.0%)
profiling	8.488 ms [8.35 ms, 8.626 ms]	3.924 ms (86.0%)
tracing	8.253 ms [8.127 ms, 8.38 ms]	3.689 ms (80.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	daniel.mohedano/impacted-tests-update
git_commit_date	1750453785	1750513056
git_commit_sha	`aa9dd27`	`67cdbe7`
release_version	1.51.0-SNAPSHOT~aa9dd27535	1.51.0-SNAPSHOT~67cdbe7862

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1750515058	1750515058
ci_job_id	992433657	992433657
ci_pipeline_id	68388472	68388472
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-pw1nxbd--project-304-concurrent-0-xa4lv9jl 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-pw1nxbd--project-304-concurrent-0-xa4lv9jl 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.4 ms) : 2351, 2448
.   : milestone, 2400,
iast (2.192 ms) : 2131, 2254
.   : milestone, 2192,
iast_GLOBAL (2.234 ms) : 2172, 2295
.   : milestone, 2234,
profiling (2.488 ms) : 2312, 2664
.   : milestone, 2488,
tracing (2.013 ms) : 1965, 2061
.   : milestone, 2013,
section candidate
no_agent (1.477 ms) : 1466, 1489
.   : milestone, 1477,
appsec (2.41 ms) : 2361, 2459
.   : milestone, 2410,
iast (2.191 ms) : 2130, 2253
.   : milestone, 2191,
iast_GLOBAL (2.242 ms) : 2180, 2305
.   : milestone, 2242,
profiling (2.025 ms) : 1976, 2074
.   : milestone, 2025,
tracing (2.013 ms) : 1966, 2061
.   : milestone, 2013,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.468 ms, 1.491 ms]	-
appsec	2.4 ms [2.351 ms, 2.448 ms]	920.215 µs (62.2%)
iast	2.192 ms [2.131 ms, 2.254 ms]	713.051 µs (48.2%)
iast_GLOBAL	2.234 ms [2.172 ms, 2.295 ms]	754.322 µs (51.0%)
profiling	2.488 ms [2.312 ms, 2.664 ms]	1.008 ms (68.2%)
tracing	2.013 ms [1.965 ms, 2.061 ms]	533.635 µs (36.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.466 ms, 1.489 ms]	-
appsec	2.41 ms [2.361 ms, 2.459 ms]	932.85 µs (63.2%)
iast	2.191 ms [2.13 ms, 2.253 ms]	714.09 µs (48.3%)
iast_GLOBAL	2.242 ms [2.18 ms, 2.305 ms]	765.34 µs (51.8%)
profiling	2.025 ms [1.976 ms, 2.074 ms]	547.438 µs (37.1%)
tracing	2.013 ms [1.966 ms, 2.061 ms]	536.289 µs (36.3%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~67cdbe7862, baseline=1.51.0-SNAPSHOT~aa9dd27535
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.973 s) : 14973000, 14973000
.   : milestone, 14973000,
appsec (14.643 s) : 14643000, 14643000
.   : milestone, 14643000,
iast (18.661 s) : 18661000, 18661000
.   : milestone, 18661000,
iast_GLOBAL (17.892 s) : 17892000, 17892000
.   : milestone, 17892000,
profiling (15.815 s) : 15815000, 15815000
.   : milestone, 15815000,
tracing (14.998 s) : 14998000, 14998000
.   : milestone, 14998000,
section candidate
no_agent (15.475 s) : 15475000, 15475000
.   : milestone, 15475000,
appsec (14.808 s) : 14808000, 14808000
.   : milestone, 14808000,
iast (18.701 s) : 18701000, 18701000
.   : milestone, 18701000,
iast_GLOBAL (17.909 s) : 17909000, 17909000
.   : milestone, 17909000,
profiling (15.081 s) : 15081000, 15081000
.   : milestone, 15081000,
tracing (14.936 s) : 14936000, 14936000
.   : milestone, 14936000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.973 s [14.973 s, 14.973 s]	-
appsec	14.643 s [14.643 s, 14.643 s]	-330.0 ms (-2.2%)
iast	18.661 s [18.661 s, 18.661 s]	3.688 s (24.6%)
iast_GLOBAL	17.892 s [17.892 s, 17.892 s]	2.919 s (19.5%)
profiling	15.815 s [15.815 s, 15.815 s]	842.0 ms (5.6%)
tracing	14.998 s [14.998 s, 14.998 s]	25.0 ms (0.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.475 s [15.475 s, 15.475 s]	-
appsec	14.808 s [14.808 s, 14.808 s]	-667.0 ms (-4.3%)
iast	18.701 s [18.701 s, 18.701 s]	3.226 s (20.8%)
iast_GLOBAL	17.909 s [17.909 s, 17.909 s]	2.434 s (15.7%)
profiling	15.081 s [15.081 s, 15.081 s]	-394.0 ms (-2.5%)
tracing	14.936 s [14.936 s, 14.936 s]	-539.0 ms (-3.5%)

...visibility/src/main/java/datadog/trace/civisibility/config/ExecutionSettingsFactoryImpl.java

...rnal-api/src/main/java/datadog/trace/api/civisibility/telemetry/CiVisibilityCountMetric.java

...nt/agent-ci-visibility/src/main/java/datadog/trace/civisibility/git/tree/ShellGitClient.java

daniel-mohedano · 2025-06-06T09:29:29Z

dd-smoke-tests/backend-mock/src/main/groovy/datadog/smoketest/MockBackend.groovy

+    itrEnabled = true
+    codeCoverageEnabled = true
+    testsSkippingEnabled = true
+    flakyRetriesEnabled = false
+    impactedTestsDetectionEnabled = false
+    knownTestsEnabled = false
+    testManagementEnabled = false
+    attemptToFixRetries = 0


The mock backend now sets each setting back to its default value before every test run, which was causing some tests to be marked with enabled settings which shouldn't be. Wasn't failing on CI because the only case of this happening just added test_management.enabled to the traces

PerfectSlayer

About internal API: if you would like to add more feature, can you add the related Javadoc and unit tests for?

internal-api/src/main/java/datadog/trace/util/Strings.java

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.errorprone:error_prone_annotations](https://errorprone.info) ([source](https://github.com/google/error-prone)) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.39.0` -> `2.40.0` | | [org.apache.commons:commons-lang3](https://commons.apache.org/proper/commons-lang/) ([source](https://gitbox.apache.org/repos/asf/commons-lang.git)) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `3.17.0` -> `3.18.0` | | [org.jetbrains.kotlinx.binary-compatibility-validator](https://github.com/Kotlin/binary-compatibility-validator) | plugin | misk/gradle/libs.versions.toml | gradle | patch | `0.18.0` -> `0.18.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.50.1` -> `1.51.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.77` -> `2.31.78` | --- ### Release Notes <details> <summary>google/error-prone (com.google.errorprone:error_prone_annotations)</summary> ### [`v2.40.0`](https://github.com/google/error-prone/releases/tag/v2.40.0): Error Prone 2.40.0 Changes: - Bug fixes and improvements - Releases (including snapshots) have migrated from [OSSRH to the Central Publisher Portal](https://central.sonatype.org/pages/ossrh-eol/#process-to-migrate) Full changelog: google/error-prone@v2.39.0...v2.40.0 </details> <details> <summary>Kotlin/binary-compatibility-validator (org.jetbrains.kotlinx.binary-compatibility-validator)</summary> ### [`v0.18.1`](https://github.com/Kotlin/binary-compatibility-validator/releases/tag/0.18.1) [Compare Source](Kotlin/binary-compatibility-validator@0.18.0...0.18.1) #### What's Changed - Fixed a bug preventing use of cross-compilation support during KLIB dump validation \[[#304](https://github.com/Kotlin/binary-compatibility-validator/issues/304)]\[[#306](https://github.com/Kotlin/binary-compatibility-validator/issues/306)] </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.51.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.51.0): 1.51.0 ### Components #### Application Security Management (IAST) - 🐛 Fix verify error when ctor params are used after a call site ([#9083](DataDog/dd-trace-java#9083) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛 Limit the maximum size of the location path in IAST vulnerabilities ([#9028](DataDog/dd-trace-java#9028) - [@jandro996](https://github.com/jandro996)) - 🐛 Fix IAST gRPC handler with null superclass ([#8984](DataDog/dd-trace-java#8984) - [@smola](https://github.com/smola)) - ✨ Optimize IAST Vulnerability Detection ([#8885](DataDog/dd-trace-java#8885) - [@jandro996](https://github.com/jandro996)) #### Application Security Management (WAF) - ✨ Upgrade libddwaf-java to 15.0.0 ([#9022](DataDog/dd-trace-java#9022) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Extract RestEasy json body response schemas ([#9015](DataDog/dd-trace-java#9015) - [@jandro996](https://github.com/jandro996)) - ✨ Extract Jersey json body response schemas ([#9014](DataDog/dd-trace-java#9014) - [@jandro996](https://github.com/jandro996)) - ✨ Extract Ratpack json body response schemas ([#9013](DataDog/dd-trace-java#9013) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Enable API Security by default and make it lazy loading ([#9009](DataDog/dd-trace-java#9009) - [@smola](https://github.com/smola)) - ✨ Extract Vert.x json body response schemas ([#9001](DataDog/dd-trace-java#9001) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Extract Play json body response schemas ([#8995](DataDog/dd-trace-java#8995) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛 Fix Jackson nodes introspection for request/response schema extraction ([#8980](DataDog/dd-trace-java#8980) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Extract Spring json body response schemas ([#8938](DataDog/dd-trace-java#8938) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Default obfuscation regexp update ([#8937](DataDog/dd-trace-java#8937) - [@sezen-datadog](https://github.com/sezen-datadog)) #### Build & Tooling - ✨ Cancel GitLab running pipeline on new PR push ([#9023](DataDog/dd-trace-java#9023) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Migrate publishing to Maven Central Portal ([#8807](DataDog/dd-trace-java#8807) - [@sarahchen6](https://github.com/sarahchen6)) #### Continuous Integration Visibility - 🐛 Fix Test Optimization to work with JDK 24 ([#9114](DataDog/dd-trace-java#9114) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add repo root as safe directory on git client creation ([#9033](DataDog/dd-trace-java#9033) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Add PR number tag and improve PR information building ([#8990](DataDog/dd-trace-java#8990) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Update impacted tests logic ([#8923](DataDog/dd-trace-java#8923) - [@daniel-mohedano](https://github.com/daniel-mohedano)) #### Data Streams Monitoring - 🧹 Clean up DSM context injection ([#8776](DataDog/dd-trace-java#8776) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Database Monitoring - 🐛 Set trace\_injected in try block ([#9025](DataDog/dd-trace-java#9025) - [@natashadada](https://github.com/natashadada)) #### Dynamic Instrumentation - 🐛 Add source file tracking enable option ([#9115](DataDog/dd-trace-java#9115) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add java.util.Date support ([#9111](DataDog/dd-trace-java#9111) - [@jpbempel](https://github.com/jpbempel)) - ✨ Update file probe format ([#9047](DataDog/dd-trace-java#9047) - [@jpbempel](https://github.com/jpbempel)) - ✨ add safe local var hoisting ([#9034](DataDog/dd-trace-java#9034) - [@jpbempel](https://github.com/jpbempel)) - 🧹 Add new config for debugger upload interval ([#8959](DataDog/dd-trace-java#8959) - [@jpbempel](https://github.com/jpbempel)) - ✨ Enable Code Origin with Dynamic instrumentation ([#8940](DataDog/dd-trace-java#8940) - [@jpbempel](https://github.com/jpbempel)) #### ML Observability (LLMObs) - 💡 LLM Observability SDK ([#8781](DataDog/dd-trace-java#8781) - [@gary-huang](https://github.com/gary-huang), [@nayeem-kamal](https://github.com/nayeem-kamal)) #### Metrics - 🐛 Ensure client stat reporter is started when the agent is not available at bootstrap ([#9082](DataDog/dd-trace-java#9082) - [@amarziali](https://github.com/amarziali)) - ✨ Create metric: appsec.waf.config\_errors ([#8394](DataDog/dd-trace-java#8394) - [@sezen-datadog](https://github.com/sezen-datadog)) #### Platform components - ✨ Introduce environment component ([#9071](DataDog/dd-trace-java#9071) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Profiling - 🐛 Remove annoying warning for smap event parsing ([#9119](DataDog/dd-trace-java#9119) - [@jbachorik](https://github.com/jbachorik)) - 🐛 Fix ByteCountingInputStream when reading past EOF ([#8988](DataDog/dd-trace-java#8988) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Realtime User Monitoring - ✨ Add RUM SDK injection for servlet based web servers ([#9110](DataDog/dd-trace-java#9110) - [@PerfectSlayer](https://github.com/PerfectSlayer) [@amarziali](https://github.com/amarziali)) #### Telemetry - ✨ Update the config origin metric to match what it's mapping ([#9045](DataDog/dd-trace-java#9045) - [@sezen-datadog](https://github.com/sezen-datadog)) #### Testing - ✨ Add testing for latest stable version (JDK 24) ([#8875](DataDog/dd-trace-java#8875) - [@sarahchen6](https://github.com/sarahchen6)) #### Trace context propagation - 🐛 Fix bug with dropping baggage when `TracePropagationBehaviorExtract=IGNORE` ([#9037](DataDog/dd-trace-java#9037) - [@mhlidd](https://github.com/mhlidd)) - 🐛 Fix ArrayIndexOutOfBoundsException in PercentEscaper ([#9032](DataDog/dd-trace-java#9032) - [@mhlidd](https://github.com/mhlidd)) #### Tracer core - 🐛 Fix `Error` handling for trace interceptors ([#9097](DataDog/dd-trace-java#9097) - [@AlexeyKuznetsov-DD](https://github.com/AlexeyKuznetsov-DD)) - 💡 Add wildcard feature for `DD_TRACE_HEADER_TAGS` and enabling for Http Response headers ([#9067](DataDog/dd-trace-java#9067) - [@mhlidd](https://github.com/mhlidd)) #### Tracer public API - 💡 Add LLM Observability SDK ([#8781](DataDog/dd-trace-java#8781) - [@gary-huang](https://github.com/gary-huang)) ### Instrumentations #### Akka instrumentation - 🐛 Fix NPE in akka-http and pekko-http integrations ([#9019](DataDog/dd-trace-java#9019) - [@mcculls](https://github.com/mcculls)) #### Eclipse Vert.x instrumentation - ✨ Extract Vert.x json body response schemas ([#9001](DataDog/dd-trace-java#9001) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Write http.route tag as soon as possible in vert.x ([#8952](DataDog/dd-trace-java#8952) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### JAX-WS instrumentation - 💡⚠️ Enable jax-ws integration by default ([#9030](DataDog/dd-trace-java#9030) - [@bm1549](https://github.com/bm1549)) - ✨ Extract Jersey json body response schemas ([#9014](DataDog/dd-trace-java#9014) - [@jandro996](https://github.com/jandro996)) #### Mule instrumentation - 🐛 Propagate grizzly http span in filters if nothing is active ([#9016](DataDog/dd-trace-java#9016) - [@amarziali](https://github.com/amarziali)) #### Play Framework instrumentation - ✨ Extract Play json body response schemas ([#8995](DataDog/dd-trace-java#8995) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Ratpack instrumentation - ✨ Extract Ratpack json body response schemas ([#9013](DataDog/dd-trace-java#9013) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Spring instrumentation - ✨ Extract Spring json body response schemas ([#8938](DataDog/dd-trace-java#8938) - [@sezen-datadog](https://github.com/sezen-datadog)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 649b690d4c9d7dcb572c457f0802b42b8e3e682e

daniel-mohedano added 18 commits May 16, 2025 11:34

remove backend request for diff

6980d02

update PRInfo building to use partial information from CI vendors

a7f738d

synchronize repo unshallow with future, avoids performing several times

a1238c0

add default_branch to settings response

cbe2a95

update settings api test

b861e7f

change repo unshallow synchronization from future-based to synchronized

cab317e

include pr info checks to CITagsProviderTest

e401549

add tests for prinfo merging

63e7236

Revert "update PRInfo building to use partial information from CI ven…

a109cfd

…dors" This reverts commit a7f738d.

Revert "include pr info checks to CITagsProviderTest"

4d18887

This reverts commit e401549.

Revert "add tests for prinfo merging"

84e1c0f

This reverts commit 63e7236.

base commit sha algorithm implementation

ec152b6

example git repo for base commit sha testing

1e843af

simplify algo implementation for easier read

fc82fcc

implement git diff based on best candidate for base commit sha

67339e9

add github actions cloning test

bfd3ad4

add several testing cases and cleaned up debug messages

5ddb58c

remove repo_origin folder

c558a86

daniel-mohedano added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Jun 4, 2025

Merge branch 'master' into daniel.mohedano/impacted-tests-update

eca82e1

daniel-mohedano added 2 commits June 4, 2025 15:44

fix forbidden apis

0fac3ae

remove smoke test (no backend implementation anymore)

0e6bc2d

daniel-mohedano requested a review from nikita-tkachenko-datadog June 5, 2025 08:18

daniel-mohedano marked this pull request as ready for review June 5, 2025 08:18

daniel-mohedano requested review from a team as code owners June 5, 2025 08:18

daniel-mohedano requested a review from PerfectSlayer June 5, 2025 08:18

nikita-tkachenko-datadog reviewed Jun 5, 2025

View reviewed changes

daniel-mohedano added 4 commits June 5, 2025 15:04

remove FileDiff

2e3362f

correctly reset all settings tags for mock backend

4198fe3

create isBlank method to avoid double negation with isNotBlank

a99cff8

clean up base branch sha logic

f060e73

datadog-datadog-prod-us1 bot reviewed Jun 6, 2025

View reviewed changes

...nt/agent-ci-visibility/src/main/java/datadog/trace/civisibility/git/tree/ShellGitClient.java Outdated Show resolved Hide resolved

fix code violation

ce0da33

daniel-mohedano commented Jun 6, 2025

View reviewed changes

daniel-mohedano added 2 commits June 6, 2025 11:30

junit 4 test import wildcard

2970e68

use relative paths in remotes and add remote origin to resources

99980be

anmarchenko approved these changes Jun 16, 2025

View reviewed changes

PerfectSlayer reviewed Jun 17, 2025

View reviewed changes

internal-api/src/main/java/datadog/trace/util/Strings.java Show resolved Hide resolved

daniel-mohedano added 4 commits June 17, 2025 11:53

add javadoc and update tests for Strings.isBlank

a738112

Merge branch 'master' into daniel.mohedano/impacted-tests-update

612ba54

Merge branch 'master' into daniel.mohedano/impacted-tests-update

d21d3b7

Merge branch 'master' into daniel.mohedano/impacted-tests-update

67cdbe7

daniel-mohedano merged commit ae1f104 into master Jun 26, 2025
484 checks passed

daniel-mohedano deleted the daniel.mohedano/impacted-tests-update branch June 26, 2025 08:34

github-actions bot added this to the 1.51.0 milestone Jun 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update impacted tests logic #8923

Update impacted tests logic #8923

Uh oh!

daniel-mohedano commented Jun 4, 2025

Uh oh!

pr-commenter bot commented Jun 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daniel-mohedano Jun 6, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Update impacted tests logic #8923

Update impacted tests logic #8923

Uh oh!

Conversation

daniel-mohedano commented Jun 4, 2025

What Does This Do

Contributor Checklist

Uh oh!

pr-commenter bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daniel-mohedano Jun 6, 2025

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pr-commenter bot commented Jun 4, 2025 •

edited

Loading