Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial version of the native agent #8066

Draft
wants to merge 1 commit into
base: malvarez/iast-extrat-taint-interfaces
Choose a base branch
from

Conversation

manuel-alvarez-alvarez
Copy link
Member

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@manuel-alvarez-alvarez manuel-alvarez-alvarez added tag: performance Performance related changes comp: asm iast Application Security Management (IAST) labels Dec 6, 2024
@pr-commenter
Copy link

pr-commenter bot commented Dec 6, 2024

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch malvarez/iast-extrat-taint-interfaces malvarez/native-java-agent
git_commit_date 1733563153 1733691647
git_commit_sha 326f822 3484d30
See matching parameters
Baseline Candidate
ci_job_date 1733692689 1733692689
ci_job_id 730188729 730188729
ci_pipeline_id 50520409 50520409
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Dec 6, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/native-java-agent
git_commit_date 1733503294 1733691647
git_commit_sha 0db6312 3484d30
release_version 1.44.0-SNAPSHOT~0db63127b0 1.44.0-SNAPSHOT~3484d30055
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1733694076 1733694076
ci_job_id 730188725 730188725
ci_pipeline_id 50520409 50520409
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 7 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:iast:Remote Config better
[-69.417µs; -18.980µs] or [-10.409%; -2.846%]
622.666µs 666.864µs
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.097 s) : 0, 1097415
Total [baseline] (8.688 s) : 0, 8688038
Agent [candidate] (1.093 s) : 0, 1093081
Total [candidate] (8.648 s) : 0, 8647873
section iast
Agent [baseline] (1.219 s) : 0, 1219180
Total [baseline] (9.195 s) : 0, 9195369
Agent [candidate] (1.222 s) : 0, 1221907
Total [candidate] (9.227 s) : 0, 9226992
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.218 s) : 0, 1218108
Total [baseline] (9.165 s) : 0, 9164694
Agent [candidate] (1.22 s) : 0, 1220457
Total [candidate] (9.233 s) : 0, 9233471
section iast_TELEMETRY_OFF
Agent [baseline] (1.211 s) : 0, 1210805
Total [baseline] (9.157 s) : 0, 9156612
Agent [candidate] (1.217 s) : 0, 1216709
Total [candidate] (9.19 s) : 0, 9190132
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.097 s -
Agent iast 1.219 s 121.764 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.218 s 120.692 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.211 s 113.39 ms (10.3%)
Total tracing 8.688 s -
Total iast 9.195 s 507.331 ms (5.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.165 s 476.656 ms (5.5%)
Total iast_TELEMETRY_OFF 9.157 s 468.574 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.093 s -
Agent iast 1.222 s 128.825 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.22 s 127.376 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.217 s 123.627 ms (11.3%)
Total tracing 8.648 s -
Total iast 9.227 s 579.119 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.233 s 585.598 ms (6.8%)
Total iast_TELEMETRY_OFF 9.19 s 542.259 ms (6.3%)
gantt
    title insecure-bank - break down per module: candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (698.61 ms) : 0, 698610
BytebuddyAgent [candidate] (695.617 ms) : 0, 695617
GlobalTracer [baseline] (318.359 ms) : 0, 318359
GlobalTracer [candidate] (317.218 ms) : 0, 317218
AppSec [baseline] (55.205 ms) : 0, 55205
AppSec [candidate] (54.539 ms) : 0, 54539
Remote Config [baseline] (702.765 µs) : 0, 703
Remote Config [candidate] (714.232 µs) : 0, 714
Telemetry [baseline] (10.719 ms) : 0, 10719
Telemetry [candidate] (11.264 ms) : 0, 11264
section iast
BytebuddyAgent [baseline] (813.111 ms) : 0, 813111
BytebuddyAgent [candidate] (814.778 ms) : 0, 814778
GlobalTracer [baseline] (305.206 ms) : 0, 305206
GlobalTracer [candidate] (305.854 ms) : 0, 305854
AppSec [baseline] (57.845 ms) : 0, 57845
AppSec [candidate] (57.705 ms) : 0, 57705
IAST [baseline] (20.988 ms) : 0, 20988
IAST [candidate] (21.613 ms) : 0, 21613
Remote Config [baseline] (656.346 µs) : 0, 656
Remote Config [candidate] (638.032 µs) : 0, 638
Telemetry [baseline] (7.56 ms) : 0, 7560
Telemetry [candidate] (7.541 ms) : 0, 7541
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (812.026 ms) : 0, 812026
BytebuddyAgent [candidate] (812.946 ms) : 0, 812946
GlobalTracer [baseline] (305.324 ms) : 0, 305324
GlobalTracer [candidate] (305.993 ms) : 0, 305993
AppSec [baseline] (57.991 ms) : 0, 57991
AppSec [candidate] (58.428 ms) : 0, 58428
IAST [baseline] (20.859 ms) : 0, 20859
IAST [candidate] (21.042 ms) : 0, 21042
Remote Config [baseline] (648.441 µs) : 0, 648
Remote Config [candidate] (658.193 µs) : 0, 658
Telemetry [baseline] (7.536 ms) : 0, 7536
Telemetry [candidate] (7.615 ms) : 0, 7615
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (806.226 ms) : 0, 806226
BytebuddyAgent [candidate] (810.99 ms) : 0, 810990
GlobalTracer [baseline] (304.469 ms) : 0, 304469
GlobalTracer [candidate] (305.86 ms) : 0, 305860
AppSec [baseline] (56.246 ms) : 0, 56246
AppSec [candidate] (57.081 ms) : 0, 57081
IAST [baseline] (22.119 ms) : 0, 22119
IAST [candidate] (21.073 ms) : 0, 21073
Remote Config [baseline] (639.233 µs) : 0, 639
Remote Config [candidate] (621.966 µs) : 0, 622
Telemetry [baseline] (7.397 ms) : 0, 7397
Telemetry [candidate] (7.374 ms) : 0, 7374
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.09 s) : 0, 1090006
Total [baseline] (10.464 s) : 0, 10464049
Agent [candidate] (1.089 s) : 0, 1088717
Total [candidate] (10.415 s) : 0, 10414691
section appsec
Agent [baseline] (1.23 s) : 0, 1230493
Total [baseline] (10.787 s) : 0, 10786991
Agent [candidate] (1.226 s) : 0, 1225833
Total [candidate] (10.75 s) : 0, 10749700
section iast
Agent [baseline] (1.218 s) : 0, 1217868
Total [baseline] (10.975 s) : 0, 10974859
Agent [candidate] (1.218 s) : 0, 1218470
Total [candidate] (10.891 s) : 0, 10891041
section profiling
Agent [baseline] (1.314 s) : 0, 1314475
Total [baseline] (10.763 s) : 0, 10762627
Agent [candidate] (1.316 s) : 0, 1316025
Total [candidate] (10.772 s) : 0, 10772088
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.09 s -
Agent appsec 1.23 s 140.487 ms (12.9%)
Agent iast 1.218 s 127.862 ms (11.7%)
Agent profiling 1.314 s 224.469 ms (20.6%)
Total tracing 10.464 s -
Total appsec 10.787 s 322.942 ms (3.1%)
Total iast 10.975 s 510.81 ms (4.9%)
Total profiling 10.763 s 298.578 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.089 s -
Agent appsec 1.226 s 137.116 ms (12.6%)
Agent iast 1.218 s 129.753 ms (11.9%)
Agent profiling 1.316 s 227.307 ms (20.9%)
Total tracing 10.415 s -
Total appsec 10.75 s 335.009 ms (3.2%)
Total iast 10.891 s 476.35 ms (4.6%)
Total profiling 10.772 s 357.397 ms (3.4%)
gantt
    title petclinic - break down per module: candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.997 ms) : 0, 693997
BytebuddyAgent [candidate] (692.77 ms) : 0, 692770
GlobalTracer [baseline] (316.845 ms) : 0, 316845
GlobalTracer [candidate] (315.681 ms) : 0, 315681
AppSec [baseline] (54.337 ms) : 0, 54337
AppSec [candidate] (54.494 ms) : 0, 54494
Remote Config [baseline] (680.939 µs) : 0, 681
Remote Config [candidate] (685.459 µs) : 0, 685
Telemetry [baseline] (10.419 ms) : 0, 10419
Telemetry [candidate] (11.408 ms) : 0, 11408
section appsec
BytebuddyAgent [baseline] (714.703 ms) : 0, 714703
BytebuddyAgent [candidate] (711.387 ms) : 0, 711387
GlobalTracer [baseline] (315.774 ms) : 0, 315774
GlobalTracer [candidate] (314.566 ms) : 0, 314566
AppSec [baseline] (167.866 ms) : 0, 167866
AppSec [candidate] (168.558 ms) : 0, 168558
IAST [baseline] (18.987 ms) : 0, 18987
IAST [candidate] (18.986 ms) : 0, 18986
Remote Config [baseline] (657.892 µs) : 0, 658
Remote Config [candidate] (654.202 µs) : 0, 654
Telemetry [baseline] (8.249 ms) : 0, 8249
Telemetry [candidate] (7.46 ms) : 0, 7460
section iast
BytebuddyAgent [baseline] (810.458 ms) : 0, 810458
BytebuddyAgent [candidate] (812.876 ms) : 0, 812876
GlobalTracer [baseline] (305.334 ms) : 0, 305334
GlobalTracer [candidate] (305.136 ms) : 0, 305136
AppSec [baseline] (58.525 ms) : 0, 58525
AppSec [candidate] (57.924 ms) : 0, 57924
IAST [baseline] (21.476 ms) : 0, 21476
IAST [candidate] (20.741 ms) : 0, 20741
Remote Config [baseline] (666.864 µs) : 0, 667
Remote Config [candidate] (622.666 µs) : 0, 623
Telemetry [baseline] (7.711 ms) : 0, 7711
Telemetry [candidate] (7.458 ms) : 0, 7458
section profiling
BytebuddyAgent [baseline] (687.228 ms) : 0, 687228
BytebuddyAgent [candidate] (688.98 ms) : 0, 688980
GlobalTracer [baseline] (432.905 ms) : 0, 432905
GlobalTracer [candidate] (433.588 ms) : 0, 433588
AppSec [baseline] (53.638 ms) : 0, 53638
AppSec [candidate] (53.414 ms) : 0, 53414
Remote Config [baseline] (684.847 µs) : 0, 685
Remote Config [candidate] (653.165 µs) : 0, 653
Telemetry [baseline] (7.727 ms) : 0, 7727
Telemetry [candidate] (7.728 ms) : 0, 7728
ProfilingAgent [baseline] (93.298 ms) : 0, 93298
ProfilingAgent [candidate] (92.605 ms) : 0, 92605
Profiling [baseline] (93.322 ms) : 0, 93322
Profiling [candidate] (92.628 ms) : 0, 92628
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-12-08T21:11:21 2024-12-08T21:19:53
git_branch master malvarez/native-java-agent
git_commit_date 1733503294 1733691647
git_commit_sha 0db6312 3484d30
release_version 1.44.0-SNAPSHOT~0db63127b0 1.44.0-SNAPSHOT~3484d30055
start_time 2024-12-08T21:11:07 2024-12-08T21:19:39
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1733693238 1733693238
ci_job_id 730188726 730188726
ci_pipeline_id 50520409 50520409
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 20 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast_NATIVE_FULL better
[-97.262µs; -50.931µs] or [-14.876%; -7.790%]
unstable
[-660.250op/s; +2326.917op/s] or [-9.904%; +34.904%]
579.736µs 7500.000op/s 653.832µs 6666.667op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.366 ms) : 1346, 1385
.   : milestone, 1366,
appsec (1.77 ms) : 1746, 1795
.   : milestone, 1770,
appsec_no_iast (1.756 ms) : 1730, 1782
.   : milestone, 1756,
iast (1.512 ms) : 1490, 1535
.   : milestone, 1512,
iast_NATIVE (1.515 ms) : 1492, 1539
.   : milestone, 1515,
profiling (1.532 ms) : 1509, 1556
.   : milestone, 1532,
tracing (1.502 ms) : 1478, 1527
.   : milestone, 1502,
section candidate
no_agent (1.352 ms) : 1332, 1372
.   : milestone, 1352,
appsec (1.805 ms) : 1781, 1830
.   : milestone, 1805,
appsec_no_iast (1.761 ms) : 1736, 1785
.   : milestone, 1761,
iast (1.522 ms) : 1500, 1544
.   : milestone, 1522,
iast_NATIVE (1.509 ms) : 1486, 1533
.   : milestone, 1509,
profiling (1.511 ms) : 1487, 1534
.   : milestone, 1511,
tracing (1.499 ms) : 1473, 1524
.   : milestone, 1499,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.366 ms [1.346 ms, 1.385 ms] -
appsec 1.77 ms [1.746 ms, 1.795 ms] 404.933 µs (29.7%)
appsec_no_iast 1.756 ms [1.73 ms, 1.782 ms] 390.575 µs (28.6%)
iast 1.512 ms [1.49 ms, 1.535 ms] 146.72 µs (10.7%)
iast_NATIVE 1.515 ms [1.492 ms, 1.539 ms] 149.929 µs (11.0%)
profiling 1.532 ms [1.509 ms, 1.556 ms] 166.866 µs (12.2%)
tracing 1.502 ms [1.478 ms, 1.527 ms] 136.785 µs (10.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.332 ms, 1.372 ms] -
appsec 1.805 ms [1.781 ms, 1.83 ms] 453.486 µs (33.5%)
appsec_no_iast 1.761 ms [1.736 ms, 1.785 ms] 408.806 µs (30.2%)
iast 1.522 ms [1.5 ms, 1.544 ms] 170.056 µs (12.6%)
iast_NATIVE 1.509 ms [1.486 ms, 1.533 ms] 157.561 µs (11.7%)
profiling 1.511 ms [1.487 ms, 1.534 ms] 159.027 µs (11.8%)
tracing 1.499 ms [1.473 ms, 1.524 ms] 146.73 µs (10.9%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.699 µs) : 359, 400
.   : milestone, 380,
iast (495.788 µs) : 474, 517
.   : milestone, 496,
iast_FULL (659.237 µs) : 638, 681
.   : milestone, 659,
iast_GLOBAL (532.051 µs) : 510, 554
.   : milestone, 532,
iast_HARDCODED_SECRET_DISABLED (492.708 µs) : 472, 514
.   : milestone, 493,
iast_INACTIVE (459.652 µs) : 438, 482
.   : milestone, 460,
iast_NATIVE (493.028 µs) : 471, 515
.   : milestone, 493,
iast_NATIVE_FULL (653.832 µs) : 632, 676
.   : milestone, 654,
iast_TELEMETRY_OFF (489.7 µs) : 468, 511
.   : milestone, 490,
tracing (452.792 µs) : 432, 474
.   : milestone, 453,
section candidate
no_agent (385.93 µs) : 364, 408
.   : milestone, 386,
iast (504.731 µs) : 483, 526
.   : milestone, 505,
iast_FULL (665.394 µs) : 644, 687
.   : milestone, 665,
iast_GLOBAL (521.17 µs) : 500, 543
.   : milestone, 521,
iast_HARDCODED_SECRET_DISABLED (499.829 µs) : 478, 521
.   : milestone, 500,
iast_INACTIVE (457.792 µs) : 437, 479
.   : milestone, 458,
iast_NATIVE (506.733 µs) : 485, 529
.   : milestone, 507,
iast_NATIVE_FULL (579.736 µs) : 558, 601
.   : milestone, 580,
iast_TELEMETRY_OFF (486.901 µs) : 466, 508
.   : milestone, 487,
tracing (456.705 µs) : 435, 478
.   : milestone, 457,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.699 µs [359.431 µs, 399.967 µs] -
iast 495.788 µs [474.496 µs, 517.079 µs] 116.089 µs (30.6%)
iast_FULL 659.237 µs [637.77 µs, 680.704 µs] 279.538 µs (73.6%)
iast_GLOBAL 532.051 µs [509.88 µs, 554.222 µs] 152.352 µs (40.1%)
iast_HARDCODED_SECRET_DISABLED 492.708 µs [471.618 µs, 513.798 µs] 113.009 µs (29.8%)
iast_INACTIVE 459.652 µs [437.784 µs, 481.52 µs] 79.953 µs (21.1%)
iast_NATIVE 493.028 µs [471.498 µs, 514.558 µs] 113.329 µs (29.8%)
iast_NATIVE_FULL 653.832 µs [632.139 µs, 675.526 µs] 274.133 µs (72.2%)
iast_TELEMETRY_OFF 489.7 µs [468.114 µs, 511.286 µs] 110.001 µs (29.0%)
tracing 452.792 µs [432.018 µs, 473.566 µs] 73.093 µs (19.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.93 µs [363.789 µs, 408.071 µs] -
iast 504.731 µs [483.048 µs, 526.414 µs] 118.801 µs (30.8%)
iast_FULL 665.394 µs [643.766 µs, 687.022 µs] 279.463 µs (72.4%)
iast_GLOBAL 521.17 µs [499.591 µs, 542.749 µs] 135.24 µs (35.0%)
iast_HARDCODED_SECRET_DISABLED 499.829 µs [478.354 µs, 521.303 µs] 113.898 µs (29.5%)
iast_INACTIVE 457.792 µs [436.785 µs, 478.799 µs] 71.862 µs (18.6%)
iast_NATIVE 506.733 µs [484.818 µs, 528.648 µs] 120.803 µs (31.3%)
iast_NATIVE_FULL 579.736 µs [558.376 µs, 601.096 µs] 193.806 µs (50.2%)
iast_TELEMETRY_OFF 486.901 µs [465.793 µs, 508.009 µs] 100.971 µs (26.2%)
tracing 456.705 µs [435.227 µs, 478.182 µs] 70.774 µs (18.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master malvarez/native-java-agent
git_commit_date 1733503294 1733691647
git_commit_sha 0db6312 3484d30
release_version 1.44.0-SNAPSHOT~0db63127b0 1.44.0-SNAPSHOT~3484d30055
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1733693757 1733693757
ci_job_id 730188727 730188727
ci_pipeline_id 50520409 50520409
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 1 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
appsec (15.226 s) : 15226000, 15226000
.   : milestone, 15226000,
iast (18.689 s) : 18689000, 18689000
.   : milestone, 18689000,
iast_GLOBAL (17.935 s) : 17935000, 17935000
.   : milestone, 17935000,
iast_NATIVE (18.835 s) : 18835000, 18835000
.   : milestone, 18835000,
profiling (14.988 s) : 14988000, 14988000
.   : milestone, 14988000,
tracing (15.027 s) : 15027000, 15027000
.   : milestone, 15027000,
section candidate
no_agent (15.219 s) : 15219000, 15219000
.   : milestone, 15219000,
appsec (14.935 s) : 14935000, 14935000
.   : milestone, 14935000,
iast (18.86 s) : 18860000, 18860000
.   : milestone, 18860000,
iast_GLOBAL (17.737 s) : 17737000, 17737000
.   : milestone, 17737000,
iast_NATIVE (29.03 s) : 29030000, 29030000
.   : milestone, 29030000,
profiling (14.898 s) : 14898000, 14898000
.   : milestone, 14898000,
tracing (15.209 s) : 15209000, 15209000
.   : milestone, 15209000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.962 s [14.962 s, 14.962 s] -
appsec 15.226 s [15.226 s, 15.226 s] 264.0 ms (1.8%)
iast 18.689 s [18.689 s, 18.689 s] 3.727 s (24.9%)
iast_GLOBAL 17.935 s [17.935 s, 17.935 s] 2.973 s (19.9%)
iast_NATIVE 18.835 s [18.835 s, 18.835 s] 3.873 s (25.9%)
profiling 14.988 s [14.988 s, 14.988 s] 26.0 ms (0.2%)
tracing 15.027 s [15.027 s, 15.027 s] 65.0 ms (0.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.219 s [15.219 s, 15.219 s] -
appsec 14.935 s [14.935 s, 14.935 s] -284.0 ms (-1.9%)
iast 18.86 s [18.86 s, 18.86 s] 3.641 s (23.9%)
iast_GLOBAL 17.737 s [17.737 s, 17.737 s] 2.518 s (16.5%)
iast_NATIVE 29.03 s [29.03 s, 29.03 s] 13.811 s (90.7%)
profiling 14.898 s [14.898 s, 14.898 s] -321.0 ms (-2.1%)
tracing 15.209 s [15.209 s, 15.209 s] -10.0 ms (-0.1%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~3484d30055, baseline=1.44.0-SNAPSHOT~0db63127b0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1456, 1479
.   : milestone, 1468,
appsec (2.337 ms) : 2296, 2378
.   : milestone, 2337,
iast (2.089 ms) : 2036, 2142
.   : milestone, 2089,
iast_GLOBAL (2.127 ms) : 2074, 2179
.   : milestone, 2127,
iast_NATIVE (2.107 ms) : 2052, 2161
.   : milestone, 2107,
profiling (2.44 ms) : 2252, 2627
.   : milestone, 2440,
tracing (1.93 ms) : 1889, 1971
.   : milestone, 1930,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (2.333 ms) : 2291, 2375
.   : milestone, 2333,
iast (2.081 ms) : 2029, 2134
.   : milestone, 2081,
iast_GLOBAL (2.133 ms) : 2080, 2186
.   : milestone, 2133,
iast_NATIVE (2.109 ms) : 2054, 2163
.   : milestone, 2109,
profiling (1.962 ms) : 1918, 2005
.   : milestone, 1962,
tracing (1.929 ms) : 1889, 1969
.   : milestone, 1929,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.456 ms, 1.479 ms] -
appsec 2.337 ms [2.296 ms, 2.378 ms] 869.138 µs (59.2%)
iast 2.089 ms [2.036 ms, 2.142 ms] 621.58 µs (42.3%)
iast_GLOBAL 2.127 ms [2.074 ms, 2.179 ms] 658.807 µs (44.9%)
iast_NATIVE 2.107 ms [2.052 ms, 2.161 ms] 639.101 µs (43.5%)
profiling 2.44 ms [2.252 ms, 2.627 ms] 971.741 µs (66.2%)
tracing 1.93 ms [1.889 ms, 1.971 ms] 462.25 µs (31.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 2.333 ms [2.291 ms, 2.375 ms] 861.118 µs (58.5%)
iast 2.081 ms [2.029 ms, 2.134 ms] 609.639 µs (41.4%)
iast_GLOBAL 2.133 ms [2.08 ms, 2.186 ms] 661.527 µs (44.9%)
iast_NATIVE 2.109 ms [2.054 ms, 2.163 ms] 636.722 µs (43.3%)
profiling 1.962 ms [1.918 ms, 2.005 ms] 490.025 µs (33.3%)
tracing 1.929 ms [1.889 ms, 1.969 ms] 457.074 µs (31.1%)

@pr-commenter
Copy link

pr-commenter bot commented Dec 6, 2024

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch malvarez/iast-extrat-taint-interfaces malvarez/native-java-agent
git_commit_date 1733563153 1733691647
git_commit_sha 326f822 3484d30
See matching parameters
Baseline Candidate
ci_job_date 1733692724 1733692724
ci_job_id 730188730 730188730
ci_pipeline_id 50520409 50520409
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume unsure
[+2616.952op/s; +9890.402op/s] or [+0.869%; +3.285%]
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/native-java-agent branch 4 times, most recently from 84aadf7 to b83c27f Compare December 6, 2024 22:34
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/iast-extrat-taint-interfaces branch from 471f00b to 326f822 Compare December 7, 2024 09:19
@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/native-java-agent branch 2 times, most recently from d49265a to 61445ac Compare December 7, 2024 11:10
Comment on lines 23 to 30
public IastGlobalContextProvider() {
TaintedObjects to =
TaintedObjectsMap.build(TaintedMap.buildWithPurge(MAP_SIZE, MAX_AGE, MAX_AGE_UNIT));
if (NativeTaintedUtils.isEnabled()) {
to = new NativeTaintedObjectsAdapter(to);
}
taintedObjects = to;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Consider adding super() or this() to your constructor (...read more)

In Java, it is suggested to call super() in an extended class. This rule will report a violation if both a call to super() and an overloaded constructor are absent.

View in Datadog  Leave us feedback  Documentation

@manuel-alvarez-alvarez manuel-alvarez-alvarez force-pushed the malvarez/native-java-agent branch 2 times, most recently from 5af331e to 90a7d83 Compare December 8, 2024 19:14

void setRanges(final Range[] ranges);
void setRanges(final Object[] ranges);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Suggested change
void setRanges(final Object[] ranges);
void setRanges(Object[] ranges);
Avoid declaring as final as it may not be used (...read more)

The rule "Avoid useless final type in interface method" advises against the unnecessary use of the final keyword in the method parameters of an interface. In Java, the final keyword is used to denote that a variable cannot be changed once assigned. However, in the context of an interface method, this is redundant as the value of the parameter cannot be changed within the method anyway.

The importance of this rule lies in the clarity and simplicity of code. Unnecessary use of final in this context can lead to confusion for those reading the code, as it suggests that there may be a specific reason for its use when there is not. It can also clutter the code, making it less readable.

Good coding practices to avoid this rule violation include simply not using the final keyword in the method parameters of an interface. This does not affect the functionality of the code, but it makes it cleaner and easier to understand. For example, instead of writing void process(final Object arg);, you can write void process(Object arg);. This maintains the same functionality but improves the readability of the code.

View in Datadog  Leave us feedback  Documentation

@manuel-alvarez-alvarez manuel-alvarez-alvarez changed the title Initial version of the native agent with tagging Initial version of the native agent Dec 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: asm iast Application Security Management (IAST) tag: performance Performance related changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant