Skip to content

Fix logged exception for dependency URIs representing directories #6792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 11, 2024
Merged

Fix logged exception for dependency URIs representing directories #6792

merged 1 commit into from
Mar 11, 2024

Conversation

@smola
Copy link
Member

@smola smola commented Mar 8, 2024 •
edited by atlassian bot
Loading

What Does This Do

Check if a dependency location is a directory (e.g. an unpacked classes directory), including directories within JARs, and handle it gracefully.

Motivation

As of v1.30.0, this would generate an internal exception, which is caught, but result in debug logs like:

[dd.trace 2024-03-08 13:32:34:092 +0100] [dd-task-scheduler] DEBUG datadog.telemetry.dependency.DependencyResolver - Failed to determine dependency for uri jar:file:/work/dd-trace-java/dd-smoke-tests/spring-boot-3.0-webmvc/build/application/libs/webmvc-3.0-smoketest.jar!/BOOT-INF/classes!/
java.io.FileNotFoundException: /work/dd-trace-java/dd-smoke-tests/spring-boot-3.0-webmvc/build/application/libs/webmvc-3.0-smoketest.jar!/BOOT-INF/classes! (No such file or directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:216)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:111)
        at datadog.telemetry.dependency.DependencyResolver.resolve(DependencyResolver.java:43)
        at datadog.telemetry.dependency.DependencyResolverQueue.pollDependency(DependencyResolverQueue.java:82)
        at datadog.telemetry.dependency.DependencyService.resolveOneDependency(DependencyService.java:38)
        at datadog.telemetry.dependency.DependencyService.run(DependencyService.java:96)
        at datadog.trace.util.AgentTaskScheduler$RunnableTask.run(AgentTaskScheduler.java:41)
        at datadog.trace.util.AgentTaskScheduler$RunnableTask.run(AgentTaskScheduler.java:36)
        at datadog.trace.util.AgentTaskScheduler$PeriodicTask.run(AgentTaskScheduler.java:311)
        at datadog.trace.util.AgentTaskScheduler$Worker.run(AgentTaskScheduler.java:266)
        at java.base/java.lang.Thread.run(Thread.java:833)

Additional Notes

Jira ticket: APPSEC-52172

@smola smola added the comp: telemetry Telemetry label Mar 8, 2024
@smola smola requested a review from a team March 8, 2024 14:49
@smola smola requested a review from a team as a code owner March 8, 2024 14:49
@smola smola added the type: bug Bug report and fix label Mar 8, 2024
@smola smola mentioned this pull request Mar 8, 2024
Merged
smola
smola commented Mar 11, 2024
View reviewed changes
telemetry/src/main/java/datadog/telemetry/dependency/DependencyResolver.java
log.debug("unsupported dependency type: {}", uri);
return Collections.emptyList();
}
if (metadata.isDirectory) {
Copy link
Member Author

@smola smola Mar 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the method move, this is the block that changes in this file for this PR.

@smola smola force-pushed the smola/APPSEC-52172-dependency-dir branch from 9f7abd9 to f255e86 Compare March 11, 2024 07:18
@pr-commenter
Copy link

pr-commenter bot commented Mar 11, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/APPSEC-52172-dependency-dir
git_commit_date 1709909278 1710141507
git_commit_sha e9c489f f255e86
release_version 1.32.0-SNAPSHOT~e9c489fd8c 1.32.0-SNAPSHOT~f255e863fa
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710144712 1710144712
ci_job_id 455973565 455973565
ci_pipeline_id 29873008 29873008
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~f255e863fa, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.091 s) : 0, 1090642
Total [baseline] (8.566 s) : 0, 8566473
Agent [candidate] (1.091 s) : 0, 1091094
Total [candidate] (8.605 s) : 0, 8604613
section iast
Agent [baseline] (1.214 s) : 0, 1214483
Total [baseline] (9.028 s) : 0, 9028489
Agent [candidate] (1.217 s) : 0, 1217305
Total [candidate] (9.055 s) : 0, 9055130
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.21 s) : 0, 1209592
Total [baseline] (9.034 s) : 0, 9034456
Agent [candidate] (1.215 s) : 0, 1214692
Total [candidate] (9.004 s) : 0, 9003623
section iast_TELEMETRY_OFF
Agent [baseline] (1.2 s) : 0, 1200268
Total [baseline] (9.025 s) : 0, 9025286
Agent [candidate] (1.201 s) : 0, 1200703
Total [candidate] (9.038 s) : 0, 9038035
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent iast 1.214 s 123.841 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.21 s 118.95 ms (10.9%)
Agent iast_TELEMETRY_OFF 1.2 s 109.626 ms (10.1%)
Total tracing 8.566 s -
Total iast 9.028 s 462.016 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.034 s 467.984 ms (5.5%)
Total iast_TELEMETRY_OFF 9.025 s 458.814 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent iast 1.217 s 126.211 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.215 s 123.598 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.201 s 109.609 ms (10.0%)
Total tracing 8.605 s -
Total iast 9.055 s 450.518 ms (5.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.004 s 399.01 ms (4.6%)
Total iast_TELEMETRY_OFF 9.038 s 433.423 ms (5.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~f255e863fa, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (703.099 ms) : 0, 703099
BytebuddyAgent [candidate] (702.668 ms) : 0, 702668
GlobalTracer [baseline] (293.375 ms) : 0, 293375
GlobalTracer [candidate] (294.341 ms) : 0, 294341
AppSec [baseline] (50.864 ms) : 0, 50864
AppSec [candidate] (50.984 ms) : 0, 50984
Remote Config [baseline] (744.703 µs) : 0, 745
Remote Config [candidate] (728.471 µs) : 0, 728
Telemetry [baseline] (7.838 ms) : 0, 7838
Telemetry [candidate] (7.788 ms) : 0, 7788
section iast
BytebuddyAgent [baseline] (806.901 ms) : 0, 806901
BytebuddyAgent [candidate] (807.895 ms) : 0, 807895
GlobalTracer [baseline] (289.937 ms) : 0, 289937
GlobalTracer [candidate] (290.551 ms) : 0, 290551
AppSec [baseline] (53.668 ms) : 0, 53668
AppSec [candidate] (54.522 ms) : 0, 54522
Remote Config [baseline] (582.318 µs) : 0, 582
Remote Config [candidate] (578.867 µs) : 0, 579
Telemetry [baseline] (6.719 ms) : 0, 6719
Telemetry [candidate] (7.465 ms) : 0, 7465
IAST [baseline] (22.02 ms) : 0, 22020
IAST [candidate] (21.555 ms) : 0, 21555
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (802.311 ms) : 0, 802311
BytebuddyAgent [candidate] (805.92 ms) : 0, 805920
GlobalTracer [baseline] (288.645 ms) : 0, 288645
GlobalTracer [candidate] (290.315 ms) : 0, 290315
AppSec [baseline] (53.626 ms) : 0, 53626
AppSec [candidate] (53.963 ms) : 0, 53963
Remote Config [baseline] (588.124 µs) : 0, 588
Remote Config [candidate] (581.732 µs) : 0, 582
Telemetry [baseline] (7.515 ms) : 0, 7515
Telemetry [candidate] (6.754 ms) : 0, 6754
IAST [baseline] (22.468 ms) : 0, 22468
IAST [candidate] (22.627 ms) : 0, 22627
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (793.924 ms) : 0, 793924
BytebuddyAgent [candidate] (794.071 ms) : 0, 794071
GlobalTracer [baseline] (289.306 ms) : 0, 289306
GlobalTracer [candidate] (289.538 ms) : 0, 289538
AppSec [baseline] (54.864 ms) : 0, 54864
AppSec [candidate] (51.404 ms) : 0, 51404
Remote Config [baseline] (605.982 µs) : 0, 606
Remote Config [candidate] (591.719 µs) : 0, 592
Telemetry [baseline] (6.545 ms) : 0, 6545
Telemetry [candidate] (7.999 ms) : 0, 7999
IAST [baseline] (20.691 ms) : 0, 20691
IAST [candidate] (22.66 ms) : 0, 22660
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~f255e863fa, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.08 s) : 0, 1080385
Total [baseline] (9.168 s) : 0, 9168107
Agent [candidate] (1.081 s) : 0, 1081153
Total [candidate] (9.157 s) : 0, 9157431
section appsec
Agent [baseline] (1.208 s) : 0, 1207604
Total [baseline] (9.321 s) : 0, 9320792
Agent [candidate] (1.222 s) : 0, 1221765
Total [candidate] (9.337 s) : 0, 9336687
section iast
Agent [baseline] (1.218 s) : 0, 1218081
Total [baseline] (9.325 s) : 0, 9325373
Agent [candidate] (1.217 s) : 0, 1217212
Total [candidate] (9.329 s) : 0, 9328813
section profiling
Agent [baseline] (1.275 s) : 0, 1275180
Total [baseline] (9.415 s) : 0, 9415421
Agent [candidate] (1.276 s) : 0, 1275675
Total [candidate] (9.357 s) : 0, 9356605
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.08 s -
Agent appsec 1.208 s 127.219 ms (11.8%)
Agent iast 1.218 s 137.696 ms (12.7%)
Agent profiling 1.275 s 194.795 ms (18.0%)
Total tracing 9.168 s -
Total appsec 9.321 s 152.685 ms (1.7%)
Total iast 9.325 s 157.266 ms (1.7%)
Total profiling 9.415 s 247.315 ms (2.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.222 s 140.612 ms (13.0%)
Agent iast 1.217 s 136.059 ms (12.6%)
Agent profiling 1.276 s 194.522 ms (18.0%)
Total tracing 9.157 s -
Total appsec 9.337 s 179.256 ms (2.0%)
Total iast 9.329 s 171.382 ms (1.9%)
Total profiling 9.357 s 199.174 ms (2.2%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~f255e863fa, baseline=1.32.0-SNAPSHOT~e9c489fd8c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (695.745 ms) : 0, 695745
BytebuddyAgent [candidate] (696.181 ms) : 0, 696181
GlobalTracer [baseline] (291.218 ms) : 0, 291218
GlobalTracer [candidate] (291.655 ms) : 0, 291655
AppSec [baseline] (50.79 ms) : 0, 50790
AppSec [candidate] (50.57 ms) : 0, 50570
Remote Config [baseline] (718.101 µs) : 0, 718
Remote Config [candidate] (729.134 µs) : 0, 729
Telemetry [baseline] (7.669 ms) : 0, 7669
Telemetry [candidate] (7.738 ms) : 0, 7738
section appsec
BytebuddyAgent [baseline] (700.671 ms) : 0, 700671
BytebuddyAgent [candidate] (707.975 ms) : 0, 707975
GlobalTracer [baseline] (291.105 ms) : 0, 291105
GlobalTracer [candidate] (296.008 ms) : 0, 296008
AppSec [baseline] (155.901 ms) : 0, 155901
AppSec [candidate] (157.206 ms) : 0, 157206
Remote Config [baseline] (610.868 µs) : 0, 611
Remote Config [candidate] (624.646 µs) : 0, 625
Telemetry [baseline] (6.873 ms) : 0, 6873
Telemetry [candidate] (6.998 ms) : 0, 6998
IAST [baseline] (17.853 ms) : 0, 17853
IAST [candidate] (18.088 ms) : 0, 18088
section iast
BytebuddyAgent [baseline] (809.37 ms) : 0, 809370
BytebuddyAgent [candidate] (808.134 ms) : 0, 808134
GlobalTracer [baseline] (289.99 ms) : 0, 289990
GlobalTracer [candidate] (290.919 ms) : 0, 290919
AppSec [baseline] (54.892 ms) : 0, 54892
AppSec [candidate] (53.105 ms) : 0, 53105
Remote Config [baseline] (585.032 µs) : 0, 585
Remote Config [candidate] (592.063 µs) : 0, 592
Telemetry [baseline] (6.674 ms) : 0, 6674
Telemetry [candidate] (7.385 ms) : 0, 7385
IAST [baseline] (21.623 ms) : 0, 21623
IAST [candidate] (22.479 ms) : 0, 22479
section profiling
BytebuddyAgent [baseline] (688.268 ms) : 0, 688268
BytebuddyAgent [candidate] (688.108 ms) : 0, 688108
GlobalTracer [baseline] (374.528 ms) : 0, 374528
GlobalTracer [candidate] (375.304 ms) : 0, 375304
AppSec [baseline] (52.79 ms) : 0, 52790
AppSec [candidate] (53.063 ms) : 0, 53063
Remote Config [baseline] (765.72 µs) : 0, 766
Remote Config [candidate] (697.614 µs) : 0, 698
Telemetry [baseline] (7.411 ms) : 0, 7411
Telemetry [candidate] (7.465 ms) : 0, 7465
ProfilingAgent [baseline] (95.184 ms) : 0, 95184
ProfilingAgent [candidate] (94.939 ms) : 0, 94939
Profiling [baseline] (95.207 ms) : 0, 95207
Profiling [candidate] (94.963 ms) : 0, 94963
Loading

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~f255e863fa, baseline=1.32.0-SNAPSHOT~e9c489fd8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (363.033 µs) : 343, 383
.   : milestone, 363,
iast (480.362 µs) : 460, 501
.   : milestone, 480,
iast_FULL (535.18 µs) : 514, 556
.   : milestone, 535,
iast_GLOBAL (486.713 µs) : 467, 507
.   : milestone, 487,
iast_HARDCODED_SECRET_DISABLED (465.014 µs) : 444, 486
.   : milestone, 465,
iast_INACTIVE (452.917 µs) : 431, 475
.   : milestone, 453,
iast_TELEMETRY_OFF (464.324 µs) : 444, 485
.   : milestone, 464,
tracing (443.444 µs) : 423, 464
.   : milestone, 443,
section candidate
no_agent (363.11 µs) : 343, 383
.   : milestone, 363,
iast (471.065 µs) : 450, 492
.   : milestone, 471,
iast_FULL (532.694 µs) : 512, 553
.   : milestone, 533,
iast_GLOBAL (493.362 µs) : 473, 514
.   : milestone, 493,
iast_HARDCODED_SECRET_DISABLED (468.301 µs) : 448, 489
.   : milestone, 468,
iast_INACTIVE (446.128 µs) : 426, 466
.   : milestone, 446,
iast_TELEMETRY_OFF (458.11 µs) : 438, 478
.   : milestone, 458,
tracing (437.393 µs) : 417, 458
.   : milestone, 437,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.033 µs [343.157 µs, 382.908 µs] -
iast 480.362 µs [459.918 µs, 500.807 µs] 117.33 µs (32.3%)
iast_FULL 535.18 µs [514.458 µs, 555.903 µs] 172.148 µs (47.4%)
iast_GLOBAL 486.713 µs [466.517 µs, 506.908 µs] 123.68 µs (34.1%)
iast_HARDCODED_SECRET_DISABLED 465.014 µs [444.429 µs, 485.598 µs] 101.981 µs (28.1%)
iast_INACTIVE 452.917 µs [431.308 µs, 474.527 µs] 89.885 µs (24.8%)
iast_TELEMETRY_OFF 464.324 µs [443.529 µs, 485.118 µs] 101.291 µs (27.9%)
tracing 443.444 µs [422.964 µs, 463.924 µs] 80.411 µs (22.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.11 µs [343.247 µs, 382.972 µs] -
iast 471.065 µs [450.167 µs, 491.963 µs] 107.955 µs (29.7%)
iast_FULL 532.694 µs [512.295 µs, 553.093 µs] 169.585 µs (46.7%)
iast_GLOBAL 493.362 µs [472.637 µs, 514.086 µs] 130.252 µs (35.9%)
iast_HARDCODED_SECRET_DISABLED 468.301 µs [447.894 µs, 488.708 µs] 105.191 µs (29.0%)
iast_INACTIVE 446.128 µs [425.817 µs, 466.439 µs] 83.018 µs (22.9%)
iast_TELEMETRY_OFF 458.11 µs [437.868 µs, 478.351 µs] 95.0 µs (26.2%)
tracing 437.393 µs [417.139 µs, 457.648 µs] 74.284 µs (20.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~f255e863fa, baseline=1.32.0-SNAPSHOT~e9c489fd8c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.336 ms) : 1317, 1355
.   : milestone, 1336,
appsec (1.789 ms) : 1765, 1813
.   : milestone, 1789,
iast (1.507 ms) : 1483, 1531
.   : milestone, 1507,
profiling (1.52 ms) : 1496, 1544
.   : milestone, 1520,
tracing (1.516 ms) : 1493, 1539
.   : milestone, 1516,
section candidate
no_agent (1.359 ms) : 1340, 1378
.   : milestone, 1359,
appsec (1.751 ms) : 1727, 1775
.   : milestone, 1751,
iast (1.52 ms) : 1496, 1543
.   : milestone, 1520,
profiling (1.512 ms) : 1488, 1535
.   : milestone, 1512,
tracing (1.523 ms) : 1500, 1546
.   : milestone, 1523,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.336 ms [1.317 ms, 1.355 ms] -
appsec 1.789 ms [1.765 ms, 1.813 ms] 452.876 µs (33.9%)
iast 1.507 ms [1.483 ms, 1.531 ms] 170.885 µs (12.8%)
profiling 1.52 ms [1.496 ms, 1.544 ms] 183.728 µs (13.8%)
tracing 1.516 ms [1.493 ms, 1.539 ms] 179.593 µs (13.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.359 ms [1.34 ms, 1.378 ms] -
appsec 1.751 ms [1.727 ms, 1.775 ms] 391.987 µs (28.8%)
iast 1.52 ms [1.496 ms, 1.543 ms] 160.69 µs (11.8%)
profiling 1.512 ms [1.488 ms, 1.535 ms] 152.855 µs (11.2%)
tracing 1.523 ms [1.5 ms, 1.546 ms] 164.27 µs (12.1%)

@smola smola merged commit aee5f6f into master Mar 11, 2024
@smola smola deleted the smola/APPSEC-52172-dependency-dir branch March 11, 2024 10:19
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 11, 2024
smola added a commit that referenced this pull request Mar 12, 2024
@smola
Fix logged exception for dependency URIs representing directories (#6792
61f87d8 
)
@smola smola mentioned this pull request Mar 12, 2024
Merged
smola added a commit that referenced this pull request Mar 12, 2024
@smola
Fix logged exception for dependency URIs representing directories (#6792
d2fd917 
) (#6802)
@smola smola modified the milestones: 1.32.0, 1.31.2 Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes

@ValentinZakharov ValentinZakharov Awaiting requested review from ValentinZakharov ValentinZakharov was automatically assigned from DataDog/asm-java

@mcculls mcculls Awaiting requested review from mcculls mcculls was automatically assigned from DataDog/apm-java

@ygree ygree Awaiting requested review from ygree ygree was automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: telemetry Telemetry type: bug Bug report and fix

Projects

None yet

Milestone

1.31.2

Development

Successfully merging this pull request may close these issues.

3 participants

@smola @manuel-alvarez-alvarez