Skip to content

Conversation

@bric3
Copy link
Contributor

@bric3 bric3 commented Sep 5, 2025

What Does This Do

⚠️ work in progress

This work explore a rewrite of the InstrumentPlugin to avoid eager configuration.

At the same time, I'm exploring the instrument post-processing to be a compiler last action,
instead of adding its own task to run after compilation and intercepting itself.

Motivation

  • Good usage of Gradle API
  • Make it lazy

Related to

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@bric3 bric3 requested review from a team as code owners September 5, 2025 12:50
@bric3 bric3 requested a review from amarziali September 5, 2025 12:50
@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2025

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@bric3 bric3 marked this pull request as draft September 5, 2025 12:50
@bric3 bric3 added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Sep 5, 2025
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, what would be the benefit? Faster build?

Comment on lines 41 to 44
project.pluginManager.withPlugin("java") {configurePostCompilationInstrumentation("java", project, extension) }
project.pluginManager.withPlugin("kotlin") {configurePostCompilationInstrumentation("kotlin", project, extension) }
project.pluginManager.withPlugin("scala") {configurePostCompilationInstrumentation("scala", project, extension) }
project.pluginManager.withPlugin("groovy") {configurePostCompilationInstrumentation("groovy", project, extension) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this way would be a bit more elegant? WDYT?

Suggested change
project.pluginManager.withPlugin("java") {configurePostCompilationInstrumentation("java", project, extension) }
project.pluginManager.withPlugin("kotlin") {configurePostCompilationInstrumentation("kotlin", project, extension) }
project.pluginManager.withPlugin("scala") {configurePostCompilationInstrumentation("scala", project, extension) }
project.pluginManager.withPlugin("groovy") {configurePostCompilationInstrumentation("groovy", project, extension) }
['java', 'kotlin', 'scala', 'groovy'].each { id ->
project.pluginManager.withPlugin(id) {
configurePostCompilationInstrumentation(id, project, extension)
}
}

*/
@SuppressWarnings('unused')
class InstrumentPlugin implements Plugin<Project> {
public static final String DEFAULT_JAVA_VERSION = 'default'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why it is default and not 8?

Copy link
Contributor Author

@bric3 bric3 Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! In the previous code javaVersion was null if not explicitly set, but declaring an additional task input via it.inputs.property("javaVersion", javaVersion) do not allow null, so I introduced a default value. The java version is maily used to affect which workQueue is chosen, so using default or 8 has no consequences in this PR.

That being said, decoupling the Java target version from the Gradle JVM will have an effect and in this case we will have to be handled, but that's a separate work.

project.afterEvaluate { p ->
instrumentJava.dependsOn compileMain_play27Java
forbiddenApisMain_play27.dependsOn instrumentMain_play27Java
// instrumentJava.dependsOn compileMain_play27Java
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like missing TODO remove here too?

Copy link
Contributor Author

@bric3 bric3 Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you are too early for the review 😅
I need to work a bit here and there on this

}
apply plugin: 'instrument'
// apply plugin: 'instrument'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to remove?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, will wait a bit 👍

@bric3 bric3 force-pushed the bdu/lazy-instrument-plugin branch from 248b4eb to 3094d50 Compare September 5, 2025 17:02
@bric3
Copy link
Contributor Author

bric3 commented Sep 10, 2025

Status report

The new approach of the instrument post-processing works in almost all simple projects, e.g.

Important

The diff was made by this custom tool https://github.com/bric3/jardiff

:dd-java-agent:instrumentation:iast-instrumenter
$ mise exec java@corretto-24 -- jardiff /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java{-copy-1,}/dd-java-agent/instrumentation/iast-instrumenter/build/classes/java/main                      17:01:29
Comparing:
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java-copy-1/dd-java-agent/instrumentation/iast-instrumenter/build/classes/java/main
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java/dd-java-agent/instrumentation/iast-instrumenter/build/classes/java/main

✔️ META-INF/services/datadog.trace.agent.tooling.InstrumenterModule
✔️ datadog/trace/instrumentation/iastinstrumenter/IastExclusionTrie.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$IastMatchers.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastHardcodedSecretListener$ReportSecretConsumer.class
✔️ datadog/trace/instrumentation/iastinstrumenter/SourceMapperImpl.class
✔️ datadog/trace/instrumentation/iastinstrumenter/StratumListener.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastHardcodedSecretListener.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$IastCallSiteSupplier.class
✔️ datadog/trace/instrumentation/iastinstrumenter/IastInstrumentation$IastMatchers$1.class
✔️ datadog/trace/instrumentation/iastinstrumenter/telemetry/TelemetryCallSiteSupplier$IteratorAdapter.class
✔️ datadog/trace/instrumentation/iastinstrumenter/telemetry/TelemetryCallSiteSupplier$1.class
✔️ datadog/trace/instrumentation/iastinstrumenter/telemetry/TelemetryCallSiteSupplier.class
✔️ datadog/trace/instrumentation/iastinstrumenter/service/CallSitesLoader.class
:dd-java-agent:instrumentation:mule-4
$  mise exec java@corretto-24 -- jardiff /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java{-copy-1,}/dd-java-agent/instrumentation/mule-4/build/classes/java/main                                 17:26:08
Comparing:
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java-copy-1/dd-java-agent/instrumentation/mule-4/build/classes/java/main
* /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java/dd-java-agent/instrumentation/mule-4/build/classes/java/main

✔️ META-INF/services/datadog.trace.agent.tooling.InstrumenterModule
✔️ datadog/trace/instrumentation/mule4/EventTracerInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/ComponentMessageProcessorInstrumentation$ProcessAdvice.class
✔️ datadog/trace/instrumentation/mule4/AbstractMuleInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/EventTracerInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/SpanState.class
✔️ datadog/trace/instrumentation/mule4/ComponentMessageProcessorInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/JpmsMuleInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/ExecutionInitialSpanInfoInstrumentation$StoreComponentAdvice.class
✔️ datadog/trace/instrumentation/mule4/EventContextInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/ComponentMessageProcessorInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/EventContextCreationAdvice.class
✔️ datadog/trace/instrumentation/mule4/NoopMuleSpan.class
✔️ datadog/trace/instrumentation/mule4/DDEventTracer.class
✔️ datadog/trace/instrumentation/mule4/ExecutionInitialSpanInfoInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/MuleDecorator.class
✔️ datadog/trace/instrumentation/mule4/EventTracerInstrumentation$SwapCoreTracerAdvice.class
✔️ datadog/trace/instrumentation/mule4/JpmsMuleInstrumentation.class
✔️ datadog/trace/instrumentation/mule4/ExecutionInitialSpanInfoInstrumentation$Muzzle.class
✔️ datadog/trace/instrumentation/mule4/EventContextInstrumentation.class
:dd-java-agent:instrumentation:play.2.4
:dd-java-agent:agent-otel:otel-bootstrap

jardiff on otel-bootstrap

Etc.

However, :dd-java-agent:instrumentation:jetty-9 is oddly configured and must be

mise exec java@corretto-24 -- jardiff /Users/brice.dutheil/go/src/github.com/DataDog/dd-trace-java{-copy-1,}/dd-java-agent/instrumentation/jetty-9/build/classes/java/main

And as such not all classes are available when post processing kicks in. Possibly this project needs a heavier setup refactoring.

Few other projects with non-standard configurations to fix

  • :dd-java-agent:instrumentation:play.2.4
  • :dd-java-agent:instrumentation:play.2.6

Executing a simple ./gradlew help :

~5K less created tasks during configuration
image

https://scans.gradle.com/s/jqasddzbs75ve/performance/configuration#summary-tasks-created-during-configuration

Note

~40k tasks created is still way too much and impacts the build performance considerably

versus master, with ~45k task created during configuration
image

https://scans.gradle.com/s/vg7nn7oitpx5k/performance/configuration#summary-tasks-created-during-configuration

@bric3 bric3 force-pushed the bdu/lazy-instrument-plugin branch from aa6d660 to 0067ada Compare September 10, 2025 16:23
@bric3 bric3 added type: enhancement Enhancements and improvements tag: needs investigation Issues needing investigations type: refactoring labels Sep 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling tag: needs investigation Issues needing investigations tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants