-
Notifications
You must be signed in to change notification settings - Fork 458
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial version of Spotless Maven plugin #188
Conversation
Commit introduces a prototype of the Maven plugin which is able to apply Eclipse formatter step to all java files in the project. It only takes a single parameter - 'eclipseFormatFile' which is a path to the formatter configuration. Maven plugin is build and packaged using Maven which is executed from the Gradle build script via Exec plugin. This seems to be the simplest way to get a JAR with "maven-plugin" packaging. Gradle does not have a native plugin to do this. This change just adds a bare minimum to verify the approach.
Huzzah!! Fireworks!!! Thanks so much @lutovich, great to finally have this off the ground! Having a functional plugin and implementation of
As steps are added and improved, Uploading this plugin to mavenCentral on each release will be trivial, I'll have to turn some cranks for the initial release but it'll be automatic from there.
Do maven plugins require that they have no dependencies? It seems that if we remove the shading, the build will get a lot simpler. The gradle project has the
I trust your research on the tradeoffs of creating maven metadata manually vs shelling out to maven. I would love to switch to
Without being inside I'm 100% fine with v1 of the maven plugin supporting only a single formatter step - this was definitely the right place to start. Just to establish what we'd need to reach near-parity with the gradle plugin:
The natural way for Spotless to work is:
I've got no idea if a structure anything like that is possible in the
If we can get v1 to support at least all of |
Hi @nedtwigg, thanks for the response!
I think it's fine for a maven plugin to have dependencies. Shading was added because plugin needs if (!project.versionGradle.endsWith('-SNAPSHOT') && project.versionLib.endsWith('-SNAPSHOT')) {
// gradle = release, lib = snapshot, therefore gradle should depend on the last stable lib
compile "com.diffplug.spotless:spotless-lib:${project.stableLib}"
compile "com.diffplug.spotless:spotless-lib-extra:${project.stableLib}"
} else {
compile project(':lib')
compile project(':lib-extra')
} I think
Can't fully assess the complexity of other build steps right now. Feels like it should be easy to add them when configuration and Feels like I have 3 main things to work on next:
Does these look fine to you? Thanks! |
Oh and one other small question. Do you prefer to treat this PR as a feature branch? I can keep adding commits to this branch and it can be merged when stable. Another option is to prettify it a bit then merge then create other small PRs for subsequent improvements. |
Yup! The point of the "else" block that you point out is to ensure that a published gradle plugin never accidentally depends on a SNAPSHOT version of the lib. That happened once on accident, so we put that in to prevent a relapse.
My preference is to treat this as a feature branch for now. If we hit a forking point (maybe we should do A or B) then I'd be happy to merge. |
I took the liberty of |
Exposed `encoding` and `lineEndings` in the plugin xml configuration. Renamed `eclipseFormatFile` to `eclipseConfigFile` and moved it inside the `<java>` tag. Maven plugins can handle nested configs using Plexus dependency injection and POJO objects. Thus introduced a dedicated class for all java related configuration called `Java`. Plugin xml configuration can now look like: ``` <plugin> <groupId>com.diffplug.spotless</groupId> <artifactId>spotless-maven-plugin</artifactId> <version>${spotless.version}</version> <configuration> <encoding>UTF-8</encoding> <lineEndings>UNIX</lineEndings> <java> <eclipseConfigFile>${basedir}/eclipse-fmt.xml</eclipseConfigFile> </java> </configuration> </plugin> ``` Extracted an abstract `AbstractSpotlessMojo` to hold all injected dependencies. It can be potentially used in future to implement the `check` goal. Changed name of `SpotlessMojo` from "spotless" to "apply" so that it can be invoked with `mvn spotless:apply`.
9d62ee3
to
c219ef1
Compare
Thanks @nedtwigg. Added another commit that improves config handling. It makes use of nested xml tags, like you suggested. Will next simplify build process to use |
Injected `MavenProject` is now replaced by injection of multiple primitive values, like `project.basedir`.
Removed shading & local repository for `lib` and `lib-extra`. Instead made plugin directly depend on stable version denoted by `project.stableLib`. Made all maven plugin API dependencies provided and their versions injected by the Gradle build script.
These don't match the current implementation exactly, but I think they're close. I don't know much about maven, lets get the docs right, and then build to the docs.
Gradle build script will now use Mustache template to create pom.xml with needed versions. This should be more reliable and future-proof than previously used `String#replace()`.
Restructured code to better support addition of new formatter steps and formatters. Spotless mojo now accepts java configuration with list of steps. It is also possible to specify global encoding, line endings and override them for java. New `pom.xml` config looks like: ``` <configuration> <encoding>UTF-8</encoding> <lineEndings>UNIX</lineEndings> <java> <encoding>US-ASCII</encoding> <lineEndings>WINDOWS</lineEndings> <steps> <eclipse> <file>${basedir}/build/eclipse-format.xml</file> <version>4.7.1</version> </eclipse> </steps> </java> </configuration> ```
Hi @nedtwigg, Thanks for adding docs! Last commit makes impl very close but not entirely follow them. Couple questions:
Thanks in advance for your answers. I'll proceed with adding more steps (as docs suggest), tests and implementation of the |
I changed the docs and artifactId to be "spotless-maven-plugin". I lean towards leaving the project's folder in the Spotless git repo to be "plugin-maven" - a little confusing, but seems a worthwhile compromise. I also added the <steps> container and changed task names to "mvn spotless:check/apply". I'd be happy to help adding steps, but I don't know how to run tests. I've got an idea for how we could reuse a lot of the plugin-gradle tests, I'll push a skeleton in the next hour or two. |
I added a first cut at an integration test framework for the maven plugin. Feel free to revert and do something completely different if you prefer. <sheepish>I think maybe we need your publish to a local maven repo back</sheepish>. I'm not sure how to make MavenRunner work without it... |
Process process = builder.start(); | ||
// slurp and return the stdout, stderr, and exitValue | ||
Slurper output = new Slurper(process.getInputStream()); | ||
Slurper error = new Slurper(process.getErrorStream()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nedtwigg If we expect MavenRunner
to be instantiated many times, I think we should consider using a shared ExecutorService
, like an Executors.newFixedThreadPool(Runtime.getRuntime().getAvailableProcessors() - 1)
. Otherwise, this whole commit looks pretty reasonable to me. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify, I'm proposing we use an ExecutorService
instead of using Thread
s directly through the Slurper
class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Roger. There will only be 10's of integration tests, so I don't think there's much of a performance hit for starting 2x threads per.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, that sounds reasonable to me.
while ((numRead = input.read(buffer)) != -1) { | ||
output.write(buffer, 0, numRead); | ||
} | ||
result = output.toString(Charset.defaultCharset().name()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nedtwigg Minor nit: I seem to remember that we have access to Guava in some way through a re-packaged version that Diffplug distributes on Maven Central and JCenter.
If we do have access to Guava in this way, then I think we should consider using ByteStreams.copy
to simplify the code inside this try statement. But I'd understand if you don't want to do this for dependency reasons or such. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call, implemented in next commit. For the rest of this review, I'd like to focus on bashing out the high-level design. We can fix performance nits and duplicate code after we have something working :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Roger! :)
It is not needed because maven plugin and tests specify custom local repository via `-Dmaven.repo.local` system property.
I think CI build is now fixed. Build and tests for maven plugin will now use custom local repository instead of What do you think about this solution? |
….spotless group before caching.
…r maven-install-plugin and the wrapper.
Huzzah!!! You fixed it!!! I think that solution is okay, but not ideal. I added a slight tweak - we now cache the |
Nice! I'll continue with include/exclude configs next. Current logic of scanning everything from |
This commit makes it possible to configure includes and excludes for paths/files with Ant style pattern in configuration groups (currently Java and Scala). It also changes the default source roots for scanning. Default policy for scanning used to recursively start from the `basedir` and include files with needed extensions, like "*.java" or "*.scala". This worked fine for simple maven projects but resulted in a lot of extra work for multi-module maven projects. Same files had to be scanned more than once during parent and then child module scanning. Now scanning will use most common source roots for supported languages. They are "scr/main/java/**/*.java", "scr/test/java/**/*.java" for Java and "scr/main/scala/**/*.scala", "scr/main/scala/**/*.sc", "scr/test/scala/**/*.scala", "scr/test/scala/**/*.sc" for Scala. So only these common source roots will be scanned by default. Mentioned patters form default includes. Default excludes are empty, except output directory (usually "target"), temporary files and VCS files are always skipped. It is possible to override includes and add excludes. This can be done like this: ``` <java> <includes> <!-- include all java files in "java" folders under "src" --> <include>src/**/java/**/*.java</include> <!-- include all java files in "java" folders under "other" --> <include>other/java/**/*.java</include> </includes> <excludes> <!-- exclude examples from formatting --> <exclude>src/test/java/**/*Example.java</exclude> </excludes> </java> ``` Similar configuration is possible for Scala. Custom includes completely override default includes. Default excludes of output directory, temporary and VCS files can't be overridden.
Hurrah! Extremely well-done. I'm ready to publish |
Could you please give me a day to play with the plugin and try it on couple projects? :) |
Absolutely! Take as much time as you need. |
I'm a bit uncertain about what to do with multi-module project support. It would definitely be nice for However, there exists a different, hackish way :) It is used by the Checkstyle plugin but is undocumented: source code. Here plugin iterates through the module hierarchy, which kind of violates the maven philosophy that module should only care about itself. This functionality makes it very convenient to specify files only once in the root module. Relevant checkstyle plugin issue and PMD issue contain a bit more details. Note that PMD authors closed the issue with "Won't fix" because this approach is considered to be a hack, though a very convenient one. Despite it's convenience, later solution is a hack. I thought about implementing it but it's probably not a good idea, especially for the first iteration. Maybe it never needs to be implemented because it's not the "true maven way". What do you think about this? I guess it's fine to proceed with 0.x release if we decide to not implement this thing. |
Generic formatting steps, like LicenseHeader can be defined both globally and inside language configs. Example: ``` <configuration> <!-- global license header config --> <licenseHeader>...</licenseHeader> <java> <!-- overridden license header for java --> <licenseHeader>...</licenseHeader> </java> </configuration> ``` Previously global config had no effect. This commit fixes the problem by passing globally configured generic steps down to `FormatterFactory` which creates steps and uses globally configured ones, unless overridden.
It seems that even the authors of the Checkstyle plugin aren't very happy with their hack.
I'm an incredibly novice maven user, but the "official" multi-module solution described in your linked checkstyle and PMD docs doesn't seem terrible. A little verbose, but it seems like everything in maven is pretty verbose ;-) Ready to release when you are! |
License header is a generic step that is applicable to both `Java` and `Scala`. This commit moves adder for it to `FormatterFactory`, which is super class of both `Java` and `Scala`. Code duplication is thus a bit reduced.
Yeah, I definitely agree that separate module just for configs is seriously verbose, even for Maven :) |
It should be available now on mavencentral as |
Hi!
This PR adds a very minimal implementation of the Maven plugin. It's primary goal is to get feedback on the general approach and potential tips for further implementation.
Plugin is able to apply Eclipse formatter step to all java files in the project. It only takes a single parameter - 'eclipseFormatFile' which is a path to the formatter configuration.
Two main blocks:
Build of the plugin
This part is quite ugly. I was not able to find a good way to make Gradle natively create a JAR equivalent to one with
<packaging>maven-plugin</packaging>
in Maven. Such packaging includes some additional plugin configuration xml files.That is why Gradle build script invokes Maven via Exec plugin and makes it assemble the final plugin JAR. Two Spotless dependencies
lib.jar
andlib-extra.jar
are installed in a directory-based Maven repo that is used during the build. They are also shaded into the final plugin JAR. Plugin sources and preparedpom.xml
are copied to a temp directory. Then simplemvn clean install
is executed there by Gradle Exec. Final JAR is copied tobuild/libs
.Plugin code
Lives in the class
SpotlessMojo
that has needed Maven components and 'eclipseFormatFile' config value injected by the Plexus container. It usesArtifactResolver
to create aMavenProvisioner
that is able to download specified Maven dependencies.Mojo creates a single
EclipseFormatterStep, then walks all
*.java` files in the compile sources roots and formats them using this step.Missing/unclear bits:
pom.xml
andbuild.gradle
.pom.xml
can be generated with all dependencies frombuild.gradle
Formatter
Maven Wrapper
to not require pre-installed Maven when buildingExistence of
mvn
command is assumed during the build. Plugin can be added to a Maven project like this:and invoked manually using this command:
"As is" PR is not really ready for merge. Maybe it can be a start?
P.S. thanks a lot for the Gradle plugin! :)