X-Pack Transport Client POM does not include all dependencies

[tvernum]

In 6.2 and earlier, the x-pack-transport POM had a dependency on x-pack-api which pulled in 3rd party dependencies such as the UnboundId LDAP SDK and Bouncy Castle (for reading TLS keys)

The (abbreviated) dependency tree was:

+- org.elasticsearch.client:x-pack-transport:jar:6.2.4:compile
|  +- org.elasticsearch.plugin:x-pack-api:jar:6.2.4:compile
|  |  +- com.unboundid:unboundid-ldapsdk:jar:3.2.0:compile
|  |  +- org.bouncycastle:bcprov-jdk15on:jar:1.58:compile
|  |  +- org.bouncycastle:bcpkix-jdk15on:jar:1.58:compile

Since 6.3 the x-pack-api POM no longer exists, and x-pack-transport simply depends on x-pack-core which does not include any 3rd party dependencies.
As far as I can tell from grepping, none of our generated POM files include a dependency for the ldapsdk (nor, I assume, bouncy castle).

This means that the published dependency in our docs is insufficient

 <dependencies>
      <!-- add the x-pack jar as a dependency -->
      <dependency>
         <groupId>org.elasticsearch.client</groupId>
         <artifactId>x-pack-transport</artifactId>
              <version>{version}</version>
      </dependency>
      ...
   </dependencies>

There is a qa test for transport client with ML, but because it uses gradle with local project dependencies, it pulls in the transitive dependency list from x-pack-core including the LDAP sdk, etc

+--- project :x-pack:plugin:core
|    [ snip ]
|    +--- com.unboundid:unboundid-ldapsdk:3.2.0
|    +--- org.bouncycastle:bcprov-jdk15on:1.58
|    +--- org.bouncycastle:bcpkix-jdk15on:1.58

[elasticmachine]

Pinging @elastic/es-core-infra

[rjernst]

I've finally figured out what's happening here, and I think the fix is straightforward, but I want to double check with others on my assumptions.

Previously we built a jar named "x-pack-api". There were a couple reasons for this, both to allow for a reduced set of classes in the future that xpack plugins would have access to, as well as to differentiate the pom file name for this jar from that which PluginBuildPlugin builds for the zip. It is the latter reason here that has caused the breakage, because x-pack-core is a "plugin", even though it is not really a plugin (it is a module, we just use the normal plugin build and the distribution build unzips the plugin zip).

I see 3 possible solutions:

1. Remove pom generation from PluginBuildPlugin altogether (for the zip file at least). This is left over from replicating what we did in 1.x/2.x for plugins, in that the zip files were published to maven central. However, the plugin installer now downloads these from the elastic download service, and the pom is unused as far as I can tell. Note there is the "client jar" case, but we already name this differently, so the pom for that would not be affected.

2. Build modules differently so they do not generate a pom. This would be a smaller step towards 1, but probably more work as it would mean carefully changing the pom generation configuration to not be added when the plugin is a module (we already have some hardcoded logic to determine that condition).

3. Add back the "x-pack-api" jar. This was pretty hacky to do, and just added bloat and hard to read configuration in the gradle build file, so IMO it is a last resort.

[nik9000]

I like the idea of removing the pom from the plugins. As much as I like having the zips available in maven central that seems like the simplest change to be honest.