Camel HDFS: does not work due to dependencies conflicts

[orpiske]

It seems that the HDFS Connector cannot be loaded due to conflicts between the dependencies used by the component and the ones available on the connect runtime.

Without manipulating the classpath, loading the dependencies fail because a dependency carried by either one of them does not seem to contain the getProperties on the Application class (which comes from javax.ws.rs:javax.ws.rs-api). Manipulating the classpath to include this library and/or exclude specific dependencies cause the connector to fail to load due to other classpath related issues (ie.: ClassNotFoundException).

I'll provide more logs later for this one.

[valdar]

As @oscerd suggested I think we must handle this at camel component level and align dependencies there if it makes sense.

[orpiske]

As @oscerd suggested I think we must handle this at camel component level and align dependencies there if it makes sense.

Thanks for the feedback. Indeed, it sounds like it would be better to do it in Camel.

Is this something we should track on Camel's Jira? I can open and issue there and reference this one.

[oscerd]

I guess we need a JIRA for this, because it won't be easy, especially in Karaf.

[orpiske]

I guess we need a JIRA for this, because it won't be easy, especially in Karaf.

I'll open one there, thanks!

[orpiske]

Done: https://issues.apache.org/jira/browse/CAMEL-15169

[orpiske]

For the integration tests, I managed to work-around this problem with a couple of exclusion hacks: https://github.com/orpiske/camel-kafka-connector/blob/hdfs-test/tests/itests-hdfs/pom.xml#L45

Nonetheless, the package might still have the problem and only work adequately once we get 3.4.0 with @oscerd patches.

[oscerd]

Have you tried with 3.4.0-Snapshot?

[orpiske]

Have you tried with 3.4.0-Snapshot?

Not yet, but I will do it soon and report back.

[orpiske]

Have you tried with 3.4.0-Snapshot?

So, I tried with and without the excludes and both failed. In both cases, the problem seems to be caused by:

Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.util.component.AttributeContainerMap.

[oscerd]

Updated to 3.4.0 and merged, I see that error, so we need to have a look

[orpiske]

Updated to 3.4.0 and merged, I see that error, so we need to have a look

Should I go ahead and disable them while we investigate?

[oscerd]

Maybe disable it for the moment