Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.10.0 JMS Sender needs common-collections-3 but only commons-collections4 is bundled #4435

Closed
miadevmk opened this issue Nov 30, 2020 · 16 comments
Labels
bug Something isn't working Fix-Commited Issue fixed and will be available in milestone Internal-Issue-Created An issue has been created in NextGen's internal issue tracker RS-5943 triaged
Milestone

Comments

@miadevmk
Copy link

miadevmk commented Nov 30, 2020

When i try to use the JMS Sender in a destination i get a runtime exeception :

java.lang.NoClassDefFoundError: org/apache/commons/collections/FastHashMap

The class is from commons-collections-3 and not available in commons-collections4. It is used by commons-beanutils (PropertyUtilsBean).

...
import org.apache.commons.beanutils.expression.Resolver;
import org.apache.commons.collections.FastHashMap;
import org.apache.commons.logging.Log;
...

Adding commons-collections-3 to the lib directories doesn't help.

Stacktrace:

ERROR 2020-11-30 13:34:47,139 [Channel Dispatch Thread < HTTP Receiver Thread on mia (5752042d-6079-41e8-8edd-730debaa8505) < qtp270665618-78] com.mirth.connect.donkey.server.channel.Channel: Error processing message in channel mia (5752042d-6079-41e8-8edd-730debaa8505).
com.mirth.connect.donkey.server.channel.ChannelException: 
	at com.mirth.connect.donkey.server.channel.Channel.dispatchRawMessage(Channel.java:1289)
	at com.mirth.connect.donkey.server.channel.SourceConnector.dispatchRawMessage(SourceConnector.java:192)
	at com.mirth.connect.donkey.server.channel.SourceConnector.dispatchRawMessage(SourceConnector.java:170)
	at com.mirth.connect.connectors.http.HttpReceiver$RequestHandler.handle(HttpReceiver.java:381)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:560)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:914)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/commons/collections/FastHashMap
	at com.mirth.connect.donkey.server.channel.Channel.handleDestinationChainThrowable(Channel.java:1872)
	at com.mirth.connect.donkey.server.channel.Channel.process(Channel.java:1807)
	at com.mirth.connect.donkey.server.channel.Channel.dispatchRawMessage(Channel.java:1267)
	... 22 more
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/collections/FastHashMap
	at org.apache.commons.beanutils.PropertyUtilsBean.getPropertyDescriptor(PropertyUtilsBean.java:963)
	at org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:935)
	at org.apache.commons.beanutils.BeanUtils.setProperty(BeanUtils.java:454)
	at com.mirth.connect.util.BeanUtil.setProperties(BeanUtil.java:24)
	at com.mirth.connect.connectors.jms.JmsDispatcher.getJmsConnection(JmsDispatcher.java:286)
	at com.mirth.connect.connectors.jms.JmsDispatcher.send(JmsDispatcher.java:140)
	at com.mirth.connect.donkey.server.channel.DestinationConnector.handleSend(DestinationConnector.java:888)
	at com.mirth.connect.donkey.server.channel.DestinationConnector.process(DestinationConnector.java:518)
	at com.mirth.connect.donkey.server.channel.DestinationChain.doCall(DestinationChain.java:121)
	at com.mirth.connect.donkey.server.channel.DestinationChain.call(DestinationChain.java:63)
	at com.mirth.connect.donkey.server.channel.Channel.process(Channel.java:1805)
	... 23 more
@tvrde
Copy link

tvrde commented Feb 22, 2021

Hello! Did you make any progress with this issue? I seem to have the same problem.

@cturczynskyj cturczynskyj added bug Something isn't working Internal-Issue-Created An issue has been created in NextGen's internal issue tracker RS-5943 triaged labels Feb 22, 2021
@cturczynskyj
Copy link
Collaborator

cturczynskyj commented Feb 22, 2021

Tested locally and found the same. Created internal issue to look into it further. Thanks for reporting!

Edit: Actually, with further testing it seems to be working fine. See below for more info.

@tvrde
Copy link

tvrde commented Feb 22, 2021

What do you think of any work arounds?

@cturczynskyj
Copy link
Collaborator

Upon further testing and learning more about our JMS connectors, I'm unable to encounter the error reported. I wasn't aware that the expectation was for users to add their own JMS libraries, whether it's ActiveMQ, JBoss, or whatever and that's why I was seeing errors when I ran it (I was trying to use ActiveMQ when it didn't exist). We'll add information about that to the User Guide soon!

Here's my testing:

I started by spinning up an ActiveMQ Docker instance for testing. Next, I added the latest ActiveMQ library (activemq-all-5.16.1.jar) to a directory resource (in this case, custom-lib). I started Connect and then I created a JMS Sender channel. I started with the ActiveMQ template and added my credentials and a queue name that corresponded with a queue on the ActiveMQ Docker instance. I then deployed the channel, sent a test message, saw no errors in Connect, and saw the message show up in the queue in the ActiveMQ Docker instance's web UI.

Some things to note:

  1. I'm using the latest ActiveMQ which maybe has different dependencies than older versions and "just works?".
  2. Ensure you set your channel dependencies to include the resource you added your JMS libraries and any JMS library dependencies to.

image

image

@pfelixz
Copy link

pfelixz commented Mar 17, 2021

I'm having the same problem, any clue howto solve it, in my case I'm using OracleAQ (oracle.jms.AQjmsQueueConnectionFactory) ...

[2021-03-17 09:56:35,174] ERROR (com.mirth.connect.server.channel.ErrorTaskHandler :25): com.mirth.connect.donkey.server.StartException: Failed to start channel JMSChannel (6b010c99-4370-4505-a812-8ca133010212).
at com.mirth.connect.donkey.server.channel.Channel.st art(Channel.java:778)
at com.mirth.connect.server.controllers.DonkeyEngineC ontroller$DeployTask.doDeploy(DonkeyEngineControll er.java:1888)
at com.mirth.connect.server.controllers.DonkeyEngineC ontroller$DeployTask.execute(DonkeyEngineControlle r.java:1777)
at com.mirth.connect.server.channel.ChannelTask.call( ChannelTask.java:67)
at com.mirth.connect.server.channel.ChannelTask.call( ChannelTask.java:16)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run (Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/collections/FastHashMap
at org.apache.commons.beanutils.PropertyUtilsBean.get PropertyDescriptor(PropertyUtilsBean.java:963)
at org.apache.commons.beanutils.BeanUtilsBean.setProp erty(BeanUtilsBean.java:935)
at org.apache.commons.beanutils.BeanUtils.setProperty (BeanUtils.java:454)
at com.mirth.connect.util.BeanUtil.setProperties(Bean Util.java:24)
at com.mirth.connect.connectors.jms.JmsClient.start(J msClient.java:134)
at com.mirth.connect.connectors.jms.JmsReceiver.onSta rt(JmsReceiver.java:62)
at com.mirth.connect.donkey.server.channel.SourceConn ector.start(SourceConnector.java:104)
at com.mirth.connect.donkey.server.channel.Channel.st art(Channel.java:754)
... 8 more

@tonygermano
Copy link
Collaborator

Can you drop the commons-collections-3 jar file in the same resource dir as your Connection Factory Class? It doesn't use the same package names as collections4, so there shouldn't be any collisions.

@pfelixz
Copy link

pfelixz commented Mar 20, 2021

Hi @tonygermano I've tried putting "aqapi.jar commons-collections-3.2.2.jar jta.jar ojdbc8.jar" on "/opt/connect/custom-lib" and also on "/opt/connect/server-lib/oracle/" after docker restart I allways get the same error, your help is greatly appreciated, please advise how can I overcome this error, thanks in advance.

image

@pfelixz
Copy link

pfelixz commented Mar 22, 2021

I was able to overcome the problem downgrading to "nextgenhealthcare/connect:3.8.1".

Now I'm unable to set the connection, without JNDI, I've tried everything on "Connection Properties": "url", "jdbc_connect_string", "jdbc_connect_str", "jdbc_url", ...

[2021-03-22 07:30:29,383] ERROR (com.mirth.connect.server.channel.ErrorTaskHandler:25): com.mirth.connect.donkey.server.StartException: Failed to start channel Oracle AQ JMS (0e8e87c2-f13c-4b4d-9875-89311c5fd7c4).

at com.mirth.connect.donkey.server.channel.Channel.start(Channel.java:759)

at com.mirth.connect.server.controllers.DonkeyEngineController$DeployTask.doDeploy(DonkeyEngineController.java:1882)

at com.mirth.connect.server.controllers.DonkeyEngineController$DeployTask.execute(DonkeyEngineController.java:1775)

at com.mirth.connect.server.channel.ChannelTask.call(ChannelTask.java:67)

at com.mirth.connect.server.channel.ChannelTask.call(ChannelTask.java:16)

at java.base/java.util.concurrent.FutureTask.run(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)

Caused by: com.mirth.connect.donkey.server.ConnectorTaskException: Failed to establish a JMS connection

at com.mirth.connect.connectors.jms.JmsClient.start(JmsClient.java:159)

at com.mirth.connect.connectors.jms.JmsReceiver.onStart(JmsReceiver.java:62)

at com.mirth.connect.donkey.server.channel.SourceConnector.start(SourceConnector.java:104)

at com.mirth.connect.donkey.server.channel.Channel.start(Channel.java:735)

... 8 more

Caused by: oracle.jms.AQjmsException: IO Error: The Network Adapter could not establish the connection

at oracle.jms.AQjmsDBConnMgr.checkForSecurityException(AQjmsDBConnMgr.java:939)

at oracle.jms.AQjmsDBConnMgr.getConnection(AQjmsDBConnMgr.java:617)

at oracle.jms.AQjmsDBConnMgr.<init>(AQjmsDBConnMgr.java:184)

at oracle.jms.AQjmsConnection.<init>(AQjmsConnection.java:175)

at oracle.jms.AQjmsConnectionFactory.createConnection(AQjmsConnectionFactory.java:611)

at com.mirth.connect.connectors.jms.JmsClient.start(JmsClient.java:138)

... 11 more

Caused by: java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection

at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:743)

at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:666)

at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)

at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:566)

at java.sql/java.sql.DriverManager.getConnection(Unknown Source)

at java.sql/java.sql.DriverManager.getConnection(Unknown Source)

at oracle.jms.AQjmsDBConnMgr.getConnection(AQjmsDBConnMgr.java:596)

... 15 more

Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection

at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:470)

at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:506)

at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:595)

at oracle.net.ns.NSProtocol.connect(NSProtocol.java:230)

at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1452)

at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:496)

... 21 more

Caused by: java.net.UnknownHostException: null: Temporary failure in name resolution

at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)

at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(Unknown Source)

at java.base/java.net.InetAddress.getAddressesFromNameService(Unknown Source)

at java.base/java.net.InetAddress$NameServiceAddresses.get(Unknown Source)

at java.base/java.net.InetAddress.getAllByName0(Unknown Source)

at java.base/java.net.InetAddress.getAllByName(Unknown Source)

at java.base/java.net.InetAddress.getAllByName(Unknown Source)

at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:120)

at oracle.net.nt.ConnOption.connect(ConnOption.java:159)

at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:428)

... 26 more

@pfelixz
Copy link

pfelixz commented Mar 23, 2021

I managed to make it work ...

  • 1st downgraded to "nextgenhealthcare/connect:3.8.1" and the error with mixed dependencies between commons-beanutils and "commons-collections4-4.4" and "commons-collections-3.2.2" disapeared.
  • 2nd I found how to set AQjmsConnectionFactory JDBC URL Connection Properties after downloading "nextgenhealthcare\connect" github code and "commons-beanutils.src", the property is called "jdbcURL" with the content "jdbc:oracle:thin:@//servername:port/instance"

@cturczynskyj
Copy link
Collaborator

@pfelixz this all should work with the correct libraries and their dependencies added as a channel library resource as I showed in the screenshots above. Did you try it this way?

@ChristopherSchultz
Copy link
Contributor

Adding to custom-lib won't work, unless you set includecustomlib=true in conf/mirth.properties which is highly discouraged (and for good reason).

@ChristopherSchultz
Copy link
Contributor

I have run across the same problem, but it's got nothing to do with JMS. In my case, I'm using commons-digester (provided via custom-lib in my case), but it's using JAXP (supplied by the JVM) which needs commons-beanutils (provided by Mirth in server-lib), so it's still good.

When commons-beanutils tries to load its own PropertyUtilsBean class, it fails due to a missing dependency on commons-collections(3).

@ChristopherSchultz
Copy link
Contributor

You can reproduce this quite easily by putting this bit of javascript code anywhere in Mirth and causing it to run:

Packages.org.apache.commons.beanutils.PropertyUtilsBean.getInstance();

@tonygermano
Copy link
Collaborator

@cturczynskyj,

@ChristopherSchultz and I worked this out in Slack. Because commons-beanutils was added to the mirth-launcher.jar manifest (and therefore the system classloader) in 3.10 in commit 506e798, The BeanUtils class will never be able to find FastHashMap from commons-collections in a resource dir nor by any mirth provided mechanism for loading classes (even server-launcher-lib is too late.)

The fix is probably to add commons-collections-3.2.2 back and add it to the manifest as well, since it is listed as a dependency of beanutils. Luckily, it has no dependencies, so the madness should end there.

A workaround could be to place the file somewhere, and append it to the launcher classpath in the appropriate mcservice.vmoptions or mcserver.vmoptions file:

-classpath/a path/to/commons-collections-3.2.2.jar

@ChristopherSchultz
Copy link
Contributor

ChristopherSchultz commented Apr 22, 2021

A workaround could be to place the file somewhere, and append it to the launcher classpath in the appropriate mcservice.vmoptions or mcserver.vmoptions file:

-classpath/a path/to/commons-collections-3.2.2.jar

This workaround works. I did it by placing commons-collections-3.2.2.jar (which has no known security issues) into server-lib/commons/ and added the following line to my mcservice.vmoptions file:

-classpath/a server-lib/commons/commons-collections-3.2.2.jar

This allows commons-beanutils to be able to load classes from commons-collections which it turns out is fairly important. There are many classes in beanutils which end up referencing commons-collections.

@ChristopherSchultz
Copy link
Contributor

Ping. This seems like an insanely easy thing to fix: just put the commons-collections v3 library back into the package.

tonygermano added a commit to tonygermano/connect that referenced this issue Nov 5, 2021
This version of the library was removed in mirth 3.10.0 when
collections4 was included. However, commons-beanutils has a dependency
on the 3.x version. At that time, commons-beanutils was also added
to the root class path, so its dependencies must also be added. See
nextgenhealthcare#4435 for details.
tonygermano added a commit to tonygermano/connect that referenced this issue Nov 5, 2021
This version of the library was removed in mirth 3.10.0 when
collections4 was included. However, commons-beanutils has a dependency
on the 3.x version. At that time, commons-beanutils was also added
to the root class path, so its dependencies must also be added. See
nextgenhealthcare#4435 for details.
@pladesma pladesma added the Fix-Commited Issue fixed and will be available in milestone label Mar 2, 2022
@pladesma pladesma added this to the 4.0.0 milestone Mar 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Fix-Commited Issue fixed and will be available in milestone Internal-Issue-Created An issue has been created in NextGen's internal issue tracker RS-5943 triaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants