Couchbase fails due to incorrect URI format

[orpiske]

When trying Couchbase with the given configuration, the connector fails complaining that the URL format is invalid:

2020-10-22 14:08:06,300 [SinkConnector-0] ERROR org.apache.camel.processor.errorhandler.DefaultErrorHandler - Failed delivery for (MessageId: E1DBFDB5E174861-0000000000000000 on ExchangeId: E1DBFDB5E174861-0000000000000000). Exhausted after delivery attempt: 1 caught: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: couchbase://http:my-redacted-host:8091?bucket=testBucket85&password=xxxxxx due to: Invalid URI. Format must be of the form couchbase:http[s]://hostname[:port]/bucket?[options...]

The configuration used for this test was:

connector.class=org.apache.camel.kafkaconnector.couchbase.CamelCouchbaseSinkConnector
camel.sink.path.port=32816
topics=org.apache.camel.kafkaconnector.couchbase.sink.CamelSinkCouchbaseITCase
tasks.max=1
camel.sink.endpoint.bucket=testBucket
camel.sink.path.protocol=http
camel.sink.endpoint.password=password
camel.sink.path.hostname=localhost
camel.sink.endpoint.username=Administrator
name=CamelCouchbaseSinkConnector
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter=org.apache.kafka.connect.storage.StringConverter

[oscerd]

I worked on this here apache/camel@46e23d1#diff-118ec4ad74df487260d52308cd40005080748272e546997dda6d23d91890ecbe

[orpiske]

@oscerd that was quick! Thanks Andrea.

[oscerd]

I'll move camel-master to 3.7 soon

[orpiske]

@oscerd I gave it a try with Couchbase and 3.7. The URL formatting via camel.sink.url is OK now.

However, when using only properties, our code is still formatting the URL incorrectly.

2020-11-16 10:07:10,748 [SinkConnector-0] ERROR org.apache.camel.processor.errorhandler.DefaultErrorHandler - Failed delivery for (MessageId: 2073D9C11D9CFB0-0000000000000000 on ExchangeId: 2073D9C11D9CFB0-0000000000000000). Exhausted after delivery attempt: 1 caught: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: couchbase://http:localhost:32858?bucket=testBucket20&password=xxxxxx due to: Invalid URI. Format must be of the form couchbase:http[s]://hostname[:port]?[options...]

[orpiske]

It does seem that the CKC gets the URL correctly, though:

2020-11-16 10:07:10,621 [SinkConnector-0] INFO org.apache.camel.kafkaconnector.utils.CamelKafkaConnectMain - Creating Camel route from(direct:start)
2020-11-16 10:07:10,621 [SinkConnector-0] INFO org.apache.camel.kafkaconnector.utils.CamelKafkaConnectMain - .to(couchbase:http:localhost:32858?bucket=testBucket20&password=RAW(password)&username=RAW(Administrator))

So, it's seems like it's somewhere else (maybe in the component code itself).

[oscerd]

Probably is in the component. I'll have a look

[oscerd]

The only point possible is at CouchBaseEndpoint level, there a constructor raising that exception message.

[orpiske]

This issue still persists and the connector cannot, yet, work with Couchbase when using the properties.

I took some time to debug the problem and this is the stack trace and where it is messing up with the URL:

normalizeUri:785, AbstractCamelContext (org.apache.camel.impl.engine) -----> this is adding a // on the couchbase URI and rendering it invalid
prepareRecipient:269, SendDynamicProcessor (org.apache.camel.processor) 
process:144, SendDynamicProcessor (org.apache.camel.processor)   
run:395, RedeliveryErrorHandler$SimpleTask (org.apache.camel.processor.errorhandler)
schedule:148, DefaultReactiveExecutor$Worker (org.apache.camel.impl.engine)
scheduleMain:60, DefaultReactiveExecutor (org.apache.camel.impl.engine)
process:147, Pipeline (org.apache.camel.processor)
process:312, CamelInternalProcessor (org.apache.camel.impl.engine)
process:84, DirectProducer (org.apache.camel.component.direct)
process:218, SharedCamelInternalProcessor (org.apache.camel.impl.engine)
process:112, SharedCamelInternalProcessor$1 (org.apache.camel.impl.engine)
process:83, DefaultAsyncProcessorAwaitManager (org.apache.camel.impl.engine)
process:109, SharedCamelInternalProcessor (org.apache.camel.impl.engine)
send:189, DefaultProducerCache (org.apache.camel.support.cache)
send:176, DefaultProducerTemplate (org.apache.camel.impl.engine)
send:148, DefaultProducerTemplate (org.apache.camel.impl.engine)
put:175, CamelSinkTask (org.apache.camel.kafkaconnector)

[oscerd]

Is this done with 3.8.0 @orpiske ?

[orpiske]

Is this done with 3.8.0 @orpiske ?

No, this is still ongoing. I'll continue investigating this one next week.

[orpiske]

This should do the trick: https://github.com/apache/camel/pull/5156/files

Thanks to @tadayosi, who found a similar problem on Netty and led me to the root cause of the problem on Couchbase.