Integration with Smallrye Reactive Messaging SQS

[adampoplawski]

Around April SQS support was added to Smallrye Reactive Messaging. I asked on Zulip if Quarkus integration is planned and were transferred here. Is this topic considered?
For potential reference https://quarkusio.zulipchat.com/#narrow/stream/187030-users/topic/smallrye.20reactive.20messaging.20SQS.20support.

Thx for support

[scrocquesel]

@adampoplawski smallrye is supported directly by quarkus core. I would open an issue there. Smallrye use only the netty transport implementation so that the whole purpose of this extension is a bit over complicated. I guess like for Apache Camel aws2 SQS which only use Apache HTTP client transport, there will be a dedicated and separated extension for smallrye.

The only things that should be needed for quarkus to support aws sdk with netty-io-client is

@BuildStep
    void runtimeInitialize(BuildProducer<RuntimeInitializedClassBuildItem> producer) {
        producer.produce(
                new RuntimeInitializedClassBuildItem("software.amazon.awssdk.core.retry.backoff.FullJitterBackoffStrategy"));
        producer.produce(
                new RuntimeInitializedClassBuildItem("software.amazon.awssdk.utils.cache.CachedSupplier"));
    }

a copy of https://github.com/quarkiverse/quarkus-amazon-services/blob/main/common/runtime/src/main/java/io/quarkus/amazon/common/runtime/CrtSubstitutions.java

and a dependency on io.quarkus:quarkus-netty.

Last, like with the Apache Camel connector, you may be able to reuse an instance from this extension as the smallrye connector allows to inject an SqsClient instance.

@ozangunalp what do you think ?

[ozangunalp]

I'll respond here first:

The connector was contributed upstream to the Smallrye Reactive Messaging, but it was designed to work with a provided SqsAsyncClient instance, for example by the Quarkus AWS extension.

The integration is very straightforward, as @scrocquesel mentioned. There is a quickstart here: https://github.com/quarkusio/quarkus-quickstarts/tree/main/amazon-sqs-connector-quickstart

With the following dependencies :

    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-messaging</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkiverse.amazonservices</groupId>
      <artifactId>quarkus-amazon-sqs</artifactId>
    </dependency>
    <dependency>
      <groupId>io.smallrye.reactive</groupId>
      <artifactId>smallrye-reactive-messaging-aws-sqs</artifactId>
    </dependency>
    <dependency>
      <groupId>software.amazon.awssdk</groupId>
      <artifactId>url-connection-client</artifactId>
    </dependency>
    <dependency>
      <groupId>software.amazon.awssdk</groupId>
      <artifactId>netty-nio-client</artifactId>
    </dependency>

We could contribute to this repository, an extension putting all these dependencies together.
@adampoplawski I can help if you are willing to contribute.

[scrocquesel]

If we don't need a specific quarkus extension for the connector itself, it is all good.

Is it documented somewhere that not all smallrye connectors require a quarkus extension.

[ozangunalp]

Is it documented somewhere that not all smallrye connectors require a quarkus extension.

Maybe I'll add a line in https://quarkus.io/guides/messaging about that.

[adampoplawski]

@ozangunalp I will start with https://github.com/quarkusio/quarkus-quickstarts/tree/main/amazon-sqs-connector-quickstart and will ping you if I will get stuck

[ozangunalp]

@adampoplawski thank you for looking into this. Ping me if you have any questions. You can check the quickstart for required dependencies.

[adampoplawski]

@ozangunalp I have created simple demo for me https://github.com/adampoplawski/quarkus-sqs
Can you refer me to some docs explaining what I need to do :)

[adampoplawski]

Hello
@ozangunalp @scrocquesel
Can I get some example/tutorial/docs etc how can I contribute?

[scrocquesel]

The quick start is enough for me now. We may need to wrap with a dedicated extension when the smallrye connector supports multiple name client.
The next version of the amazon sqs extension will support that with @AmazonClient("name") annotation. I guess smallrye would use their @identifier and we will have to make the glue.

[adampoplawski]

@scrocquesel thx. So I will leave it with you and will contribute to other issue :)

[ozangunalp]

here is a draft for a simple wrapper extension : #1431

Feel free to change anything needed.

[scrocquesel]

Thanks, @adampoplawski could you try add the quick start sample as an integration test in the integration test module ?

[adampoplawski]

@scrocquesel yes, not today but I will look next week.

[adampoplawski]

@scrocquesel @ozangunalp
Changes are on branch ozangunalp:sqs_messaging_connector

Can we somehow get quarkiverse-aws from this brach to add integration test locally?

[scrocquesel]

Create another PR based on this branch

[scrocquesel]

#1438 is off from a branch of this repo. Feel free to contribute the branch with PRs.
I guess @Tincoquesse already come up with IT tests. Don't do it twice.

[adampoplawski]

@scrocquesel
Yes me and @Tincoquesse are working together in same SoftwareHouse so we are aligned :) I see there is some PR hanging #1438. I will be checking and we will rebase after marge and provide next PR

[adampoplawski]

@scrocquesel I will take care of integration test this week :)

[adampoplawski]

@scrocquesel how can I push to this repo, to separate branch? Maybe there is instruction for it?
remote: Permission to quarkiverse/quarkus-amazon-services.git denied to adampoplawski.
fatal: unable to access 'https://github.com/quarkiverse/quarkus-amazon-services.git/': The requested URL returned error: 403

[scrocquesel]

Push to your fork and open a pull request

[scrocquesel]

The extension has been released. I opened another issue for the integration test #1466

Thank you all for this contribution.