Enable Kafka Streams binder for Graal native compilation

[sobychacko]

@sobychacko commented on Tue Apr 27 2021

---

@sabbyanandan commented on Tue May 04 2021

Verification on the KTable is still pending.

[onobc]

Most likely this is blocked by the JVM reachability for the state store JNI as described in spring-attic/spring-native#680

[sdeleuze]

My tests on https://github.com/spring-projects/spring-aot-smoke-tests/tree/main/cloud-stream-kafka (confirmed by the CI) seems to indicate it is now working, should this issue be updated accordingly?

[onobc]

@sdeleuze that AOT smoke test only covers the Kafka binder, not the KafkaStreams binder that this is referring to (naming... they sound very much like "the thing").

Once this issue is addressed (or it is no longer an issue) then another AOT smoke test should be added to cover it as well.

[pdebuitlear]

Support for Kafka Streams doesn't work out of the box, some additional hints are needed. See this project for a working example. There are also hints in that project for the Confluent schema registry and the Confluent serializers.

[sobychacko]

@pdebuitlear We haven't addressed native changes for the Kafka Streams binder yet, but we will do it as part of one of the upcoming 4.x releases. Thanks for sharing the project, as that will be a good reference.

[artembilan]

Looking to that project, @pdebuitlear , I would say that I don't see anything Spring Cloud Stream related or even Spring at all.
It really would be better to have Native images support directly from those libs, so any target implementation would benefit from a common place, not just this Spring Cloud Stream Kafka Streams Binder.
So, I would suggest to consider raising improvement tickets in respective Kafka Streams and Confluent Platform projects respectively.

[pdebuitlear]

Looking to that project, @pdebuitlear , I would say that I don't see anything Spring Cloud Stream related or even Spring at all. It really would be better to have Native images support directly from those libs, so any target implementation would benefit from a common place, not just this Spring Cloud Stream Kafka Streams Binder. So, I would suggest to consider raising improvement tickets in respective Kafka Streams and Confluent Platform projects respectively.

Not even these?
https://github.com/pdebuitlear/kstreams-demo/blob/main/src/main/java/com/example/demo/config/hints/KafkaStreamsHints.java

[artembilan]

Of course no, because those types and resources are exactly from Kafka Streams project. And I believe that we have to expose them even if we just try to build plain Kafka Streams application, even without Spring for Apache Kafka .

[garyrussell]

Spring for Apache Kafka added a few hints for basic Kafka, and Kafka Streams, support, but this was for expediency, to prove out the core project can support native images; it is impractical for Spring projects to maintain ongoing, extensive, hints for all dependencies.

There are some sample smoke tests for the basic hints. https://github.com/spring-projects/spring-aot-smoke-tests/tree/main/integration