Add dependency of correct version kafka-streams-test-utils

[jukkakarvanen]

TopologyTestDriver based Kafka Streams test require dependency package org.apache-kafka:kafka-streams-test-utils. I have not seen this referenced in spring-kafka or spring-cloud packages. So now you need to kafka.version of this package in sync with spring-kafka-test kafka version.

It would be a good idea to add kafka-streams-test-utils as dependence to spring-kafka-test or add separate spring-kafka-streams-test package?

This is relating the discussion in spring-cloud/spring-cloud-stream-samples#144 and @garyrussell request to create issue here.

[artembilan]

I think the story is really about Spring Boot dependency management.
Alongside with all other Apache Kafka dependencies: https://github.com/spring-projects/spring-boot/blob/master/spring-boot-project/spring-boot-dependencies/pom.xml#L1754

Add this one for the kafka-streams-test-utils and that's all.

There problem is that that there is nothing in spring-kafka-test about Kafka Streams, so such an extra dependency what bring anything valuable.

We may reconsider though if there are some features to to the spring-kafka-test for Kafka Streams.
Nevertheless a dependency for this artifact must be declared in Spring Boot for the proper version management. Otherwise it may happen that Spring Kafka manages one Apache Kafka version, but Spring Boot another. So, many Kafka dependencies (see list mentioned in a link above) are going to be remapped to Boot version, but this one for the
kafka-streams-test-utils is not and you are going to be forced to manage this dependency yourself again.

Does it make sense?

I might just miss a big picture here, so I'm open for the further discussion.
Thanks

[jukkakarvanen]

@garyrussell:

Yes, we can add a streams test module to the spring-kafka project; but it will also need a change to initializr to pull it in to test scope when KafkaStreams is selected (like is currently done for s-k-test when kafka is selected).

I expect Gary was thinking handling the dependency here in spring kafka and that way keep the version in sync with other spring-kafka kafka depenedencies. If it would be added only to Spring Boot dependency management there is the risk you mentioned about different Kafka versions in Spring Boot and Spring Kafka.

[garyrussell]

As I said in the other thread; we will also need a change to initializr to pull in the test jar in test scope anyway.

I am not sure, yet, whether we can add value, but it wouldn't hurt to add a sub-project anyway; even if it just pulls in the dependency for now.

Or, maybe we can just add the streams deps to the existing test jar, which is only in test scope - maybe that would be simpler.

[jukkakarvanen]

Yes, I agree. Quoting myself from other thread.

Adding spring-kafka-streams-test to drag in kafka-streams-test-utils would be the ideal solution, but when spring-kafka-test is mainly in test scope, so adding dependency to it would not impact to runtime dependencies needed if kafka streams not used.

[artembilan]

There is no yet need in the spring-kafka-streams-test, but if it would I think the spring-kafka-test is fully enough a an umbrella artifact to keep all the Spring testing utilities for Apache Kafka.
So, thinking about this one more time, I agree to add kafka-streams-test-utils dependency into the spring-kafka-test: in the end it is going to be used in the test scope anyway.

Nevertheless we should consider to add it into the Spring Boot dependency management for consistency. In the end exactly Boot dictates us what runtime dependencies are going to be used in our project by default.

[garyrussell]

This means we won't need an initializr change.

I issued a PR to add it to boot's dependency management.

[jukkakarvanen]

This is good. Thanks.