#8732 Enhance KafkaBridge resource with consumer inactivity timeout and HTTP consumer/producer parts enablement #9820

steffen-karlsson · 2024-03-12T18:57:32Z

Type of change

Enhancement / new feature: Enhance KafkaBridge resource with consumer inactivity timeout and HTTP consumer/producer parts enablement #8732

Description

Adding CRD Kubernetes API support for following KafkaBridge properties:
http.consumer.timeoutSeconds - for deleting inactive consumers after a timeout (disabled by default)
http.consumer.enabled - to enable/disable the HTTP consumer part (enabled by default)
http.producer.enabled - to enable/disable the HTTP producer part (enabled by default)

Checklist

Please go through this checklist and make sure all applicable tasks have been done

Write tests
Make sure all tests pass
Update documentation
Check RBAC rights for Kubernetes / OpenShift roles
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md
Supply screenshots for visual changes, such as Grafana dashboards

steffen-karlsson · 2024-03-12T19:09:36Z

Missing last part of documentation, will do when discussion in issue is settled: #8732

scholzj

I left some comments. I the CI failed because of spotbugs. It would also fail because the CRD files are not copied to all the places. You can use make crd_install for that.

api/src/main/java/io/strimzi/api/kafka/model/bridge/KafkaBridgeHttpConfig.java

api/src/main/java/io/strimzi/api/kafka/model/common/ClientTls.java

…Connect and KafkaBridge Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

…oducer` Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

…ridge

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

…r and dereferenced in io.strimzi.operator.cluster.model.KafkaBridgeCluster.addCorsEnvVars(List) [io.strimzi.operator.cluster.model.KafkaBridgeCluster] At KafkaBridgeCluster.java:[line 453] UWF_FIELD_NOT_INITIALIZED_IN_CONSTRUCTOR Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

…not true by default Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

api/src/main/java/io/strimzi/api/kafka/model/bridge/KafkaBridgeConsumerSpec.java

api/src/main/java/io/strimzi/api/kafka/model/bridge/KafkaBridgeProducerSpec.java

api/src/main/java/io/strimzi/api/kafka/model/bridge/KafkaBridgeSpec.java

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/KafkaBridgeClusterTest.java

CHANGELOG.md

api/src/main/java/io/strimzi/api/kafka/model/bridge/KafkaBridgeConsumerSpec.java

api/src/main/java/io/strimzi/api/kafka/model/bridge/KafkaBridgeProducerSpec.java

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaBridgeCluster.java

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/KafkaBridgeClusterTest.java

documentation/api/io.strimzi.api.kafka.model.bridge.KafkaBridgeConsumerSpec.adoc

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

…eConsumerSpec.adoc Co-authored-by: Jakub Scholz <www@scholzj.com> Signed-off-by: Steffen Wirenfeldt Karlsson <steffen.karlsson@maersk.com>

Signed-off-by: Steffen Wirenfeldt Karlsson <steffen.karlsson@maersk.com>

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/KafkaBridgeClusterTest.java

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

scholzj

LGTM. Thanks.

ppatierno

LGTM. Thanks!

scholzj · 2024-05-15T15:29:06Z

/azp run regression

azure-pipelines · 2024-05-15T15:29:17Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-05-15T20:20:37Z

Thanks for the PR.

steffen-karlsson force-pushed the 8732-enhance-kafkabridge branch from e36cb5c to b71de01 Compare March 12, 2024 19:01

steffen-karlsson mentioned this pull request Mar 12, 2024

#8732 Enhance KafkaBridge resource with consumer inactivity timeout and HTTP consumer/producer parts enablement strimzi/strimzi-kafka-bridge#882

Merged

steffen-karlsson force-pushed the 8732-enhance-kafkabridge branch 2 times, most recently from 129ac78 to b5a884f Compare March 12, 2024 19:44

scholzj added this to the 0.41.0 milestone Mar 12, 2024

scholzj requested a review from ppatierno March 12, 2024 22:41

scholzj reviewed Mar 12, 2024

View reviewed changes

steffen-karlsson force-pushed the 8732-enhance-kafkabridge branch 3 times, most recently from 92bcc15 to 2233def Compare March 13, 2024 17:36

Steffen Karlsson added 11 commits March 13, 2024 18:37

Improved reusability by moving into separate function to use in Kafka…

0fb606b

…Connect and KafkaBridge Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Adding objects to include into http for enabling consumer and `pr…

211d1f1

…oducer` Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Adding timeoutSeconds , consumer and producer to http config

bfb4299

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Rewriting env generation to reduce cognitive complexity

695b5da

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Adding tests for timeoutSeconds , consumer and producer

59678c4

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Updating HttpConfig with timeoutSeconds

144151a

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Fixing bugs by pipeline

ae65e04

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Removing host again as per suggestion

2da7d13

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Adding bridge CRD

452d181

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Moving to cluster operator to ModelUtils

6300db7

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Unused imports

8ee9489

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

steffen-karlsson force-pushed the 8732-enhance-kafkabridge branch from 188fd0d to 8ee9489 Compare March 13, 2024 17:37

steffen-karlsson mentioned this pull request Mar 14, 2024

Raising proposal based on PR discussion for: Enhance KafkaBridge resurce with consumer inactivity timeout and HTTP consumer/producer parts enablement strimzi/proposals#111

Merged

Steffen Karlsson added 2 commits May 7, 2024 08:46

Merge remote-tracking branch 'upstream/main' into 8732-enhance-kafkab…

54bd399

…ridge

Move timeoutSeconds to consumer config and rename to config

40bcdd8

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

steffen-karlsson marked this pull request as ready for review May 7, 2024 07:10

Steffen Karlsson added 3 commits May 7, 2024 09:20

Updating crd and docs to new names

a55454d

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Revert logic back to main, too much commit

2bb1287

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

scholzj modified the milestones: 0.41.0, 0.42.0 May 7, 2024

Steffen Karlsson added 2 commits May 7, 2024 19:13

Moving and to and instead of

d9f0402

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Updating documentation

d726cac

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

steffen-karlsson requested a review from scholzj May 7, 2024 17:17

Steffen Karlsson added 3 commits May 7, 2024 20:02

Update derived resources

07be48d

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Fixing IT test for KafkaBridge

eba352a

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Consumer and Producer should always be initated otherwise enabled is …

8d07b18

…not true by default Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

ppatierno reviewed May 8, 2024

View reviewed changes

scholzj reviewed May 9, 2024

View reviewed changes

Steffen Karlsson and others added 8 commits May 10, 2024 08:49

Extending documentation as per request

26928f7

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Review comment

9759bfc

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Moved to Release 0.42.0

f148dda

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Adding JsonInclude.Include.NON_DEFAULT

72b3083

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Update documentation/api/io.strimzi.api.kafka.model.bridge.KafkaBridg…

38b1faa

…eConsumerSpec.adoc Co-authored-by: Jakub Scholz <www@scholzj.com> Signed-off-by: Steffen Wirenfeldt Karlsson <steffen.karlsson@maersk.com>

Merge branch 'main' into 8732-enhance-kafkabridge

c21af96

Signed-off-by: Steffen Wirenfeldt Karlsson <steffen.karlsson@maersk.com>

Updating CRDs

9acb682

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Remove default initialisation

2aef28e

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

steffen-karlsson commented May 15, 2024

View reviewed changes

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/KafkaBridgeClusterTest.java Show resolved Hide resolved

Steffen Karlsson added 2 commits May 15, 2024 11:25

Update derived resources

3663890

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

Removed because of review comments

9e7e000

Signed-off-by: Steffen Karlsson <steffen.karlsson@maersk.com>

steffen-karlsson requested review from ppatierno and scholzj May 15, 2024 12:05

scholzj approved these changes May 15, 2024

View reviewed changes

ppatierno approved these changes May 15, 2024

View reviewed changes

scholzj linked an issue May 15, 2024 that may be closed by this pull request

Enhance KafkaBridge resource with consumer inactivity timeout and HTTP consumer/producer parts enablement #8732

Closed

scholzj merged commit 2bf2be1 into strimzi:main May 15, 2024

#8732 Enhance KafkaBridge resource with consumer inactivity timeout and HTTP consumer/producer parts enablement #9820

#8732 Enhance KafkaBridge resource with consumer inactivity timeout and HTTP consumer/producer parts enablement #9820

Uh oh!

Conversation

steffen-karlsson commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of change

Description

Checklist

Uh oh!

steffen-karlsson commented Mar 12, 2024

Uh oh!

scholzj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

scholzj left a comment

Choose a reason for hiding this comment

Uh oh!

ppatierno left a comment

Choose a reason for hiding this comment

Uh oh!

scholzj commented May 15, 2024

Uh oh!

azure-pipelines bot commented May 15, 2024

Uh oh!

scholzj commented May 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

steffen-karlsson commented Mar 12, 2024 •

edited

Loading