[Enhancement] Figure out how to enable features such as shared consumers in Kafka

[scholzj]

Kafka is now using the kafka-features.sh to enable features such as Shared consumers / Queues. While users should be able to enable it manually using the tool or Admin API, should we have something in the Kafka CR to manage this?

[katheris]

My understanding is that the kafka-features.sh tool is for enabling features that are still in preview right? My gut reaction would be that because of this there is no need to have something in the Kafka CR. However on further investigation I think it could be useful for people wanting to try out preview features.

My understanding is that you run the kafka-features.sh to list or manage the versions of preview features. Once you've set a version this is stored with the cluster metadata. I think adding support to the Kafka CR could be useful for several things:

• Users could check the Kafka CR status for enabled features
• Users could update feature versions via the CR, rather than needing to use another tool
• For dev clusters with ephemeral storage or single node clusters Strimzi could take care of re-enabling the correct features after a restart

[ppatierno]

I like the idea of having something within the Kafka CR to enabling/disabling features as well as status section reporting their state. I think this would need a proposal because of some additional API to do so.
If we think that the Kafka CR should drive as everything in Strimzi, the operator should also revert features' states if the user change them out of band via the kafka-features.sh or admin API and not using the Kafka CR.

[scholzj]

Yes, this would definitely need a proposal.

[im-konge]

Triaged on 26.6.2025: We agreed on implementing this, but it needs a proposal. However, we should also investigate how the kafka-features.sh should be used or what is the future of this tool (which should be described in the proposal as well).

[tinaselenge]

I am happy to work on this.

[Harithasridhar1306]

@scholzj @ppatierno
These are a few things I realized after exploring how Kafka preview features are managed, and here's a proposal for supporting them in Strimzi:

Kafka preview features need to be enabled manually using kafka-features.sh or the Admin API. While this works in stable clusters, it's not persistent in dev environments. There's currently no way to express this declaratively in the Kafka CR.

I see two possible methods to address this:

Method 1: Native Support in the Strimzi Kafka CR

This makes a strong case for enabling FluxCD-style declarative management through the Kafka CR. By putting the preview feature config in the CR,:

• Track all changes in Git
• Automatically reconcile via Strimzi
• Prevent feature drift from manual changes
• Keep dev/staging/prod environments consistent
  Proposal:
• Extend the Kafka CRD schema to include a previewFeatures field.
• On reconciliation, the Operator checks current feature states using the Kafka Admin API.
• The Operator applies feature versions using the Admin API or kafka-features.sh.
• Example:
  spec:
  kafka:
  version: 3.7.0
  previewFeatures:
  - name: kafka.shared.consumer
  enabled: true
  version: 1

Method 2: External Script or Init Container

• Let users run a sidecar or init container that executes kafka-features.sh at startup.
• The script could be mounted via a ConfigMap or maintained externally.
• Doesn’t require changes to Strimzi

[ppatierno]

@Harithasridhar1306 thanks for taking a look at this. Not so many details but my gut feeling is for method 1, by using the Kafka CR. Anyway, making a proposal in Strimzi would be following the process of writing it and opening a PR on the dedicated repo here https://github.com/strimzi/proposals/ with all the needed sections and details.
At the same time I see @tinaselenge self-assigned this issue, but not sure if she is already working on it and if you would be interested to work on it instead.

[tinaselenge]

@Harithasridhar1306 If you are interested in working on this issue, please feel free to take it as I haven't started on it. My initial thought on this was also to use Kafka CR.

[Harithasridhar1306]

@Harithasridhar1306 thanks for taking a look at this. Not so many details but my gut feeling is for method 1, by using the Kafka CR. Anyway, making a proposal in Strimzi would be following the process of writing it and opening a PR on the dedicated repo here https://github.com/strimzi/proposals/ with all the needed sections and details. At the same time I see @tinaselenge self-assigned this issue, but not sure if she is already working on it and if you would be interested to work on it instead.

I'm also leaning toward Method 1 via the Kafka CR for the long-term solution, since it fits better with Strimzi's declarative and GitOps-friendly approach.

yep--i can work on this

[ppatierno]

@Harithasridhar1306 that's cool thanks! Let us know if you need any help on writing a Strimzi proposal. You can start from a template but also take a look at other proposals to understand the structure.
@tinaselenge so I am going to assign this to @Harithasridhar1306, it seems you are ok with that ;-) Thanks!