Added Proposal Kafka Preview Feature Support in Strimzi

[Harithasridhar1306]

This proposal introduces native and optional support in Strimzi for managing Kafka preview features in a declarative and persistent way.

[ppatierno]

@Harithasridhar1306 I didn't take a look at the content yet but if you could adhere to the markdown we have in the template here https://raw.githubusercontent.com/strimzi/proposals/refs/heads/main/000-template.md at least in terms of headers and titles (by using one or more # depending on the level).
Also always split each paragraph with one sentence per line, because it makes the review much easier in order to provide specific feedback. After these changes I will look deeply at the content itself. Thanks!

[scholzj]

I think you should start by detailed explanation of how Kafka is using / plans to use the features. That was not clear last time, and having a proper understanding is key to getting the API right.

Also, the point of the proposal is not to give choices but to propose the right solution. So you should pick one solution, that should be what the proposal is focusing on and any other alternatives should be in the rejected alternatives. If at the end it turns out the proposed solution is not the right one, it would need to be rewritten. But it is not vote between different variants. In this case, I think the declarative variant is the one you hould develop. The rest should be in the rejected alternatives.

[scholzj]

I left some comments and questions. In general, fixing the formatting as I siggested in the comments would improve the readability so much ...

[scholzj]

Format this as code with ```yaml

[scholzj]

So, what happens when the features GA? How will we validate them? How will the error handling look like?

[Harithasridhar1306]

Validation:

1. During reconciliation, Strimzi will query the broker’s supported features via the Kafka Admin API (DescribeFeatures).
2. If the feature is GA and still listed in .spec.kafka.previewFeatures, the operator will check whether:

• Kafka still allows managing it through the UpdateFeatures API.
• The requested version is valid for the current Kafka version.

Error Handling:

• If the feature is GA, enabled by default, and cannot be disabled, the operator will skip applying the change.Record a CannotBeDisabled condition in .status with a reason explaining that the feature is GA and mandatory.

• If the feature is GA but supports version changes or disabling, the operator will reconcile as per the CR.

• If the CR requests an unsupported version (either new or not available in this version), the operator will--Skips applying the change.

• Record a VersionNotSupported condition in .status listing the available versions from Kafka’s feature metadata.

Open to suggestions tho

[scholzj]

TBH, I'm not sure it is that simple.

I can see how when the feature moves to GA we just issue some warning condition. But if it enabled some feature and that feature version is not supported because it was replaced with a new version, it sounds like ignoring it by throwing a warning would essentially just disable the feature? That seems wrong if there is newer version? What does Kafka itself do in such situation?

Also, I think this is where you need to introduce proper context from how the Kafka community expects to use these features. Will each Kafka version support multiple versions of some feature (it sounds like it from the API you propose and that Kafka uses?)? Is the GA just a special version of the feature?

I think understanding it properly is important. And if it is not clear how Kafka will use it, we might not be able to decide on this without screwing up our API. So we would be blocked. To be honest, I think I didi not properly understood how this will evolve (maybe I missed somehting, maybe the problem is that Kafka community itself did not understood how this will evolve) when I wrote https://github.com/strimzi/proposals/blob/main/061-kraft-upgrades-and-downgrades.md as the metadata version seems to be related to this as well.

As a sidenote: You seem to talk about the status conditions in response to multiple comments. But there is nothing about it in the proposal. If that is how you plan to deal with these, it is important to cover it there as that is the only way how to have the discussion centered in a single place. And not have the discussion independently in multiple places and possibly lead to conflicting outcomes.

[Harithasridhar1306]

The operator will handle GA transitions during the reconciliation loop. It will use the Kafka DescribeFeatures API to discover which versions are supported by the current Kafka cluster. If a feature listed in .spec.kafka.previewFeatures has already become GA, Strimzi will first determine whether Kafka treats GA as just another version of the feature or whether it implies that the feature is now always enabled. If Kafka allows multiple versions of the same feature to coexist (as the API suggests), the operator will treat GA as the “stable” version and attempt to reconcile to the closest supported version available. If the CR requests a version that Kafka no longer supports (e.g., a preview version that has been removed), Strimzi will not silently disable the feature — instead, it will record a VersionNotSupported condition in .status and list which versions Kafka still supports, so the user is guided to update the CR. If Kafka makes the GA version mandatory and non-configurable, the operator will skip reconciling that field and surface a CannotBeDisabled condition in .status to make the user aware.

[scholzj]

If Kafka allows multiple versions of the same feature to coexist (as the API suggests), the operator will treat GA as the “stable” version and attempt to reconcile to the closest supported version available.

That is not really how declarative management works. You cannot just decide to use something what the operator things is similar and ignore the difference. There is also probably no way how to determine what is the closest version.

Strimzi will not silently disable the feature — instead, it will record a VersionNotSupported condition in .status and list which versions Kafka still supports, so the user is guided to update the CR. If Kafka makes the GA version mandatory and non-configurable, the operator will skip reconciling that field and surface a CannotBeDisabled condition in .status to make the user aware.

So, will this fail the reocnciliation and for example block an upgrade?

[scholzj]

You should provide details of how exactly will the operator do it -> which phase, which classes, etc. Also, do we need some trakcing of the current state of these features?

[tinaselenge]

+1

[scholzj]

Are these 100% previews of new features only?

[tinaselenge]

How about a feature that is not preview, but Kafka introduced a new version? For example, users want to try the new version of this feature.

What happens when share.version is production ready and enabled by default but the user wants disable it for some reason?

[Harithasridhar1306]

The .spec.kafka.previewFeatures field will only control features that Kafka exposes via the Feature API — whether they are in preview or already GA but still configurable.

1. If the feature is GA but still configurable:
   The operator will allow the user to set it explicitly in previewFeatures even though the name might be misleading, we can rename it to kafkaFeatures if needed.

2. If the feature is GA and not configurable anymore:
   The operator will ignore the CR entry for that feature, set a Warning condition in .status explaining that it’s GA and mandatory, and proceed with the rest.

3. If the feature is GA with multiple versions:
   Users can still request a specific version in .spec.kafka.previewFeatures and the operator will attempt to set it if Kafka permits.

[ppatierno]

I think we should change from previewFeatures to just features (it's clear they are Kafka features, because we are under the spec.kafka section).

[tinaselenge]

I agree. I think previewFeatures is misleading for GA features.

[tinaselenge]

Thank you for the proposal. I left some comments.

[tinaselenge]

Suggested change

	- Feature name (e.g., kafka.shared.consumer)
	- Feature name (e.g., share.version)

[tinaselenge]

Are these limitation of your proposed solution or another solution that is in the the reject alternatives? Or should be in the Motivation section?

[tinaselenge]

Isn't Admin API handling will be part the changes in the first point? Strimzi image is always updated anyway in a new version. So this sounds like you would change the existing image?

[im-konge]

I guess the "potential" is wrong word TBH.
With every change to the code (and every build of the image) you will have some updates in comparison with the current images.
Also, it's not a project, so I would remove this.

[im-konge]

Just thinking (as I saw it on one of the previous lines) - now when Strimzi and Kafka support just KRaft, should we mention Zookeeper here? I know what you mean, but I guess we just care about KRaft.

[im-konge]

Also, how it will work during upgrades of the Kafka version? Is there a way how to check what are the available features that can be enabled and in case that in the list of the previewFeatures inside the .spec.kafka will be something wrong (not available anymore or graduated to GA), should it be ignored and mentioned in the status of the Kafka CR?

[im-konge]

The same thing should be with the downgrade, some of the features will not be available if you will do downgrade of the Kafka cluster from one version to the other, so that should be handled as well.

[Harithasridhar1306]

Also, how it will work during upgrades of the Kafka version? Is there a way how to check what are the available features that can be enabled and in case that in the list of the previewFeatures inside the .spec.kafka will be something wrong (not available anymore or graduated to GA), should it be ignored and mentioned in the status of the Kafka CR?

Upgrade Scenario:

Queries the broker’s available features via Admin API.

Compares them with .spec.kafka.previewFeatures.

For each entry in .spec:

If still preview and supported → apply as requested.

If graduated to GA → skip enabling (Kafka enables it by default); record in .status with reason=FeatureGraduatedToGA.

If removed → skip enabling; record in .status with reason=FeatureUnavailable.

The cluster stays running without error, but users get visible feedback in status.conditions.

is what i was thinking -open to suggestions

[Harithasridhar1306]

The same thing should be with the downgrade, some of the features will not be available if you will do downgrade of the Kafka cluster from one version to the other, so that should be handled as well.

Downgrade Scenario:

If a feature is no longer supported in the downgraded version, the operator disables it via Admin API and records in .status with reason=FeatureUnsupportedInVersion.

is what i was thinking -open to suggestions

[im-konge]

Yeah you should mention it in the proposal, so it is covered.

[im-konge]

How the validation will look like? How the user will be informed that there is something wrong? Also, should we keep some information about enabled features inside the .status section of the Kafka CR?

[Harithasridhar1306]

The operator will validate requested features before applying them. If there's an invalid feature name, unsupported version, or other mismatch, the operator will:

Skip the problematic entry to avoid cluster failure, and

Record an entry under .status.previewFeatures including: feature name, requested versus applied version, state (Enabled, Skipped, etc.), and a clear reason (InvalidFeatureName, UnsupportedVersion).
This lets users immediately see what was applied and why something failed.

• name: kafka.old.preview
  requestedVersion: 1
  appliedVersion: null
  state: Skipped
  reason: "Feature not available in Kafka 3.5."

  • name: kafka.new.featureX
    requestedVersion: 2
    appliedVersion: null
    state: Skipped
    reason: "Invalid feature name."

  hope this answers your question

[im-konge]

That's good, could you please add it to the proposal? I guess it's important to know how it will be handled. Thanks :)

[im-konge]

I don't understand this section - in benefits section you mention how it is declarative, here you mention that it is not declarative.
Also that it doesn't have any automatic reconciliation, which contradicts the fact you mentioned above that during the reconciliation the features will be checked etc.

So I would either remove this or move it somewhere else (because you anyway mentioned the currect situation above).

[Harithasridhar1306]

sorry, my bad ---this limitation was for the rejected version
i shall move it

[im-konge]

I guess the "potential" is wrong word TBH.
With every change to the code (and every build of the image) you will have some updates in comparison with the current images.
Also, it's not a project, so I would remove this.

[im-konge]

Suggested change

	The previewFeatures field will be optional, and clusters without it will behave as they do today.
	The `previewFeatures` field will be optional, and clusters without it will behave as they do today.

[im-konge]

This is good, but you should maybe mention why it is rejected alternative.

[Harithasridhar1306]

right
i shall add that too

[ppatierno]

Suggested change

	Kafka preview features (such as `kafka.shared.consumer`) need to be manually enabled using the ¨kafka-features.sh` or the Kafka Admin API.
	Kafka preview features (such as `kafka.shared.consumer`) need to be manually enabled using the `kafka-features.sh` or programmatically by using the Kafka Admin API.

[tinaselenge]

It is kafka share consumer, not shared consumer.

[ppatierno]

What does exactly means static vs dynamic?

[Harithasridhar1306]

By static clusters, I mean long-lived, relatively stable Kafka environments where the configuration is applied once and rarely changes—for example, production clusters that are provisioned manually or with minimal automation. In these cases, enabling a preview feature manually through kafka-features.sh or the Admin API is manageable, since the cluster is not frequently recreated.

By dynamic clusters, I mean short-lived or continuously reconciled environments, such as clusters for testing, CI/CD pipelines, or GitOps-managed deployments where the operator ensures the declared state is always applied. In these scenarios, manually running kafka-features.sh isn’t practical because the feature enablement would be lost on cluster re-creation or drifted from the GitOps source of truth.

[scholzj]

I do not think this is a terminology we use anywhere, so you should explain it in the proposal.

However, I think you got one thing wrong:

• The static production clusters often use GitOps and often cannot use the kafka-features.sh command because the only proper and approved way to change things is through GitOps processes (i.e. they would for example need to manage it through a Kubernetes Job calling kafka-features.sh which is submitted through GitOps).
• The dynamic shortlived clusters would nto want to use kafka-features.sh every five minutes to set it. But then - you IMHO do not use that often GitOps to create a cluster for a very short time and then delete it again through GitOps. So the GitOps part of tje argument here - I do not get it.

[Harithasridhar1306]

I see your point about the terminology not being commonly used, and I’ll adjust the proposal to avoid introducing “static/dynamic” and instead describe clusters in terms of lifecycle and management style.

You’re also right that production clusters often rely on GitOps as the only approved mechanism for changes, so running kafka-features.sh directly wouldn’t fit those processes. I’ll reframe the explanation to emphasize that:
Prod clusters: usually GitOps-managed, requiring any feature enablement to be expressed declaratively, rather than direct CLI commands.

Non-prod clusters : recreated frequently, so manual commands don’t persist across rebuilds. While GitOps is less common here, automation via CI pipelines or scripts is needed instead of manual steps.

[ppatierno]

Suggested change

	### Kafka Preview Features Explained
	### Kafka Preview Features Explained

[ppatierno]

Suggested change

	* Enabled/disabled flag
	* Enabled/disabled flag

[ppatierno]

If I use the kafka-features.sh tool and describe the features, what I get is something like this:

Feature: eligible.leader.replicas.version	SupportedMinVersion: 0	SupportedMaxVersion: 1	FinalizedVersionLevel: 1	Epoch: 8836
Feature: group.version	SupportedMinVersion: 0	SupportedMaxVersion: 1	FinalizedVersionLevel: 1	Epoch: 8836
Feature: kraft.version	SupportedMinVersion: 0	SupportedMaxVersion: 1	FinalizedVersionLevel: 0	Epoch: 8836
Feature: metadata.version	SupportedMinVersion: 3.3-IV3	SupportedMaxVersion: 4.0-IV3	FinalizedVersionLevel: 4.0-IV3	Epoch: 8836
Feature: transaction.version	SupportedMinVersion: 0	SupportedMaxVersion: 2	FinalizedVersionLevel: 2	Epoch: 8836

How the fields provided by the above output match with what you are listing as "feature state"?

[Harithasridhar1306]

The “enabled/disabled” flag in the proposal is really a simplification — what Kafka actually exposes through the Admin API and via kafka-features.sh is the finalized version level of each feature. If the finalized version level is greater than 0 or greater than the minimum supported version, depending on the feature, then we can treat the feature as effectively “enabled.” If it’s set to 0 or not finalized, we can treat it as “disabled.” The SupportedMinVersion and SupportedMaxVersion values describe the full range of versions that the cluster supports, while the Epoch just tracks changes over time. So, when the proposal refers to a “feature state,” it is really about reconciling the finalized version level and whether it falls in the enabled or disabled range, not inventing a new flag outside of what Kafka already provides.

[scholzj]

Please keep in mind that the APi cannot be changed. So while such as implification might seem to make sense, we need to be 101% sure that it will work in all cases for the next N years. Obviously, that is not always easy to know/decide. So the choice between a custom abstraction in our API and mirroring of the source API is always comploicated and hard.

[Harithasridhar1306]

I completely agree that we cannot and should not change the Kafka API itself. My intention with the “enabled/disabled” simplification is not to introduce a new abstraction that overrides Kafka’s semantics, but rather to describe how Strimzi might present the existing finalized version level in a more user-friendly way. Internally, it would still rely on the Kafka Admin API fields (FinalizedVersionLevel, SupportedMinVersion, etc.) without altering their meaning. I’ll make sure to clarify this better in the proposal so it’s clear that we’re only mapping what Kafka already exposes, not creating a divergent API surface.

[ppatierno]

I am not sure why the Kafka community named FinalizedVersionLevel as the filed to specified the version but I think we should go a similar way and not just using enable/disabled while underneath it could mean a version value as 0,1,2 or whatever.

[ppatierno]

I would not mention ZooKeeper anymore, now that it's not supported.

[ppatierno]

Suggested change

	## Proposal
	## Proposal

[ppatierno]

I think we should change from previewFeatures to just features (it's clear they are Kafka features, because we are under the spec.kafka section).

[ppatierno]

I am not sure we should talk about "Preview" features. I think we should remove the "Preview" part of the naming everywhere because even if at the beginning some features are preview, at some point they are GAs, and they can have newer versions time to time.

[ppatierno]

we should definitively go via Kafka Admin API and not using any bin tool within the operator.
If anything is missing via the API then it could be room for improvements within the Kafka project itself imho.

[Harithasridhar1306]

You’re absolutely right — the Operator should rely on the Kafka Admin API rather than invoking kafka-features.sh. The reference to kafka-features.sh in the proposal was meant only as an example of how users interact with preview features, not as an implementation detail for Strimzi. For the reconciliation logic itself, the plan is to query and update feature states through the Admin API.

[ppatierno]

The reference to kafka-features.sh in the proposal was meant only as an example of how users interact with preview features, not as an implementation detail for Strimzi. For the reconciliation logic itself, the plan is to query and update feature states through the Admin API.

But this is not what the above point 2) states. It says the operator uses Admin API or kafka-features.sh and the "or" is wrong. The operator uses Admin API only, so it should be fixed.