Documentation | Stackable Data Platform | Platform Docs | Discussions | Discord
This is a Kubernetes operator to manage Apache Kafka clusters.
It is part of the Stackable Data Platform, a curated selection of the best open source data apps like Kafka, Druid, Trino or Spark, all working together seamlessly. Based on Kubernetes, it runs everywhere – on prem or in the cloud.
You can install the operator using stackablectl or helm.
Read on to get started with it, or see it in action in one of our demos.
You can follow this tutorial to create a Kafka cluster with three replicas.
The stable documentation for this operator can be found here. If you are interested in the most recent state of this repository, check out the nightly docs instead.
The documentation for all Stackable products can be found at docs.stackable.tech.
If you have a question about the Stackable Data Platform contact us via our homepage or ask a public questions in our Discussions forum.
This operator is managing Kafka in Kubernetes. An operator observes custom resources in a Kubernetes cluster, and manages the application it's responsible for according to the desired state specified in those custom resources.
Here's an example how a custom resource to bring up a 3-replica Kafka cluster can look like:
---
apiVersion: kafka.stackable.tech/v1alpha1
kind: KafkaCluster
metadata:
name: simple-kafka
spec:
image:
productVersion: 3.3.1
stackableVersion: "23.4.0-rc2"
clusterConfig:
zookeeperConfigMapName: simple-kafka-znode
tls:
serverSecretClass: null
brokers:
roleGroups:
default:
replicas: 3
If you want to find out more about it, take a look at this getting started guide.
This operator is written by Stackable. It uses kube.rs to interact with the Kubernetes control plane.
We test it extensively using Kuttl-powered integration tests on managed Kubernetes of multiple cloud platforms and our own bare-metal clusters.
This operator is written and maintained by Stackable and it is part of a larger data platform.
Stackable makes it easy to operate data applications in any Kubernetes cluster.
The data platform offers many operators, new ones being added continuously. All our operators are designed and built to be easily interconnected and to be consistent to work with.
The Stackable GmbH is the company behind the Stackable Data Platform. Offering professional services, paid support plans and custom development.
We love open-source!
We develop and test our operators on the following cloud platforms:
- AKS on Microsoft Azure
- EKS on Amazon Web Services (AWS)
- GKE on Google Cloud Platform (GCP)
- IONOS Cloud Managed Kubernetes
- K3s
- Kubernetes 1.21-1.25
We are currently working to support:
- OpenShift
These are the operators that are currently part of the Stackable Data Platform:
- Stackable Operator for Apache Airflow
- Stackable Operator for Apache Druid
- Stackable Operator for Apache HBase
- Stackable Operator for Apache Hadoop HDFS
- Stackable Operator for Apache Hive
- Stackable Operator for Apache Kafka
- Stackable Operator for Apache NiFi
- Stackable Operator for Apache Spark
- Stackable Operator for Apache Superset
- Stackable Operator for Trino
- Stackable Operator for Apache ZooKeeper
And our internal operators:
Contributions are welcome. Follow our Contributors Guide to learn how you can contribute.
Open Software License version 3.0.
You can use this project under different licenses. Get started with the community edition! If you want professional support, we offer subscription plans.