kafka
: This class handles the Kafka requirements.kafka::broker
: This class handles the Kafka (broker).kafka::consumer
: This class handles the Kafka (consumer).kafka::mirror
: This class handles the Kafka (mirror).kafka::params
: This class provides default parameters.kafka::producer
: This class handles the Kafka (producer).
kafka::broker::config
: This class handles the Kafka (broker) config.kafka::broker::install
: This class handles the Kafka (broker) package.kafka::broker::service
: This class handles the Kafka (broker) service.kafka::consumer::config
: This class handles the Kafka (consumer) config.kafka::consumer::install
: This class handles the Kafka (consumer) package.kafka::consumer::service
: This class handles the Kafka (consumer) service.kafka::mirror::config
: This class handles the Kafka (mirror) config.kafka::mirror::install
: This class handles the Kafka (mirror) package.kafka::mirror::service
: This class handles the Kafka (mirror) service.kafka::producer::config
: This class handles the Kafka (producer) config.kafka::producer::install
: This class handles the Kafka (producer) package.kafka::producer::service
: This class handles the Kafka (producer) service.
kafka::topic
: This defined type handles the creation of Kafka topics.
This class handles the Kafka requirements.
class { 'kafka': }
The following parameters are available in the kafka
class:
kafka_version
scala_version
install_dir
mirror_url
manage_java
package_dir
package_name
mirror_subpath
proxy_server
proxy_port
proxy_host
proxy_type
package_ensure
user_name
user_shell
group_name
system_user
system_group
user_id
group_id
manage_user
manage_group
config_dir
log_dir
install_mode
Data type: String[1]
The version of Kafka that should be installed.
Default value: $kafka::params::kafka_version
Data type: String[1]
The scala version what Kafka was built with.
Default value: $kafka::params::scala_version
Data type: Stdlib::Absolutepath
The directory to install Kafka to.
Default value: $kafka::params::install_dir
Data type: Stdlib::HTTPUrl
The url where the Kafka is downloaded from.
Default value: $kafka::params::mirror_url
Data type: Boolean
Install java if it's not already installed.
Default value: $kafka::params::manage_java
Data type: Stdlib::Absolutepath
The directory to install Kafka.
Default value: $kafka::params::package_dir
Data type: Optional[String[1]]
Package name, when installing Kafka from a package.
Default value: $kafka::params::package_name
Data type: Optional[String[1]]
The sub directory where the source is downloaded from.
Default value: $kafka::params::mirror_subpath
Data type: Optional[String[1]]
Set proxy server, when installing Kafka from source.
Default value: $kafka::params::proxy_server
Data type: Optional[String[1]]
Set proxy port, when installing Kafka from source.
Default value: $kafka::params::proxy_port
Data type: Optional[String[1]]
Set proxy host, when installing Kafka from source.
Default value: $kafka::params::proxy_host
Data type: Optional[String[1]]
Set proxy type, when installing Kafka from source.
Default value: $kafka::params::proxy_type
Data type: String[1]
Package version or ensure state, when installing Kafka from a package.
Default value: $kafka::params::package_ensure
Data type: String[1]
User to run Kafka as.
Default value: $kafka::params::user_name
Data type: Stdlib::Absolutepath
Login shell of the Kafka user.
Default value: $kafka::params::user_shell
Data type: String[1]
Group to run Kafka as.
Default value: $kafka::params::group_name
Data type: Boolean
Whether the Kafka user is a system user or not.
Default value: $kafka::params::system_user
Data type: Boolean
Whether the Kafka group is a system group or not.
Default value: $kafka::params::system_group
Data type: Optional[Integer]
Create the Kafka user with this ID.
Default value: $kafka::params::user_id
Data type: Optional[Integer]
Create the Kafka group with this ID.
Default value: $kafka::params::group_id
Data type: Boolean
Create the Kafka user if it's not already present.
Default value: $kafka::params::manage_user
Data type: Boolean
Create the Kafka group if it's not already present.
Default value: $kafka::params::manage_group
Data type: Stdlib::Absolutepath
The directory to create the Kafka config files to.
Default value: $kafka::params::config_dir
Data type: Stdlib::Absolutepath
The directory for Kafka log files.
Default value: $kafka::params::log_dir
Data type: Stdlib::Filemode
The permissions for the install directory.
Default value: $kafka::params::install_mode
This class handles the Kafka (broker).
class { 'kafka::broker':
config => {
'broker.id' => '0',
'zookeeper.connect' => 'localhost:2181'
}
}
The following parameters are available in the kafka::broker
class:
kafka_version
scala_version
install_dir
mirror_url
manage_java
package_dir
package_name
package_ensure
user_name
user_shell
group_name
user_id
group_id
manage_user
manage_group
config_mode
config_dir
log_dir
bin_dir
service_name
manage_service
service_ensure
service_restart
service_requires
limit_nofile
limit_core
timeout_stop
exec_stop
daemon_start
env
config
heap_opts
jmx_opts
log4j_opts
manage_log4j
log4j_content
log_file_size
log_file_count
opts
Data type: String[1]
The version of Kafka that should be installed.
Default value: $kafka::params::kafka_version
Data type: String[1]
The scala version what Kafka was built with.
Default value: $kafka::params::scala_version
Data type: Stdlib::Absolutepath
The directory to install Kafka to.
Default value: $kafka::params::install_dir
Data type: Stdlib::HTTPUrl
The url where the Kafka is downloaded from.
Default value: $kafka::params::mirror_url
Data type: Boolean
Install java if it's not already installed.
Default value: $kafka::params::manage_java
Data type: Stdlib::Absolutepath
The directory to install Kafka.
Default value: $kafka::params::package_dir
Data type: Optional[String[1]]
Package name, when installing Kafka from a package.
Default value: $kafka::params::package_name
Data type: String[1]
Package version or ensure state, when installing Kafka from a package.
Default value: $kafka::params::package_ensure
Data type: String[1]
User to run Kafka as.
Default value: $kafka::params::user_name
Data type: Stdlib::Absolutepath
Login shell of the Kafka user.
Default value: $kafka::params::user_shell
Data type: String[1]
Group to run Kafka as.
Default value: $kafka::params::group_name
Data type: Optional[Integer]
Create the Kafka user with this ID.
Default value: $kafka::params::user_id
Data type: Optional[Integer]
Create the Kafka group with this ID.
Default value: $kafka::params::group_id
Data type: Boolean
Create the Kafka user if it's not already present.
Default value: $kafka::params::manage_user
Data type: Boolean
Create the Kafka group if it's not already present.
Default value: $kafka::params::manage_group
Data type: Stdlib::Filemode
The permissions for the config files.
Default value: $kafka::params::config_mode
Data type: Stdlib::Absolutepath
The directory to create the Kafka config files to.
Default value: $kafka::params::config_dir
Data type: Stdlib::Absolutepath
The directory for Kafka log files.
Default value: $kafka::params::log_dir
Data type: Stdlib::Absolutepath
The directory where the Kafka scripts are.
Default value: $kafka::params::bin_dir
Data type: String[1]
Set the name of the service.
Default value: 'kafka'
Data type: Boolean
Install the systemd service.
Default value: $kafka::params::manage_service
Data type: Enum['running', 'stopped']
Set the ensure state of the service.
Default value: $kafka::params::service_ensure
Data type: Boolean
Whether the configuration files should trigger a service restart.
Default value: $kafka::params::service_restart
Data type: Array[String[1]]
Set the list of services required to be running before Kafka.
Default value: $kafka::params::service_requires
Data type: Optional[String[1]]
Set the 'LimitNOFILE' option of the systemd service.
Default value: $kafka::params::limit_nofile
Data type: Optional[String[1]]
Set the 'LimitCORE' option of the systemd service.
Default value: $kafka::params::limit_core
Data type: Optional[String[1]]
Set the 'TimeoutStopSec' option of the systemd service.
Default value: $kafka::params::timeout_stop
Data type: Boolean
Set the 'ExecStop' option of the systemd service to 'kafka-server-stop.sh'.
Default value: $kafka::params::exec_stop
Data type: Boolean
Use the '-daemon' option when starting Kafka with 'kafka-server-start.sh'.
Default value: $kafka::params::daemon_start
Data type: Hash
A hash of the environment variables to set.
Default value: {}
Data type: Hash[String[1], Any]
A hash of the broker configuration options.
Default value: {}
Data type: String[1]
Set the Java heap size.
Default value: $kafka::params::broker_heap_opts
Data type: String[1]
Set the JMX options.
Default value: $kafka::params::broker_jmx_opts
Data type: String[1]
Set the Log4j options.
Default value: $kafka::params::broker_log4j_opts
Data type: Boolean
Enable or disable the management of the log4j configuration file.
Default value: $kafka::params::manage_log4j
Data type: Optional[String[1]]
Provide custom content for the log4j configuration file if manage_log4j is enabled, instead of using the content provided in this module.
Default value: undef
Data type: Pattern[/[1-9][0-9]*[KMG]B/]
Set the max size of the Kafka log files before they are rolled.
Default value: $kafka::params::log_file_size
Data type: Integer[1, 50]
Set the number of Kafka log files to keep.
Default value: $kafka::params::log_file_count
Data type: String[0]
Set the Kafka options.
Default value: $kafka::params::broker_opts
This class handles the Kafka (consumer).
class { 'kafka::consumer':
config => {
'client.id' => '0',
'zookeeper.connect' => 'localhost:2181'
}
}
The following parameters are available in the kafka::consumer
class:
kafka_version
scala_version
install_dir
mirror_url
manage_java
package_dir
package_name
package_ensure
user_name
user_shell
group_name
user_id
group_id
manage_user
manage_group
config_mode
config_dir
log_dir
bin_dir
service_name
manage_service
service_ensure
service_restart
service_requires
limit_nofile
limit_core
env
config
service_config
jmx_opts
log4j_opts
manage_log4j
log_file_size
log_file_count
Data type: String[1]
The version of Kafka that should be installed.
Default value: $kafka::params::kafka_version
Data type: String[1]
The scala version what Kafka was built with.
Default value: $kafka::params::scala_version
Data type: Stdlib::Absolutepath
The directory to install Kafka to.
Default value: $kafka::params::install_dir
Data type: Stdlib::HTTPUrl
The url where the Kafka is downloaded from.
Default value: $kafka::params::mirror_url
Data type: Boolean
Install java if it's not already installed.
Default value: $kafka::params::manage_java
Data type: Stdlib::Absolutepath
The directory to install Kafka.
Default value: $kafka::params::package_dir
Data type: Optional[String[1]]
Package name, when installing Kafka from a package.
Default value: $kafka::params::package_name
Data type: String[1]
Package version or ensure state, when installing Kafka from a package.
Default value: $kafka::params::package_ensure
Data type: String[1]
User to run Kafka as.
Default value: $kafka::params::user_name
Data type: Stdlib::Absolutepath
Login shell of the Kafka user.
Default value: $kafka::params::user_shell
Data type: String[1]
Group to run Kafka as.
Default value: $kafka::params::group_name
Data type: Optional[Integer]
Create the Kafka user with this ID.
Default value: $kafka::params::user_id
Data type: Optional[Integer]
Create the Kafka group with this ID.
Default value: $kafka::params::group_id
Data type: Boolean
Create the Kafka user if it's not already present.
Default value: $kafka::params::manage_user
Data type: Boolean
Create the Kafka group if it's not already present.
Default value: $kafka::params::manage_group
Data type: Stdlib::Filemode
The permissions for the config files.
Default value: $kafka::params::config_mode
Data type: Stdlib::Absolutepath
The directory to create the Kafka config files to.
Default value: $kafka::params::config_dir
Data type: Stdlib::Absolutepath
The directory for Kafka log files.
Default value: $kafka::params::log_dir
Data type: Stdlib::Absolutepath
The directory where the Kafka scripts are.
Default value: $kafka::params::bin_dir
Data type: String[1]
Set the name of the service.
Default value: 'kafka-consumer'
Data type: Boolean
Install the systemd service.
Default value: $kafka::params::manage_service
Data type: Enum['running', 'stopped']
Set the ensure state of the service.
Default value: $kafka::params::service_ensure
Data type: Boolean
Whether the configuration files should trigger a service restart.
Default value: $kafka::params::service_restart
Data type: Array[String[1]]
Set the list of services required to be running before Kafka.
Default value: $kafka::params::service_requires
Data type: Optional[String[1]]
Set the 'LimitNOFILE' option of the systemd service.
Default value: $kafka::params::limit_nofile
Data type: Optional[String[1]]
Set the 'LimitCORE' option of the systemd service.
Default value: $kafka::params::limit_core
Data type: Hash
A hash of the environment variables to set.
Default value: {}
Data type: Hash[String[1], Any]
A hash of the consumer configuration options.
Default value: {}
Data type: Hash[String[1],String[1]]
A hash of the kafka-console-consumer.sh
script options.
Default value: {}
Data type: String[1]
Set the JMX options.
Default value: $kafka::params::consumer_jmx_opts
Data type: String[1]
Set the Log4j options.
Default value: $kafka::params::consumer_log4j_opts
Data type: Boolean
Enable or disable the management of the log4j configuration file.
Default value: $kafka::params::manage_log4j
Data type: Pattern[/[1-9][0-9]*[KMG]B/]
Set the max size of the Kafka log files before they are rolled.
Default value: $kafka::params::log_file_size
Data type: Integer[1, 50]
Set the number of Kafka log files to keep.
Default value: $kafka::params::log_file_count
This class handles the Kafka (mirror).
class { 'kafka::mirror':
consumer_config => {
'group.id' => 'kafka-mirror',
'zookeeper.connect' => 'localhost:2181'
},
producer_config => {
'zookeeper.connect' => 'localhost:2181',
},
service_config => {
'whitelist' => '.*',
}
}
The following parameters are available in the kafka::mirror
class:
kafka_version
scala_version
install_dir
mirror_url
manage_java
package_dir
package_name
package_ensure
user_name
user_shell
group_name
user_id
group_id
manage_user
manage_group
config_mode
config_dir
log_dir
bin_dir
service_name
manage_service
service_ensure
service_restart
service_requires
limit_nofile
limit_core
env
consumer_config
producer_config
service_config
heap_opts
jmx_opts
log4j_opts
manage_log4j
log_file_size
log_file_count
Data type: String[1]
The version of Kafka that should be installed.
Default value: $kafka::params::kafka_version
Data type: String[1]
The scala version what Kafka was built with.
Default value: $kafka::params::scala_version
Data type: Stdlib::Absolutepath
The directory to install Kafka to.
Default value: $kafka::params::install_dir
Data type: Stdlib::HTTPUrl
The url where the Kafka is downloaded from.
Default value: $kafka::params::mirror_url
Data type: Boolean
Install java if it's not already installed.
Default value: $kafka::params::manage_java
Data type: Stdlib::Absolutepath
The directory to install Kafka.
Default value: $kafka::params::package_dir
Data type: Optional[String[1]]
Package name, when installing Kafka from a package.
Default value: $kafka::params::package_name
Data type: String[1]
Package version or ensure state, when installing Kafka from a package.
Default value: $kafka::params::package_ensure
Data type: String[1]
User to run Kafka as.
Default value: $kafka::params::user_name
Data type: Stdlib::Absolutepath
Login shell of the Kafka user.
Default value: $kafka::params::user_shell
Data type: String[1]
Group to run Kafka as.
Default value: $kafka::params::group_name
Data type: Optional[Integer]
Create the Kafka user with this ID.
Default value: $kafka::params::user_id
Data type: Optional[Integer]
Create the Kafka group with this ID.
Default value: $kafka::params::group_id
Data type: Boolean
Create the Kafka user if it's not already present.
Default value: $kafka::params::manage_user
Data type: Boolean
Create the Kafka group if it's not already present.
Default value: $kafka::params::manage_group
Data type: Stdlib::Filemode
The permissions for the config files.
Default value: $kafka::params::config_mode
Data type: Stdlib::Absolutepath
The directory to create the Kafka config files to.
Default value: $kafka::params::config_dir
Data type: Stdlib::Absolutepath
The directory for Kafka log files.
Default value: $kafka::params::log_dir
Data type: Stdlib::Absolutepath
The directory where the Kafka scripts are.
Default value: $kafka::params::bin_dir
Data type: String[1]
Set the name of the service.
Default value: 'kafka-mirror'
Data type: Boolean
Install the systemd service.
Default value: $kafka::params::manage_service
Data type: Enum['running', 'stopped']
Set the ensure state of the service.
Default value: $kafka::params::service_ensure
Data type: Boolean
Whether the configuration files should trigger a service restart.
Default value: $kafka::params::service_restart
Data type: Array[String[1]]
Set the list of services required to be running before Kafka.
Default value: $kafka::params::service_requires
Data type: Optional[String[1]]
Set the 'LimitNOFILE' option of the systemd service.
Default value: $kafka::params::limit_nofile
Data type: Optional[String[1]]
Set the 'LimitCORE' option of the systemd service.
Default value: $kafka::params::limit_core
Data type: Hash
A hash of the environment variables to set.
Default value: {}
Data type: Hash[String[1],String[1]]
A hash of the consumer configuration options.
Default value: {}
Data type: Hash[String[1],String[1]]
A hash of the producer configuration options.
Default value: {}
Data type: Hash[String[1],String[1]]
A hash of the mirror script options.
Default value: {}
Data type: String[1]
Set the Java heap size.
Default value: $kafka::params::mirror_heap_opts
Data type: String[1]
Set the JMX options.
Default value: $kafka::params::mirror_jmx_opts
Data type: String[1]
Set the Log4j options.
Default value: $kafka::params::mirror_log4j_opts
Data type: Boolean
Enable or disable the management of the log4j configuration file.
Default value: $kafka::params::manage_log4j
Data type: Pattern[/[1-9][0-9]*[KMG]B/]
Set the max size of the Kafka log files before they are rolled.
Default value: $kafka::params::log_file_size
Data type: Integer[1, 50]
Set the number of Kafka log files to keep.
Default value: $kafka::params::log_file_count
This class provides default parameters.
This class handles the Kafka (producer).
class { 'kafka::producer':
config => {
'client.id' => '0',
'zookeeper.connect' => 'localhost:2181'
}
}
The following parameters are available in the kafka::producer
class:
input
kafka_version
scala_version
install_dir
mirror_url
manage_java
package_dir
package_name
package_ensure
user_name
user_shell
group_name
user_id
group_id
manage_user
manage_group
config_mode
config_dir
log_dir
bin_dir
service_name
manage_service
service_ensure
service_restart
service_requires
limit_nofile
limit_core
env
config
service_config
jmx_opts
log4j_opts
Data type: Optional[String[1]]
Set named pipe as input.
Data type: String[1]
The version of Kafka that should be installed.
Default value: $kafka::params::kafka_version
Data type: String[1]
The scala version what Kafka was built with.
Default value: $kafka::params::scala_version
Data type: Stdlib::Absolutepath
The directory to install Kafka to.
Default value: $kafka::params::install_dir
Data type: Stdlib::HTTPUrl
The url where the Kafka is downloaded from.
Default value: $kafka::params::mirror_url
Data type: Boolean
Install java if it's not already installed.
Default value: $kafka::params::manage_java
Data type: Stdlib::Absolutepath
The directory to install Kafka.
Default value: $kafka::params::package_dir
Data type: Optional[String[1]]
Package name, when installing Kafka from a package.
Default value: $kafka::params::package_name
Data type: String[1]
Package version or ensure state, when installing Kafka from a package.
Default value: $kafka::params::package_ensure
Data type: String[1]
User to run Kafka as.
Default value: $kafka::params::user_name
Data type: Stdlib::Absolutepath
Login shell of the Kafka user.
Default value: $kafka::params::user_shell
Data type: String[1]
Group to run Kafka as.
Default value: $kafka::params::group_name
Data type: Optional[Integer]
Create the Kafka user with this ID.
Default value: $kafka::params::user_id
Data type: Optional[Integer]
Create the Kafka group with this ID.
Default value: $kafka::params::group_id
Data type: Boolean
Create the Kafka user if it's not already present.
Default value: $kafka::params::manage_user
Data type: Boolean
Create the Kafka group if it's not already present.
Default value: $kafka::params::manage_group
Data type: Stdlib::Filemode
The permissions for the config files.
Default value: $kafka::params::config_mode
Data type: Stdlib::Absolutepath
The directory to create the Kafka config files to.
Default value: $kafka::params::config_dir
Data type: Stdlib::Absolutepath
The directory for Kafka log files.
Default value: $kafka::params::log_dir
Data type: Stdlib::Absolutepath
The directory where the Kafka scripts are.
Default value: $kafka::params::bin_dir
Data type: String[1]
Set the name of the service.
Default value: 'kafka-producer'
Data type: Boolean
Install systemd service.
Default value: $kafka::params::manage_service
Data type: Enum['running', 'stopped']
Set the ensure state of the service.
Default value: $kafka::params::service_ensure
Data type: Boolean
Whether the configuration files should trigger a service restart.
Default value: $kafka::params::service_restart
Data type: Array[String[1]]
Set the list of services required to be running before Kafka.
Default value: $kafka::params::service_requires
Data type: Optional[String[1]]
Set the 'LimitNOFILE' option of the systemd service.
Default value: $kafka::params::limit_nofile
Data type: Optional[String[1]]
Set the 'LimitCORE' option of the systemd service.
Default value: $kafka::params::limit_core
Data type: Hash
A hash of the environment variables to set.
Default value: {}
Data type: Hash[String[1], Any]
A hash of the producer configuration options.
Default value: {}
Data type: Hash[String[1],String[1]]
A hash of the kafka-console-producer.sh
script options.
Default value: {}
Data type: String[1]
Set the JMX options.
Default value: $kafka::params::producer_jmx_opts
Data type: String[1]
Set the Log4j options.
Default value: $kafka::params::producer_log4j_opts
This defined type handles the creation of Kafka topics.
kafka::topic { 'test':
ensure => present,
zookeeper => 'localhost:2181',
replication_factor => 1,
partitions => 1,
}
The following parameters are available in the kafka::topic
defined type:
Data type: Optional[String[1]]
Should the topic be created.
Default value: undef
Data type: Optional[String[1]]
The connection string for the ZooKeeper connection in the form host:port. Multiple hosts can be given to allow fail-over. Kafka < 3.0.0 only!
Default value: undef
Data type: Optional[String[1]]
The Kafka server to connect to in the form host:port. Kafka >= 2.2.0 only!
Default value: undef
Data type: Integer
The replication factor for each partition in the topic being created. If not supplied, defaults to the cluster default.
Default value: 1
Data type: Integer
The number of partitions for the topic being created or altered. If not supplied for create, defaults to the cluster default.
Default value: 1
Data type: String[1]
The directory where the file kafka-topics.sh is located.
Default value: '/opt/kafka/bin'
Data type: Optional[Hash[String[1],String[1]]]
A topic configuration override for the topic being created or altered. See the Kafka documentation for full details on the topic configs.
Default value: undef