NIFI-10977 Added documentation on Kubernetes Clustering

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes apache#8612.

nifi-docs/src/main/asciidoc/administration-guide.adoc

@@ -2447,14 +2447,6 @@ configured in the _state-management.xml_ file. See <<state_providers>> for more
 ** `nifi.cluster.node.protocol.max.threads` - The maximum number of threads that should be used to communicate with other nodes in the cluster. This property
 defaults to `50`. A thread pool is used for replicating requests to all nodes. The thread pool will increase the number of active threads to the limit
 set by this property. It is typically recommended that this property be set to 4-8 times the number of nodes in your cluster. There could be up to `n+2` threads for a given request, where `n` = number of nodes in your cluster. As an example, if 4 requests are made, a 5 node cluster will use `4 * 7 = 28` threads.
-** `nifi.zookeeper.connect.string` - The Connect String that is needed to connect to Apache ZooKeeper. This is a comma-separated list
-of hostname:port pairs. For example, `localhost:2181,localhost:2182,localhost:2183`. This should contain a list of all ZooKeeper
-instances in the ZooKeeper quorum.
-** `nifi.zookeeper.root.node` - The root ZNode that should be used in ZooKeeper. ZooKeeper provides a directory-like structure
-for storing data. Each 'directory' in this structure is referred to as a ZNode. This denotes the root ZNode, or 'directory',
-that should be used for storing data. The default value is `/root`. This is important to set correctly, as which cluster
-the NiFi instance attempts to join is determined by which ZooKeeper instance it connects to and the ZooKeeper Root Node
-that is specified.
 ** `nifi.cluster.flow.election.max.wait.time` - Specifies the amount of time to wait before electing a Flow as the "correct" Flow.
 If the number of Nodes that have voted is equal to the number specified by the `nifi.cluster.flow.election.max.candidates`
 property, the cluster will not wait this long. The default value is `5 mins`. Note that the time starts as soon as the first vote
@@ -2463,10 +2455,53 @@ is cast.
 of Flows. This allows the Nodes in the cluster to avoid having to wait a long time before starting processing if we reach
 at least this number of nodes in the cluster.
 
-Now, it is possible to start up the cluster. It does not matter which order the instances start up. Navigate to the URL for
-one of the nodes, and the User Interface should look similar to the following:
+=== ZooKeeper Clustering
+
+The following application properties support clustering with Apache ZooKeeper:
+
+* `nifi.cluster.leader.election.implementation`
+
+The Leader Election Implementation must be set to `CuratorLeaderElectionManager` for clustering with Apache ZooKeeper.
+The implementation defaults to ZooKeeper-based clustering when this property is not specified.
+
+* `nifi.zookeeper.connect.string`
+
+The Connect String that is needed to connect to Apache ZooKeeper. This is a comma-separated list
+of hostname:port pairs. For example, `localhost:2181,localhost:2182,localhost:2183`. This should contain a list of all ZooKeeper
+instances in the ZooKeeper quorum.
+
+* `nifi.zookeeper.root.node`
+
+The root ZNode that should be used in ZooKeeper. ZooKeeper provides a directory-like structure
+for storing data. Each 'directory' in this structure is referred to as a ZNode. This denotes the root ZNode, or 'directory',
+that should be used for storing data. The default value is `/root`. This is important to set correctly, as which cluster
+the NiFi instance attempts to join is determined by which ZooKeeper instance it connects to and the ZooKeeper Root Node
+that is specified.
+
+=== Kubernetes Clustering
+
+Kubernetes Clustering requires authorization to interact with Kubernetes Leases using
+the following link:https://kubernetes.io/docs/reference/access-authn-authz/authorization/#determine-the-request-verb[API request verbs]:
+
+- `create`
+- `get`
+- `update`
+
+The following application properties support clustering with Kubernetes:
 
-image:ncm.png["Clustered User Interface"]
+* `nifi.cluster.leader.election.implementation`
+
+The Leader Election Implementation must be set to `KubernetesLeaderElectionManager` for clustering with Kubernetes.
+The implementation creates and manages link:https://kubernetes.io/docs/concepts/architecture/leases/[Kubernetes Leases]
+for cluster coordination and primary node tracking.
+
+The service account under which NiFi is running must be granted the required permissions for successful cluster operation.
+
+* `nifi.cluster.leader.election.kubernetes.lease.prefix`
+
+The prefix string applied to Kubernetes Leases defaults to an empty string. Running Apache NiFi clusters in separate
+Kubernetes namespaces is the standard expectation with default application properties. Configuring a unique lease prefix per
+cluster is required when running multiple NiFi clusters in the same Kubernetes namespace to avoid conflicts on lease objects.
 
 [[cluster_firewall_configuration]]
 === Cluster Firewall Configuration
@@ -2555,8 +2590,29 @@ of the `property` that the State Provider supports. The textual content of the p
 Once these State Providers have been configured in the _state-management.xml_ file (or whatever file is configured), those Providers may be
 referenced by their identifiers.
 
+While there are not many properties that need to be configured for these providers, they were externalized into a separate _state-management.xml_
+file, rather than being configured via the _nifi.properties_ file, simply because different implementations may require different properties,
+and it is easier to maintain and understand the configuration in an XML-based file such as this, than to mix the properties of the Provider
+in with other NiFi framework-specific properties.
+
+It should be noted that if Processors and other components save state using the Clustered scope, the Local State Provider will be used
+if the instance is a standalone instance (not in a cluster) or is disconnected from the cluster. This also means that if a standalone instance
+is migrated to become a cluster, then that state will no longer be available, as the component will begin using the Clustered State Provider
+instead of the Local State Provider.
+
+If NiFi is configured to run in a standalone mode, the `cluster-provider` element need not be populated in the _state-management.xml_
+file and will actually be ignored if they are populated. However, the `local-provider` element must always be present and populated.
+Additionally, if NiFi is run in a cluster, each node must also have the `cluster-provider` element present and properly configured.
+Otherwise, NiFi will fail to startup.
+
+==== Local State Provider
+
 By default, the Local State Provider is configured to be a `WriteAheadLocalStateProvider` that persists the data to the
-`$NIFI_HOME/state/local` directory. The default Cluster State Provider is configured to be a `ZooKeeperStateProvider`. The default
+`$NIFI_HOME/state/local` directory.
+
+==== ZooKeeper Cluster State Provider
+
+The default Cluster State Provider is configured to be a `ZooKeeperStateProvider`. The default
 ZooKeeper-based provider must have its `Connect String` property populated before it can be used. It is also advisable, if multiple NiFi instances
 will use the same ZooKeeper instance, that the value of the `Root Node` property be changed. For instance, one might set the value to
 `/nifi/<team name>/production`. A `Connect String` takes the form of comma separated <host>:<port> tuples, such as
@@ -2569,21 +2625,32 @@ If `CreatorOnly` is specified, then only the user that created the data is allow
 In order to use the `CreatorOnly` option, NiFi must provide some form of authentication. See the <<zk_access_control>>
 section below for more information on how to configure authentication.
 
-If NiFi is configured to run in a standalone mode, the `cluster-provider` element need not be populated in the _state-management.xml_
-file and will actually be ignored if they are populated. However, the `local-provider` element must always be present and populated.
-Additionally, if NiFi is run in a cluster, each node must also have the `cluster-provider` element present and properly configured.
-Otherwise, NiFi will fail to startup.
+==== Kubernetes ConfigMap Cluster State Provider
 
-While there are not many properties that need to be configured for these providers, they were externalized into a separate _state-management.xml_
-file, rather than being configured via the _nifi.properties_ file, simply because different implementations may require different properties,
-and it is easier to maintain and understand the configuration in an XML-based file such as this, than to mix the properties of the Provider
-in with all of the other NiFi framework-specific properties.
+The Kubernetes ConfigMap State Provider supports shared cluster state when running in Kubernetes.
 
-It should be noted that if Processors and other components save state using the Clustered scope, the Local State Provider will be used
-if the instance is a standalone instance (not in a cluster) or is disconnected from the cluster. This also means that if a standalone instance
-is migrated to become a cluster, then that state will no longer be available, as the component will begin using the Clustered State Provider
-instead of the Local State Provider.
+The provider stores component state in
+link:https://kubernetes.io/docs/concepts/configuration/configmap/[Kubernetes ConfigMaps]
+and requires authorization to interact with ConfigMaps using
+the following link:https://kubernetes.io/docs/reference/access-authn-authz/authorization/#determine-the-request-verb[API request verbs]:
+
+- `create`
+- `delete`
+- `get`
+- `list`
+- `patch`
+- `update`
+
+As described in Kubernetes documentation, data stored in a ConfigMap is limited to 1 MB. Components that use cluster state must limit
+the amount of information stored.
+
+The Kubernetes ConfigMap State Provider supports the following configuration properties:
+
+- `ConfigMap Name Prefix`
 
+The prefix string applied to Kubernetes ConfigMap names defaults to an empty string. Running Apache NiFi clusters in separate
+Kubernetes namespaces is the standard expectation with default application properties. Configuring a unique ConfigMap prefix per
+cluster is required when running multiple NiFi clusters in the same Kubernetes namespace to avoid conflicts on ConfigMap objects.
 
 [[embedded_zookeeper]]
 === Embedded ZooKeeper Server
@@ -3994,6 +4061,9 @@ link:https://kubernetes.io/docs/concepts/architecture/leases/[Kubernetes Leases]
 will be read from the
 link:https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/[Service Account] namespace secret.
 The Kubernetes namespace will be set to `default` if the Service Account secret is not found.
+|`nifi.cluster.leader.election.kubernetes.lease.prefix`|The prefix string applied to Kubernetes Leases created
+for tracking cluster leader election. Configuring a prefix is necessary when running more than one
+Apache NiFi cluster in the same Kubernetes Namespace. The default value is blank.
 |`nifi.cluster.node.address`|The fully qualified address of the node. It is blank by default.
 |`nifi.cluster.node.protocol.port`|The node's protocol port. It is blank by default.
 |`nifi.cluster.node.protocol.max.threads`|The maximum number of threads that should be used to communicate with other nodes in the cluster. This property defaults to `50`. When a request is made to one node, it must be forwarded to the coordinator. The coordinator then replicates it to all nodes. There could be up to `n+2` threads for a given request, where `n` = number of nodes in your cluster. As an example, if 4 requests are made, a 5 node cluster will use `4 * 7 = 28` threads.