Update to Nifi 1.16.1

- Nifi update: 1.16.1
- Container images updates (busybox, alpine,...)
- README typos
- params.env examples

README.md

@@ -5,11 +5,11 @@
 Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
 Ref: <https://nifi.apache.org/>
 
-This repo provides tools and instructions for deploying and using Apache Nifi on OpenShift, witht the following characteristics:
+This repo provides tools and instructions for deploying and using Apache Nifi on OpenShift, with the following characteristics:
 
 * **UBI8-OpenJDK11 base images** for all NiFi components (main app, registry,toolkit).
 * Container images conform to OpenShift standards, especially **without requiring scc** (standard provided images had to run as AnyUID).
-* External Zookeer deployed automatically (Bitnami Zookeeper images are used).
+* External Zookeeper deployed automatically (Bitnami Zookeeper images are used).
 * Cluster-mode enabled, with a default to 3 NiFi nodes (1 to 5 nodes available as overlays).
 * Registry enabled to manage and version workflows.
 * **Secured Route** automatically created (Re-Encrypt base on self-signed NiFi certificate).
@@ -20,7 +20,7 @@ This repo provides tools and instructions for deploying and using Apache Nifi on
 Several components are deployed to cover as many use cases as possible:
 
 * Nifi CA: generates certificates for secured internal communication and authentication.
-* Nifi Registry: provides a central location for storage and management of shared resources across one or more instances of NiFi and/or MiNiFi. For exxample, allows workflow versioning and change tracking.
+* Nifi Registry: provides a central location for storage and management of shared resources across one or more instances of NiFi and/or MiNiFi. For example, allows workflow versioning and change tracking.
 * Zookeeper: an external zookeeper is deployed to allow flexibility and customization of the manifest.
 * NiFi: statefulset deployment, with several parameters available (see below).
 
@@ -30,11 +30,11 @@ When deployed, a secured Route is automatically created, with reencryption using
 
 There are tons of parameters that you can play with in Apache NiFi. Therefore, instead of providing an Helm chart and trying to expose and maintain all those parameters to cover every specific use cases, I chose to use Kustomize definitions and provide different base deployments for different authentication mechanisms and cluster sizes. Those "base examples" are then easy to modify for your own needs.
 
-Here are the steps to follow:
+Here are the steps to follow (from the base folder `deployment`):
 
-* Create the namespace where you want to deploy Apache Nifi.
-* From the `deployment` folder, edit the file `base/kustomization.yaml` and replace the `namespace` variable to match the name you want.
-* Edit the file `base/params.env` and fill in the information (description for each variable is in the main README).
+* Create the namespace where you want to deploy Apache Nifi (defaulted to `nifi` in the configuration files).
+* If you chose a namespace different than `nifi`, edit the file `base/kustomization.yaml` and replace the `namespace` variable to match this name.
+* Copy the file `base/params.env.example` to `base/params.env` and fill in the information (description for each variable below).
 * Switch to the `base` folder or to the **overlay** you want to use.
 * Use kustomize to build the deployment:
 
@@ -44,7 +44,7 @@ kustomize build -o /path/to/output/folder
 
 * From the build folder, do `oc apply -f .` to deploy Nifi!
 
-Alternative method: with a recent version of OpenShift (=>4.8), you can also do direcly `oc apply -k .` instead of the build + apply
+NOTE: with a recent version of OpenShift (=>4.8), you can also do directly `oc apply -k .` instead of the build + apply
 
 ## Parameters
 
@@ -56,7 +56,7 @@ The base configuration deploys a NiFi cluster with a SingleUser authentication m
 * `certificate_store_password`: the password used for the KeyStore and TrustStore (needed to create SSL contexts for components like listenHTTP)
 * `uid`: a uid that will be used by the root schema and base flow of the installation (you can easily generate one on <https://www.uuidgenerator.net/version4>).
 
-NOTE: don't modify the nodes_number directly in the `params.env` file! Use one of the overlays to modify the number of nodes.
+NOTE: don't modify the nodes_number directly in the `params.env` file! Use one of the overlays to modify the number of nodes (yeah, I know, that's a limitation of kustomize).
 
 ## Overlays
 
@@ -70,7 +70,7 @@ To enable OIDC authentication, you can use the overlay `oidc`.
 
 Supplemental parameters to change in `params.env`:
 
-* `oidc_discovery_url`: Full adress for the `.well-known/openid-configuration` standard discovery file. In case of using a provider running inside the cluster (e.g. a deployment of Keycloak), it has to be the full external secured Route).
+* `oidc_discovery_url`: Full address for the `.well-known/openid-configuration` standard discovery file. In case of using a provider running inside the cluster (e.g. a deployment of Keycloak), it has to be the full external secured Route).
 * `oidc_client_id`: Client ID from the OIDC configuration.
 * `oidc_client_secret`: Client Secret from the OIDC configuration
 * `oidc_claim_identifying_user`: the claim in the returned Token that will be used to identify users (like `email`). Specific to your OIDC provider.
@@ -169,7 +169,7 @@ The following properties are used. All the stores have the same password, set at
 * Keystore Password: the `certificate_store_password` parameter you set in the `params.env` file for deployment.
 * Key Password: the `certificate_store_password` parameter you set in the `params.env` file for deployment.
 * Keystore Type: `JKS`
-* Trustore Filename: `/opt/nifi/nifi-current/config-data/certs/truststore.jks`
+* Truststore Filename: `/opt/nifi/nifi-current/config-data/certs/truststore.jks`
 * Truststore Password: the `certificate_store_password` parameter you set in the `params.env` file for deployment.
-* Trustore Type: `JKS`
+* Truststore Type: `JKS`
 * TLS Protocol: `TLS`

containers/nifi-registry/DockerImage.txt

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-apache/nifi-registry:1.15.2
+apache/nifi-registry:1.16.1

containers/nifi-registry/Dockerfile

@@ -22,7 +22,7 @@ LABEL site="https://nifi.apache.org"
 
 ARG UID=1001
 ARG GID=0
-ARG NIFI_REGISTRY_VERSION=1.15.2
+ARG NIFI_REGISTRY_VERSION=1.16.1
 ARG MIRROR=https://archive.apache.org/dist
 
 ENV NIFI_REGISTRY_BASE_DIR /opt/nifi-registry

containers/nifi-registry/build.notes

@@ -1 +1 @@
-docker build --build-arg MIRROR=https://dlcdn.apache.org --build-arg NIFI_REGISTRY_BINARY_URL=nifi/1.15.2/nifi-registry-1.15.2-bin.tar.gz -t nifi-registry:1.15.2 .
+docker build --build-arg MIRROR=https://dlcdn.apache.org --build-arg NIFI_REGISTRY_BINARY_URL=nifi/1.16.1/nifi-registry-1.16.1-bin.tar.gz -t nifi-registry:1.16.1 .

containers/nifi-toolkit/Dockerfile

@@ -21,7 +21,7 @@ LABEL maintainer "Apache NiFi <dev@nifi.apache.org>"
 
 ARG UID=1001
 ARG GID=0
-ARG NIFI_TOOLKIT_VERSION=1.15.2
+ARG NIFI_TOOLKIT_VERSION=1.16.1
 ARG MIRROR=https://archive.apache.org/dist
 
 ENV NIFI_TOOLKIT_BASE_DIR=/opt/nifi-toolkit

containers/nifi-toolkit/build.notes

@@ -1 +1 @@
-docker build --build-arg MIRROR=https://dlcdn.apache.org/ -t nifi-toolkit:1.15.2 .
+docker build --build-arg MIRROR=https://dlcdn.apache.org/ -t nifi-toolkit:1.16.1 .

containers/nifi/DockerImage.txt

@@ -13,4 +13,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-nifi:1.15.2
+nifi:1.16.1

containers/nifi/Dockerfile

@@ -16,15 +16,15 @@
 # under the License.
 #
 ARG IMAGE_NAME=registry.access.redhat.com/ubi8/openjdk-11
-ARG IMAGE_TAG=1.10
+ARG IMAGE_TAG=1.13-1
 FROM ${IMAGE_NAME}:${IMAGE_TAG}
 ARG MAINTAINER="Apache NiFi <dev@nifi.apache.org>"
 LABEL maintainer="${MAINTAINER}"
 LABEL site="https://nifi.apache.org"
 
 ARG UID=1001
 ARG GID=0
-ARG NIFI_VERSION=1.15.2
+ARG NIFI_VERSION=1.16.1
 ARG BASE_URL=https://archive.apache.org/dist
 ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
 ARG DISTRO_PATH=${DISTRO_PATH:-${NIFI_VERSION}}

containers/nifi/README.md

@@ -40,5 +40,3 @@ NIFI_BASE_DIR=/opt/nifi
 ```
 docker run --rm --entrypoint /bin/bash apache/nifi:1.12.0 -c 'readlink /opt/nifi/nifi-1.12.0'                                   /opt/nifi/nifi-current
 ```
-
-

containers/nifi/build.notes

@@ -1 +1 @@
-./DockerBuild.sh 1001 0 https://dlcdn.apache.org/ https://dlcdn.apache.org/ 1.15.2
+./DockerBuild.sh 1001 0 https://dlcdn.apache.org/ https://dlcdn.apache.org/ 1.16.1

containers/nifi/pom.xml

@@ -26,7 +26,7 @@
         <docker.maintainer><![CDATA[Apache NiFi <dev@nifi.apache.org>]]></docker.maintainer>
         <!-- files need to exist at https://archive.apache.org/dist/nifi/${docker.nifi.version}/nifi-[toolkit-]${docker.nifi.version}-bin.zip
             so this version needs to be *before* the current ${project.version} -->
-        <docker.nifi.version>1.16.0</docker.nifi.version>
+        <docker.nifi.version>1.16.1</docker.nifi.version>
     </properties>
 
     <profiles>

deployment/base/kustomization.yaml

@@ -170,19 +170,19 @@ images:
     newTag: 1.35.0
   - name: alpine
     newName: alpine
-    newTag: 3.15.0
+    newTag: 3.16.0
   - name: nifi
     newName: quay.io/rh-data-services/nifi-openshift
-    newTag: 1.15.2
+    newTag: 1.16.1
   - name: nifi-registry
     newName: quay.io/rh-data-services/nifi-registry-openshift
-    newTag: 1.15.2
+    newTag: 1.16.1
   - name: nifi-toolkit
     newName: quay.io/rh-data-services/nifi-toolkit-openshift
-    newTag: 1.15.2
+    newTag: 1.16.1
   - name: zookeeper
     newName: bitnami/zookeeper
-    newTag: 3.7.0-debian-10-r262
+    newTag: 3.7.1
   - name: oc-client
     newName: quay.io/openshift/origin-cli
-    newTag: 4.9.0
+    newTag: 4.10.0

deployment/base/params.env.example-basic

@@ -0,0 +1,17 @@
+nodes_number=3
+storage_class=gp2
+admin_identity=admin
+admin_password=12CharacterOrPlusPassword
+certificate_store_password=trustmaster
+uid=3abd9dad-79c2-49a5-acac-3a4434600d44
+oidc_discovery_url=
+oidc_client_id=
+oidc_client_secret=
+oidc_claim_identifying_user=
+ldap_host=
+ldap_searchBase=
+ldap_admin=
+ldap_pass=
+ldap_searchFilter=
+ldap_userIdentityAttribute=
+ldap_userIdentityMapping=

deployment/base/params.env.example-oidc

@@ -0,0 +1,17 @@
+nodes_number=3
+storage_class=gp2
+admin_identity=admin@nifi.local
+admin_password=
+certificate_store_password=trustmaster
+uid=3abd9dad-79c2-49a5-acac-3a4434600d44
+oidc_discovery_url=https://keycloak-keycloak.cluster.address/auth/realms/opendatahub/.well-known/openid-configuration
+oidc_client_id=nifi-client
+oidc_client_secret=4443c959-ec4e-4686-baf9-fcecdacbe64a
+oidc_claim_identifying_user=email   
+ldap_host=
+ldap_searchBase=
+ldap_admin=
+ldap_pass=
+ldap_searchFilter=
+ldap_userIdentityAttribute=
+ldap_userIdentityMapping=