feat(nifi): Add NiFi OPA Authorizer (#1058)

labrenbe · maltesander · NickLarsenNZ · web-flow · commit b87b5cc9843a · 2025-05-19T10:31:46.000Z
* build nifi-opa-plugin from source with workaround * add changelog entry * add patch * build nifi-opa-plugin from branch feat/reworked-opa-response * Update nifi/Dockerfile Co-authored-by: Malte Sander <malte.sander.it@gmail.com> * chore(⏲): Use fast download mirrors (#1061) * chore: Use fast download mirrors * chore: Show download progress bar * chore: Improve gpg key warning message * chore: Show upload progress bar Thanks to <https://bashupload.com/how_to_upload_progress_curl> * Apply suggestions from code review Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech> * chore: Make output more consistent --------- Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech> * chore(⏲): Add missing tool update tasks to templates (#1062) * chore(issue_templates): Use YY.M.X placeholders * chore(issue_templates): Remove tasklists :sob: * chore(issue_templates): Add missing tool update tasks for cyclonedx and auditable * feat: move patch apply logic to patchable (#1032) * wip * Update druid/Dockerfile Co-authored-by: Natalie Klestrup Röijezon <nat@nullable.se> * fix: remove unnecessary check / shadow repo root var * fix: druid src path * fix: druid src path * feat: introduce stackable-devel image * fix: use PathBuf in ProductVersionContext * chore: align zookeeper patch directory structure * fix: stackable-devel dnf and shell config * chore: switch patch process in other products * fix: hive build * fix: trino build * fix: spark build * chore: make hadolint happy * fix: remove hbase intermediate sources / remove unnecessary and operator * fix: permissions in patchable build process * chore: remove unnecessary curl command in build process * chore: move adding of JMX config and start-metastore script from builder stage to final stage * chore: remove git repo in trino and hbase-operator-tools to avoid maven commit plugin bug --------- Co-authored-by: Natalie Klestrup Röijezon <nat@nullable.se> * fix: add missing patchable config for Kafka 3.8.0 (#1065) * fix(pr-template): Fix the rendering of the list (#1073) * feat: make image namespace a workflow input (default sdp) (#1072) * feat: make image namespace a workflow input (default sdp) * feat: add registry namespace input to build workflows and remove default * ci(mirror): Include image name and version in run-name (#1089) * ci(mirror): Include image and version in run-name * chore: Use colon * fix: spark connect client Harbor credentials (#1088) * fix: spark connect client Harbor credentials (#1092) * feat: connect client image includes JupyterLab (#1071) * feat: install demo dependencies * spark-connect-client is now built directly off of spark-k8s * run pre-commit hooks * fix shellcheck sc2102 * chore: Bump tools ahead of 25.7.0 (#1090) * chore(jmx_exporter): Bump products to use 1.2.0 * chore(tools): Bump kubectl (1.33.0 and yq (4.45.2) * chore(cyclonedx-bom): Bump to 6.0.0 * chore: Update changelog * chore: Changelog formatting * chore(java): Add JDK 24 (#1097) * chore(java): Add JDK 24 * chore: Update changelog * chore(vector): Bump to 0.46.1 (#1098) * chore(vector): Bump to 0.46.1 * chore(vector): Bump products to use 0.46.1 * chore(zookeeper): Remove 3.9.2 (#1093) * chore: Changelog formatting * chore(zookeeper): Remove 2.9.2 * chore: Update changelog * chore: Update changelog * chore(ubi-rust-builders): Update container images ahead of Stackable Release 25.7.0 (#1091) * chore(ubi-rust-builders): Remove ubi8-rust-builder * chore(ubi-rust-builders): Update base image and use protoc 30.2 * Apply suggestions from code review Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech> Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de> --------- Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech> Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de> * chore(opa): Update versions ahead of 25.7.0 (#1103) * chore(opa): Remove 0.67.1 * chore(opa): Remove legacy bundle-builder * chore(opa): Bump ubi9 base image * chore(opa): Add 1.4.2 * fix(opa): Manually install Go version NOTE: The dnf package was 1.23.6 and OPA required 1.23.8 NOTE: I tried making the version paramaterized, but bake wouldn't allow it (it worked fine with `docker build` and `docker build xbuild`). * ci(mirror): Add golang * chore(opa): Use mirrored golang image * chore(opa): Parameterise golang container version * chore: Update changelog * fix: Bump vector to 0.46.1 for java-base:24 (#1104) * chore(superset): Add 4.1.2 (#1102) * chore(superset): Add 4.1.2 * fix(superset): Remove invalid parts of the file * chore: Update changelog * feat(nifi): Add nifi-iceberg-bundle (#1060) * feat(nifi): Add nifi-iceberg-bundle * changelog * Add SBOM to final image * Use version 0.0.1 * hadolint * Bump to 0.0.2 * Bump to 0.0.3 * fix(nifi): Delete correct intermediate folder (#1106) * fix(nifi): Delete correct intermediate folder * changelog * fix Dockerfile * build nifi opa plugin from tag * Update CHANGELOG.md Co-authored-by: Malte Sander <malte.sander.it@gmail.com> * fix hadolint * Update nifi/Dockerfile Co-authored-by: Malte Sander <malte.sander.it@gmail.com> --------- Co-authored-by: Malte Sander <malte.sander.it@gmail.com> Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com> Co-authored-by: Techassi <sascha.lautenschlaeger@stackable.tech> Co-authored-by: Lukas Krug <lukas.voetmand@stackable.tech> Co-authored-by: Natalie Klestrup Röijezon <nat@nullable.se> Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com> Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -38,6 +38,8 @@ All notable changes to this project will be documented in this file.
   `check-permissions-ownership.sh` provided in stackable-base image ([#1025]).
 - zookeeper: check for correct permissions and ownerships in /stackable folder via
   `check-permissions-ownership.sh` provided in stackable-base image ([#1043]).
+- nifi: Build and add OPA authorizer plugin nar ([#1058]).
+- nifi: Add [nifi-iceberg-bundle](https://github.com/stackabletech/nifi-iceberg-bundle) for NiFi `2.2.0` ([#1060], [#1106]).
 - java: Add JDK 24 ([#1097]).
 - ci: Add golang image to mirror workflow ([#1103]).
 - omid: bump version to 1.1.3 ([#1105])
@@ -111,6 +113,7 @@ All notable changes to this project will be documented in this file.
 [#1054]: https://github.com/stackabletech/docker-images/pull/1054
 [#1055]: https://github.com/stackabletech/docker-images/pull/1055
 [#1056]: https://github.com/stackabletech/docker-images/pull/1056
+[#1058]: https://github.com/stackabletech/docker-images/pull/1058
 [#1060]: https://github.com/stackabletech/docker-images/pull/1060
 [#1090]: https://github.com/stackabletech/docker-images/pull/1090
 [#1091]: https://github.com/stackabletech/docker-images/pull/1091
diff --git a/nifi/Dockerfile b/nifi/Dockerfile
@@ -97,7 +97,7 @@ mkdir -p /stackable
 
 # NiFI 1.x natively supports Iceberg, no need to build an iceberg-bundle for it
 if [[ "${PRODUCT}" != 1.* ]] ; then
-    curl "https://github.com/stackabletech/nifi-iceberg-bundle/archive/refs/tags/${NIFI_ICEBERG_BUNDLE}.tar.gz" | tar -xzC .
+    curl -L "https://github.com/stackabletech/nifi-iceberg-bundle/archive/refs/tags/${NIFI_ICEBERG_BUNDLE}.tar.gz" | tar -xzC .
     cd nifi-iceberg-bundle-${NIFI_ICEBERG_BUNDLE} || exit
 
     sed -i -e "s/{{ NIFI_VERSION }}/${PRODUCT}/g" pom.xml
@@ -126,6 +126,35 @@ chmod g=u /stackable/*.cdx.json
 fi
 EOF
 
+FROM stackable/image/java-devel AS opa-authorizer-builder
+
+ARG NIFI_OPA_AUTHORIZER_PLUGIN
+ARG STACKABLE_USER_UID
+ARG PRODUCT
+
+USER ${STACKABLE_USER_UID}
+WORKDIR /build
+
+RUN <<EOF
+mkdir -p /stackable
+
+curl -L "https://github.com/DavidGitter/nifi-opa-plugin/archive/refs/tags/v${NIFI_OPA_AUTHORIZER_PLUGIN}.tar.gz" | tar -xzC .
+cd nifi-opa-plugin-${NIFI_OPA_AUTHORIZER_PLUGIN}/authorizer || exit
+
+mvn \
+    --batch-mode \
+    --no-transfer-progress \
+    clean package \
+    -DskipTests \
+    -Pnifi-${PRODUCT}
+
+cp ./target/opa-authorizer.nar /stackable/opa-authorizer.nar
+cp ../LICENSE /stackable/LICENSE
+
+# Set correct permissions
+chmod g=u /stackable/opa-authorizer.nar
+EOF
+
 FROM stackable/image/java-base AS final
 
 ARG PRODUCT
@@ -146,6 +175,8 @@ COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-iceberg-bundle-builder /stackab
 COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-iceberg-bundle-builder /stackable/*.cdx.json /stackable/nifi-${PRODUCT}/lib/
 COPY --chown=${STACKABLE_USER_UID}:0 --from=nifi-builder /stackable/git-sync /stackable/git-sync
 
+COPY --chown=${STACKABLE_USER_UID}:0 --from=opa-authorizer-builder /stackable/opa-authorizer.nar /stackable/nifi-${PRODUCT}/extensions/opa-authorizer.nar
+COPY --chown=${STACKABLE_USER_UID}:0 --from=opa-authorizer-builder /stackable/LICENSE /licenses/NIFI_OPA_PLUGIN_LICENSE
 COPY --chown=${STACKABLE_USER_UID}:0 nifi/stackable/bin /stackable/bin
 COPY --chown=${STACKABLE_USER_UID}:0 nifi/licenses /licenses
 COPY --chown=${STACKABLE_USER_UID}:0 nifi/python /stackable/python
diff --git a/nifi/versions.py b/nifi/versions.py
@@ -4,18 +4,21 @@
         "java-base": "11",
         "java-devel": "11",  # There is an error when trying to use the jdk 21 (since nifi 1.26.0)
         "git_sync": "v4.4.0",
+        "nifi_opa_authorizer_plugin": "0.1.0",
     },
     {
         "product": "1.28.1",
         "java-base": "11",
         "java-devel": "11",
         "git_sync": "v4.4.0",
+        "nifi_opa_authorizer_plugin": "0.1.0",
     },
     {
         "product": "2.4.0",
         "java-base": "21",
         "java-devel": "21",
         "git_sync": "v4.4.0",
         "nifi_iceberg_bundle": "0.0.4",
+        "nifi_opa_authorizer_plugin": "0.1.0",
     },
 ]
