Skip to content

MLE-20181 update scanning logic to include other dependent images #323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
2fc77a1
Merge pull request #310 from marklogic/develop
pengzhouml Apr 4, 2025
6d6b51c
MLE-20917 update notice file for Helm 2.0.1 Release (#309)
pengzhouml Apr 4, 2025
e0cb821
update the license because of name change
Apr 7, 2025
4038439
MLE-20917 update docker version for security vulnerabilities fix
Apr 8, 2025
09cca44
Merge pull request #312 from marklogic/bugfix/MLE-20917-update-docker…
rwinieski Apr 10, 2025
3ca0eda
Change the Chart version to 2.1.0
Apr 14, 2025
833b8f8
update notice file
Apr 14, 2025
d3ca61e
update copyright infomation for license
Apr 14, 2025
93542e2
adjust modification copyright
Apr 14, 2025
f729731
MLE-21263: Remove PID for Logging redirection (#317)
pengzhouml Apr 25, 2025
22e7b16
MLE-21265: update liveness probe (#318)
pengzhouml May 1, 2025
d8d9664
MLE-21119 update copyright message with new format (#319)
pengzhouml May 1, 2025
6b29ad9
remove security attribute
rwinieski May 6, 2025
026d155
Merge branch 'release/2.1.0' into MLE-21334/Remove-attributes-from-co…
rwinieski May 19, 2025
bddd8d8
Merge pull request #320 from marklogic/MLE-21334/Remove-attributes-fr…
rwinieski May 19, 2025
cb761bf
MLE-21473 add missing namespace to script configmap
May 20, 2025
59f10f2
Add namesapce to service account
May 20, 2025
4fb34e7
update scanning logic to include other dependent images
May 20, 2025
10a873f
Merge pull request #322 from marklogic/bugfix/MLE-21473-cm-namespace
rwinieski May 21, 2025
ca74c8b
add HAP image in main values
rwinieski May 22, 2025
afed5b3
update Readme
rwinieski May 26, 2025
29e0f26
Merge pull request #324 from marklogic/MLE-21676/Define-HAProxy-Image
rwinieski May 26, 2025
cba408f
Update to handle new haproxy image location
May 27, 2025
6a78139
Merge branch 'release/2.1.0' into MLE-20181_scan-all-dependent-images
vitalykorolev May 27, 2025
31a33f6
fix syntax
May 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,7 @@ void imageScan() {
sh '''rm -f dep-image-scan.txt'''

// trigger BlackDuck scan
def rawImageList = readFile(file: 'helm_image.list').trim()
def imageList = rawImageList.endsWith(',') ? rawImageList[0..-2] : rawImageList
def imageList = readFile(file: 'helm_image.list').trim()
build job: 'securityscans/Blackduck/KubeNinjas/kubernetes-helm', wait: false, parameters: [ string(name: 'branch', value: "${env.BRANCH_NAME}"), string(name: 'CONTAINER_IMAGES', value: "${imageList}") ]
}

Expand Down
2 changes: 1 addition & 1 deletion LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Copyright © 2022-2025 MarkLogic Corporation.
Copyright © 2022-2025 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Expand Down
171 changes: 84 additions & 87 deletions NOTICE.txt

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,9 @@ Following table lists all the parameters supported by the latest MarkLogic Helm
| `logCollection.files.auditLogs` | Parameter to enable collection of MarkLogics audit logs when log collection is enabled | `true` |
| `logCollection.outputs` | Configure desired output for fluent-bit | `""` |
| `haproxy.enabled` | Parameter to enable the HAProxy Load Balancer for MarkLogic Server | `false` |
| `haproxy.image.repository` | Repository for HAProxy image | `haproxytech/haproxy-alpine` |
| `haproxy.image.tag` | Tag for HAProxy image | `2.9.4` |
| `haproxy.image.pullPolicy` | Haproxy iamge Pull Policy | `IfNotPresent` |
| `haproxy.existingConfigmap` | Name of an existing configmap with configuration for HAProxy | `marklogic-haproxy` |
| `haproxy.replicaCount` | Number of HAProxy Deployment | `2` |
| `haproxy.restartWhenUpgrade.enabled` | Automatically roll Deployments for every helm upgrade | `true` |
Expand Down
2 changes: 1 addition & 1 deletion charts/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ keywords:
sources:
- https://github.com/marklogic/marklogic-kubernetes
- https://www.marklogic.com/
version: 2.0.1
version: 2.1.0
2 changes: 1 addition & 1 deletion charts/charts/haproxy/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Modifications copyright © 2022-2025 MarkLogic Corporation.
Modifications copyright © 2022-2025 Progress Software Corporation and/or its subsidiaries or affiliates.
This file is modified from the original file to support the MarkLogic Kubernetes Helm Chart.
*/}}

Expand Down
2 changes: 1 addition & 1 deletion charts/charts/haproxy/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Modifications copyright © 2022-2025 MarkLogic Corporation.
Modifications copyright © 2022-2025 Progress Software Corporation and/or its subsidiaries or affiliates.
This file is modified from the original file to support the MarkLogic Kubernetes Helm Chart.
*/}}

Expand Down
2 changes: 1 addition & 1 deletion charts/charts/haproxy/templates/service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Modifications copyright © 2022-2025 MarkLogic Corporation.
Modifications copyright © 2022-2025 Progress Software Corporation and/or its subsidiaries or affiliates.
This file is modified from the original file to support the MarkLogic Kubernetes Helm Chart.
*/}}

Expand Down
2 changes: 1 addition & 1 deletion charts/charts/haproxy/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Modifications copyright © 2022-2025 MarkLogic Corporation.
# Modifications copyright © 2022-2025 Progress Software Corporation and/or its subsidiaries or affiliates.
# This file is modified from the original file to support the MarkLogic Kubernetes Helm Chart.

## Default values for HAProxy
Expand Down
16 changes: 8 additions & 8 deletions charts/templates/configmap-haproxy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ data:
balance leastconn
option forwardfor
http-request replace-path {{ $appservicespath }}(/)?(.*) /\2
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand All @@ -132,7 +132,7 @@ data:
balance leastconn
option forwardfor
http-request replace-path {{ $adminpath }}(/)?(.*) /\2
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand All @@ -152,7 +152,7 @@ data:
balance leastconn
option forwardfor
http-request replace-path {{ $managepath }}(/)?(.*) /\2
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand All @@ -177,7 +177,7 @@ data:
balance leastconn
option forwardfor
http-request replace-path {{$path}}(/)?(.*) /\2
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand Down Expand Up @@ -209,7 +209,7 @@ data:
mode http
balance leastconn
option forwardfor
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand Down Expand Up @@ -238,7 +238,7 @@ data:
mode http
balance leastconn
option forwardfor
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand Down Expand Up @@ -267,7 +267,7 @@ data:
mode http
balance leastconn
option forwardfor
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand Down Expand Up @@ -300,7 +300,7 @@ data:
mode http
balance leastconn
option forwardfor
cookie haproxy insert indirect httponly secure nocache maxidle 30m maxlife 4h
cookie haproxy insert indirect httponly nocache maxidle 30m maxlife 4h
stick-table type string len 32 size 10k expire 4h
stick store-response res.cook(HostId)
stick store-response res.cook(SessionId)
Expand Down
61 changes: 4 additions & 57 deletions charts/templates/configmap-scripts.yaml
Original file line number Diff line number Diff line change
@@ -1,48 +1,13 @@
# This configMap contains scirpts for MarkLogic Helm Chart:
# liveness-probe.sh
# copy-certs.sh
# prestop-hook.sh
# poststart-hook.sh
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "marklogic.fullname" . }}-scripts
namespace: {{ .Release.Namespace }}
data:
liveness-probe.sh: |
#!/bin/bash
pid=$(ps aux | grep "/bin/bash /usr/local/bin/start-marklogic.sh" | grep -v grep | awk '{print $2}')
log () {
local TIMESTAMP=$(date +"%Y-%m-%d %T.%3N")
# Check to make sure pod doesn't terminate if PID value is empty for any reason
if [ -n "$pid" ]; then
echo "${TIMESTAMP} $@" > /proc/$pid/fd/1
fi
}

# Check if ML service is running. Exit with 1 if it is other than running
if [ -e /etc/init.d/MarkLogic ]; then
ml_status=$(/etc/init.d/MarkLogic status)
else
ml_status=$(/etc/MarkLogic/MarkLogic-service.sh status)
fi

if [[ "$ml_status" =~ "running" ]]; then
http_code=$(curl -o /tmp/probe_response.txt -s -w "%{http_code}" "http://${HOSTNAME}:8001/admin/v1/timestamp")
curl_code=$?
http_resp=$(cat /tmp/probe_response.txt)

if [[ $curl_code -ne 0 && $http_code -ne 401 ]]; then
log "Info: [Liveness Probe] Error with MarkLogic"
log "Info: [Liveness Probe] Curl response code: "$curl_code
log "Info: [Liveness Probe] Http response code: "$http_code
log "Info: [Liveness Probe] Http response message: "$http_resp
fi
rm -f /tmp/probe_response.txt
exit 0
else
exit 1
fi

copy-certs.sh: |
#!/bin/bash
MARKLOGIC_ADMIN_USERNAME="$(< /run/secrets/ml-secrets/username)"
Expand Down Expand Up @@ -118,15 +83,9 @@ data:
MARKLOGIC_ADMIN_USERNAME="$(< /run/secrets/ml-secrets/username)"
MARKLOGIC_ADMIN_PASSWORD="$(< /run/secrets/ml-secrets/password)"

pid=$(ps aux | grep "/bin/bash /usr/local/bin/start-marklogic.sh" | grep -v grep | awk '{print $2}')

log () {
local TIMESTAMP=$(date +"%Y-%m-%d %T.%3N")
# Check to make sure pod doesn't terminate if PID value is empty for any reason
# If PID value is empty preStart hook logs are not recorded
if [ -n "$pid" ]; then
echo "${TIMESTAMP} $@" > /proc/$pid/fd/1
fi
echo "${TIMESTAMP} $@" > /proc/1/fd/1
}

log "Info: [prestop] Prestop Hook Execution"
Expand Down Expand Up @@ -195,8 +154,6 @@ data:
echo "IS_BOOTSTRAP_HOST false"
fi

pid=$(ps aux | grep "/bin/bash /usr/local/bin/start-marklogic.sh" | grep -v grep | awk '{print $2}')

###############################################################
# Logging utility
###############################################################
Expand All @@ -215,13 +172,8 @@ data:

log () {
local TIMESTAMP=$(date +"%Y-%m-%d %T.%3N")
# Check to make sure pod doesn't terminate if PID value is empty for any reason
# If PID value is empty postStart hook logs are not recorded
message="${TIMESTAMP} [postStart] $@"
if [ -n "$pid" ]; then
echo $message > /proc/$pid/fd/1
fi

echo $message > /proc/1/fd/1
echo $message >> /tmp/script.log
}

Expand Down Expand Up @@ -889,13 +841,8 @@ data:
#!/bin/bash
log () {
local TIMESTAMP=$(date +"%Y-%m-%d %T.%3N")
# Check to make sure pod doesn't terminate if PID value is empty for any reason
if [ -n "$pid" ]; then
echo "${TIMESTAMP} $@" > /proc/$pid/fd/1
fi
echo "${TIMESTAMP} $@" > /proc/1/fd/1
}

pid=$(ps aux | grep "/bin/bash /usr/local/bin/start-marklogic.sh" | grep -v grep | awk '{print $2}')

log "Info: [root-rootless-upgrade] Execution Start"

Expand Down
1 change: 1 addition & 0 deletions charts/templates/serviceaccount.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "marklogic.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
labels:
{{- include "marklogic.labels" . | nindent 4 }}
imagePullSecrets:
Expand Down
6 changes: 2 additions & 4 deletions charts/templates/statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -200,10 +200,8 @@ spec:
{{- end }}
{{- if .Values.livenessProbe.enabled }}
livenessProbe:
exec:
command:
- /bin/bash
- /tmp/helm-scripts/liveness-probe.sh
tcpSocket:
port: 8001
initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.livenessProbe.timeoutSeconds }}
Expand Down
4 changes: 4 additions & 0 deletions charts/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,10 @@ logCollection:
## It also support multi-statement transaction and ODBC connections.
haproxy:
enabled: false
image:
repository: haproxytech/haproxy-alpine
tag: "2.9.4"
pullPolicy: IfNotPresent

## Name of an existing configmap with configuration for HAProxy
existingConfigmap: marklogic-haproxy
Expand Down
31 changes: 21 additions & 10 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/marklogic/marklogic-kubernetes
go 1.23.6

require (
github.com/docker/docker v24.0.7+incompatible
github.com/docker/docker v28.0.4+incompatible
github.com/gruntwork-io/terratest v0.48.2
github.com/imroc/req/v3 v3.50.0
github.com/stretchr/testify v1.10.0
Expand All @@ -16,7 +16,6 @@ require (
github.com/Microsoft/go-winio v0.4.14 // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/cloudflare/circl v1.5.0 // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/emicklei/go-restful/v3 v3.11.2 // indirect
Expand Down Expand Up @@ -58,7 +57,7 @@ require (
github.com/xrash/smetrics v0.0.0-20231213231151-1d8dd44e695e // indirect
golang.org/x/exp v0.0.0-20241215155358-4a5509556b9e // indirect
golang.org/x/mod v0.22.0 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/sync v0.11.0 // indirect
golang.org/x/tools v0.28.0 // indirect
)

Expand Down Expand Up @@ -101,11 +100,15 @@ require (
github.com/aws/aws-sdk-go-v2/service/sts v1.33.1 // indirect
github.com/aws/smithy-go v1.22.1 // indirect
github.com/boombuler/barcode v1.0.1 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-errors/errors v1.5.1 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sql-driver/mysql v1.8.1 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
Expand All @@ -124,6 +127,7 @@ require (
github.com/json-iterator/go v1.1.12 // indirect
github.com/mattn/go-zglob v0.0.4 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/spdystream v0.2.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
Expand All @@ -133,15 +137,22 @@ require (
github.com/spf13/pflag v1.0.5 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/sdk v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.uber.org/mock v0.5.0 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/crypto v0.33.0 // indirect
golang.org/x/net v0.35.0 // indirect
golang.org/x/oauth2 v0.24.0 // indirect
golang.org/x/sys v0.29.0 // indirect
golang.org/x/term v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/sys v0.30.0 // indirect
golang.org/x/term v0.29.0 // indirect
golang.org/x/text v0.22.0 // indirect
golang.org/x/time v0.8.0 // indirect
google.golang.org/protobuf v1.35.1 // indirect
google.golang.org/protobuf v1.36.5 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand All @@ -153,4 +164,4 @@ require (
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)
)
Loading