Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

Commit

Permalink
Merge branch 'master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
@cartonalexandre
cartonalexandre authored Apr 22, 2020
2 parents 289ffb3 + 521eb28 commit ffce42f
Show file tree
Hide file tree
Showing 31 changed files with 1,349 additions and 405 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ index.yaml
*.tgz
.idea/
/venv
.vscode/
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,5 @@ The charts are [currently tested](https://devops-ci.elastic.co/job/elastic+helm-

## Helm versions

While we are checking backward compatibility, the charts are only tested with Helm version mentioned in [helm-tester Dockerfile](helpers/helm-tester/Dockerfile) (currently 2.16.5).
While we are checking backward compatibility, the charts are only tested with Helm version mentioned in [helm-tester Dockerfile](helpers/helm-tester/Dockerfile) (currently 2.16.6).
Note that we don't support [Helm 3](https://v3.helm.sh/) version.
8 changes: 4 additions & 4 deletions elasticsearch/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,10 @@ helm install --name elasticsearch elastic/elasticsearch --set imageTag=7.6.2
| `esConfig` | Allows you to add any config files in `/usr/share/elasticsearch/config/` such as `elasticsearch.yml` and `log4j2.properties`. See [values.yaml](https://github.com/elastic/helm-charts/tree/master/elasticsearch/values.yaml) for an example of the formatting. | `{}` |
| `extraEnvs` | Extra [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config) which will be appended to the `env:` definition for the container | `[]` |
| `envFrom` | Templatable string of envFrom to be passed to the [environment from variables](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#configure-all-key-value-pairs-in-a-configmap-as-container-environment-variables) which will be appended to the `envFrom:` definition for the container | `[]` |
| `extraVolumes` | Templatable string of additional volumes to be passed to the `tpl` function | `""` |
| `extraVolumeMounts` | Templatable string of additional volumeMounts to be passed to the `tpl` function | `""` |
| `extraContainers` | Templatable string of additional containers to be passed to the `tpl` function | `""` |
| `extraInitContainers` | Templatable string of additional init containers to be passed to the `tpl` function | `""` |
| `extraVolumes` | Templatable string of additional volumes to be passed to the `tpl` function | `[]` |
| `extraVolumeMounts` | Templatable string of additional volumeMounts to be passed to the `tpl` function | `[]` |
| `extraContainers` | Templatable string of additional containers to be passed to the `tpl` function | `[]` |
| `extraInitContainers` | Templatable string of additional init containers to be passed to the `tpl` function | `[]` |
| `secretMounts` | Allows you easily mount a secret as a file inside the statefulset. Useful for mounting certificates and other secrets. See [values.yaml](https://github.com/elastic/helm-charts/tree/master/elasticsearch/values.yaml) for an example | `[]` |
| `image` | The Elasticsearch docker image | `docker.elastic.co/elasticsearch/elasticsearch` |
| `imageTag` | The Elasticsearch docker image tag | `7.6.2` |
Expand Down
2 changes: 1 addition & 1 deletion elasticsearch/examples/security/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ pull-elasticsearch-image:
secrets:
docker rm -f elastic-helm-charts-certs || true
rm -f elastic-certificates.p12 elastic-certificate.pem elastic-stack-ca.p12 || true
password=$$([ ! -z "$$ELASTIC_PASSWORD" ] && echo $$ELASTIC_PASSWORD || echo $$(docker run --rm $(ELASTICSEARCH_IMAGE) /bin/sh -c "< /dev/urandom tr -cd '[:alnum:]' | head -c20")) && \
password=$$([ ! -z "$$ELASTIC_PASSWORD" ] && echo $$ELASTIC_PASSWORD || echo $$(docker run --rm busybox:1.31.1 /bin/sh -c "< /dev/urandom tr -cd '[:alnum:]' | head -c20")) && \
docker run --name elastic-helm-charts-certs -i -w /app \
$(ELASTICSEARCH_IMAGE) \
/bin/sh -c " \
Expand Down
77 changes: 58 additions & 19 deletions elasticsearch/templates/statefulset.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,14 @@ spec:
{{- end }}
{{ end }}
{{- if .Values.extraVolumes }}
# Currently some extra blocks accept strings
# to continue with backwards compatibility this is being kept
# whilst also allowing for yaml to be specified too.
{{- if eq "string" (printf "%T" .Values.extraVolumes) }}
{{ tpl .Values.extraVolumes . | indent 8 }}
{{- else }}
{{ toYaml .Values.extraVolumes | indent 8 }}
{{- end }}
{{- end }}
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
Expand Down Expand Up @@ -184,7 +191,14 @@ spec:
{{- end }}
{{ end }}
{{- if .Values.extraInitContainers }}
# Currently some extra blocks accept strings
# to continue with backwards compatibility this is being kept
# whilst also allowing for yaml to be specified too.
{{- if eq "string" (printf "%T" .Values.extraInitContainers) }}
{{ tpl .Values.extraInitContainers . | indent 6 }}
{{- else }}
{{ toYaml .Values.extraInitContainers | indent 6 }}
{{- end }}
{{- end }}
containers:
- name: "{{ template "elasticsearch.name" . }}"
Expand All @@ -203,28 +217,39 @@ spec:
# Once it has started only check that the node itself is responding
START_FILE=/tmp/.es_start_file
http () {
local path="${1}"
if [ -n "${ELASTIC_USERNAME}" ] && [ -n "${ELASTIC_PASSWORD}" ]; then
BASIC_AUTH="-u ${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"
else
BASIC_AUTH=''
fi
curl -XGET -s -k --fail ${BASIC_AUTH} {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}${path}
}
if [ -n "${ELASTIC_USERNAME}" ] && [ -n "${ELASTIC_PASSWORD}" ]; then
BASIC_AUTH="-u ${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"
else
BASIC_AUTH=''
fi
if [ -f "${START_FILE}" ]; then
echo 'Elasticsearch is already running, lets check the node is healthy and there are master nodes available'
http "/_cluster/health?timeout=0s"
echo 'Elasticsearch is already running, lets check the node is healthy'
HTTP_CODE=$(curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}/)
RC=$?
if [[ ${RC} -ne 0 ]]; then
echo "curl -XGET -s -k \${BASIC_AUTH} -o /dev/null -w '%{http_code}' {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}/ failed with RC ${RC}"
exit ${RC}
fi
# ready if HTTP code 200, 503 is tolerable if ES version is 6.x
if [[ ${HTTP_CODE} == "200" ]]; then
exit 0
elif [[ ${HTTP_CODE} == "503" && "{{ include "elasticsearch.esMajorVersion" . }}" == "6" ]]; then
exit 0
else
echo "curl -XGET -s -k \${BASIC_AUTH} -o /dev/null -w '%{http_code}' {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}/ failed with HTTP code ${HTTP_CODE}"
exit 1
fi
else
echo 'Waiting for elasticsearch cluster to become ready (request params: "{{ .Values.clusterHealthCheckParams }}" )'
if http "/_cluster/health?{{ .Values.clusterHealthCheckParams }}" ; then
touch ${START_FILE}
exit 0
else
echo 'Cluster is not yet ready (request params: "{{ .Values.clusterHealthCheckParams }}" )'
exit 1
fi
echo 'Waiting for elasticsearch cluster to become ready (request params: "{{ .Values.clusterHealthCheckParams }}" )'
if curl -XGET -s -k --fail ${BASIC_AUTH} {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}/_cluster/health?{{ .Values.clusterHealthCheckParams }} ; then
touch ${START_FILE}
exit 0
else
echo 'Cluster is not yet ready (request params: "{{ .Values.clusterHealthCheckParams }}" )'
exit 1
fi
fi
{{ toYaml .Values.readinessProbe | indent 10 }}
ports:
Expand Down Expand Up @@ -295,7 +320,14 @@ spec:
subPath: {{ $path }}
{{- end -}}
{{- if .Values.extraVolumeMounts }}
# Currently some extra blocks accept strings
# to continue with backwards compatibility this is being kept
# whilst also allowing for yaml to be specified too.
{{- if eq "string" (printf "%T" .Values.extraVolumeMounts) }}
{{ tpl .Values.extraVolumeMounts . | indent 10 }}
{{- else }}
{{ toYaml .Values.extraVolumeMounts | indent 10 }}
{{- end }}
{{- end }}
{{- if .Values.masterTerminationFix }}
{{- if eq .Values.roles.master "true" }}
Expand Down Expand Up @@ -360,5 +392,12 @@ spec:
{{ toYaml .Values.lifecycle | indent 10 }}
{{- end }}
{{- if .Values.extraContainers }}
# Currently some extra blocks accept strings
# to continue with backwards compatibility this is being kept
# whilst also allowing for yaml to be specified too.
{{- if eq "string" (printf "%T" .Values.extraContainers) }}
{{ tpl .Values.extraContainers . | indent 6 }}
{{- else }}
{{ toYaml .Values.extraContainers | indent 6 }}
{{- end }}
{{- end }}
6 changes: 6 additions & 0 deletions elasticsearch/templates/test/test-elasticsearch-health.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ metadata:
annotations:
"helm.sh/hook": test-success
spec:
securityContext:
{{ toYaml .Values.podSecurityContext | indent 4 }}
containers:
- name: "{{ .Release.Name }}-{{ randAlpha 5 | lower }}-test"
image: "{{ .Values.image }}:{{ .Values.imageTag }}"
Expand All @@ -15,4 +17,8 @@ spec:
- |
#!/usr/bin/env bash -e
curl -XGET --fail '{{ template "elasticsearch.uname" . }}:{{ .Values.httpPort }}/_cluster/health?{{ .Values.clusterHealthCheckParams }}'
{{- if .Values.imagePullSecrets }}
imagePullSecrets:
{{ toYaml .Values.imagePullSecrets | indent 4 }}
{{- end }}
restartPolicy: Never
58 changes: 57 additions & 1 deletion elasticsearch/tests/elasticsearch_test.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,6 @@ def test_defaults():

assert "curl" in c["readinessProbe"]["exec"]["command"][-1]
assert "http://127.0.0.1:9200" in c["readinessProbe"]["exec"]["command"][-1]
assert "/_cluster/health?timeout=0s" in c["readinessProbe"]["exec"]["command"][-1]

# Resources
assert c["resources"] == {
Expand Down Expand Up @@ -320,6 +319,29 @@ def test_adding_a_extra_volume_with_volume_mount():
} in extraVolumeMounts


def test_adding_a_extra_volume_with_volume_mount_as_yaml():
config = """
extraVolumes:
- name: extras
emptyDir: {}
extraVolumeMounts:
- name: extras
mountPath: /usr/share/extras
readOnly: true
"""
r = helm_template(config)
extraVolume = r["statefulset"][uname]["spec"]["template"]["spec"]["volumes"]
assert {"name": "extras", "emptyDir": {}} in extraVolume
extraVolumeMounts = r["statefulset"][uname]["spec"]["template"]["spec"][
"containers"
][0]["volumeMounts"]
assert {
"name": "extras",
"mountPath": "/usr/share/extras",
"readOnly": True,
} in extraVolumeMounts


def test_adding_a_extra_container():
config = """
extraContainers: |
Expand All @@ -336,6 +358,22 @@ def test_adding_a_extra_container():
} in extraContainer


def test_adding_a_extra_container_as_yaml():
config = """
extraContainers:
- name: do-something
image: busybox
command: ['do', 'something']
"""
r = helm_template(config)
extraContainer = r["statefulset"][uname]["spec"]["template"]["spec"]["containers"]
assert {
"name": "do-something",
"image": "busybox",
"command": ["do", "something"],
} in extraContainer


def test_adding_a_extra_init_container():
config = """
extraInitContainers: |
Expand All @@ -354,6 +392,24 @@ def test_adding_a_extra_init_container():
} in extraInitContainer


def test_adding_a_extra_init_container_as_yaml():
config = """
extraInitContainers:
- name: do-something
image: busybox
command: ['do', 'something']
"""
r = helm_template(config)
extraInitContainer = r["statefulset"][uname]["spec"]["template"]["spec"][
"initContainers"
]
assert {
"name": "do-something",
"image": "busybox",
"command": ["do", "something"],
} in extraInitContainer


def test_sysctl_init_container_disabled():
config = """
sysctlInitContainer:
Expand Down
8 changes: 4 additions & 4 deletions elasticsearch/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -119,21 +119,21 @@ persistence:
enabled: true
annotations: {}

extraVolumes: ""
extraVolumes: []
# - name: extras
# emptyDir: {}

extraVolumeMounts: ""
extraVolumeMounts: []
# - name: extras
# mountPath: /usr/share/extras
# readOnly: true

extraContainers: ""
extraContainers: []
# - name: do-something
# image: busybox
# command: ['do', 'something']

extraInitContainers: ""
extraInitContainers: []
# - name: do-something
# image: busybox
# command: ['do', 'something']
Expand Down
13 changes: 13 additions & 0 deletions filebeat/examples/6.x/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,16 @@ imageTag: 6.8.8
extraEnvs:
- name: ELASTICSEARCH_HOSTS
value: six-master:9200

filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: docker
containers.ids:
- '*'
processors:
- add_kubernetes_metadata:
in_cluster: true
output.elasticsearch:
host: '${NODE_NAME}'
hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
13 changes: 8 additions & 5 deletions filebeat/examples/security/values.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: docker
containers.ids:
- '*'
- type: container
paths:
- /var/log/containers/*.log
processors:
- add_kubernetes_metadata:
in_cluster: true
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
output.elasticsearch:
username: '${ELASTICSEARCH_USERNAME}'
Expand Down
6 changes: 6 additions & 0 deletions filebeat/templates/daemonset.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@ spec:
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: varlog
hostPath:
path: /var/log
- name: varrundockersock
hostPath:
path: /var/run/docker.sock
Expand Down Expand Up @@ -152,6 +155,9 @@ spec:
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: varlog
mountPath: /var/log
readOnly: true
# Necessary when using autodiscovery; avoid mounting it otherwise
# See: https://www.elastic.co/guide/en/beats/filebeat/master/configuration-autodiscover.html
- name: varrundockersock
Expand Down
12 changes: 8 additions & 4 deletions filebeat/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,15 @@
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: docker
containers.ids:
- '*'
- type: container
paths:
- /var/log/containers/*.log
processors:
- add_kubernetes_metadata: ~
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
output.elasticsearch:
host: '${NODE_NAME}'
Expand Down
2 changes: 1 addition & 1 deletion helpers/helm-tester/Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
FROM python:3.7

ENV HELM_VERSION=2.16.5
ENV HELM_VERSION=2.16.6

RUN wget --no-verbose https://storage.googleapis.com/kubernetes-helm/helm-v${HELM_VERSION}-linux-amd64.tar.gz && \
tar xfv helm-v${HELM_VERSION}-linux-amd64.tar.gz && \
Expand Down
2 changes: 1 addition & 1 deletion helpers/terraform/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ FROM centos:7
ENV VAULT_VERSION 0.9.3
ENV TERRAFORM_VERSION=0.11.7
ENV KUBECTL_VERSION=1.15.4
ENV HELM_VERSION=2.16.5
ENV HELM_VERSION=2.16.6
ENV DOCKER_VERSION=18.09.7

RUN yum -y install \
Expand Down
Loading

0 comments on commit ffce42f

Please sign in to comment.