Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.
This repository has been archived by the owner on May 16, 2023. It is now read-only.

Elasticsearch fails readiness probe due to escaping issue #625

Closed

Description

The readiness probe for the elasticsearch StatefulSet is implemented without proper escaping:

In my case:

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: elastic
  namespace: monitoring
spec:
  chart: elasticsearch
  repo: https://helm.elastic.co
  targetNamespace: monitoring
  valuesContent: |
    antiAffinity: soft
    roles:
      master: "true"
      ingest: "true"
      data: "true"
    protocol: http
    esConfig:
      elasticsearch.yml: |
        xpack.security.enabled: true
        xpack.security.transport.ssl.enabled: false
        xpack.security.http.ssl.enabled: false
    extraEnvs:
      - name: ELASTIC_PASSWORD
        valueFrom:
          secretKeyRef:
            name: elastic-credentials
            key: password
      - name: ELASTIC_USERNAME
        valueFrom:
          secretKeyRef:
            name: elastic-credentials
            key: username
$ k get secrets elastic-credentials -o jsonpath={.data.password} | base64 -d
S(fMW|(iS6Z"k"@nRK(:

(don't worry, the password is a dummy one)

This is the line where I think the error is generated from:

BASIC_AUTH="-u ${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"

And this is the event log:

curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' http://127.0.0.1:9200/ failed with RC 7
8m34s       Warning   Unhealthy                         pod/elasticsearch-master-2                                          Readiness probe failed: Elasticsearch is already running, lets check the node is healthy
curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' http://127.0.0.1:9200/ failed with RC 7
8m33s       Warning   Unhealthy                         pod/elasticsearch-master-0                                          Readiness probe failed: Elasticsearch is already running, lets check the node is healthy
curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' http://127.0.0.1:9200/ failed with RC 7
4m33s       Warning   Unhealthy                         pod/elasticsearch-master-0                                          Readiness probe failed: Elasticsearch is already running, lets check the node is healthy
curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' http://127.0.0.1:9200/ failed with HTTP code 401
4m15s       Warning   Unhealthy                         pod/elasticsearch-master-1                                          Readiness probe failed: Elasticsearch is already running, lets check the node is healthy
curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' http://127.0.0.1:9200/ failed with HTTP code 401
4m14s       Warning   Unhealthy                         pod/elasticsearch-master-2   

I'll propose a fix ASAP


I stand corrected, simply quoting $BASIC_AUTH here should be enough:

HTTP_CODE=$(curl -XGET -s -k ${BASIC_AUTH} -o /dev/null -w '%{http_code}' {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}/)

The following curl calls will probably need the same treatment too:

if curl -XGET -s -k --fail ${BASIC_AUTH} {{ .Values.protocol }}://127.0.0.1:{{ .Values.httpPort }}/_cluster/health?{{ .Values.clusterHealthCheckParams }} ; then

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions