Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

FAQ and examples improvements #598

Merged
merged 15 commits into from
Apr 28, 2020
Next Next commit
[elasticsearch] add readme for examples
  • Loading branch information
jmlrt committed Apr 28, 2020
commit 93a07e4ae3ef83273a0679ae2a2ef731100d2b39
209 changes: 76 additions & 133 deletions elasticsearch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,23 +13,17 @@
- [Migration from helm/charts stable](#migration-from-helmcharts-stable)
- [Configuration](#configuration)
- [Deprecated](#deprecated)
- [Try it out](#try-it-out)
- [Default](#default)
- [Multi](#multi)
- [Security](#security)
- [FAQ](#faq)
- [How to deploy this chart on a specific K8S distribution?](#how-to-deploy-this-chart-on-a-specific-k8s-distribution)
- [How to deploy dedicated nodes types?](#how-to-deploy-dedicated-nodes-types)
- [Clustering and Node Discovery](#clustering-and-node-discovery)
- [How to deploy clusters with security (authentication and TLS) enabled?](#how-to-deploy-clusters-with-security-authentication-and-tls-enabled)
- [How to install plugins?](#how-to-install-plugins)
- [How to use the keystore?](#how-to-use-the-keystore)
- [Basic example](#basic-example)
- [Multiple keys](#multiple-keys)
- [Custom paths and keys](#custom-paths-and-keys)
- [How to enable snapshotting?](#how-to-enable-snapshotting)
- [Local development environments](#local-development-environments)
- [Minikube](#minikube)
- [Docker for Mac - Kubernetes](#docker-for-mac---kubernetes)
- [KIND - Kubernetes](#kind---kubernetes)
- [MicroK8S](#microk8s)
- [Clustering and Node Discovery](#clustering-and-node-discovery)
- [Contributing](#contributing)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
Expand Down Expand Up @@ -124,12 +118,6 @@ inside of the container. Doing this makes it much easier for this chart to
support multiple versions with minimal changes.


## Migration from helm/charts stable

If you currently have a cluster deployed with the [helm/charts stable][] chart
you can follow the [migration guide][].


## Configuration

| Parameter | Description | Default |
Expand Down Expand Up @@ -207,47 +195,80 @@ you can follow the [migration guide][].
| `fsGroup` | The Group ID (GID) for [securityContext][] so that the Elasticsearch user can read from the persistent volume | `""` |


## Try it out
## FAQ

In [examples][] you will find some example configurations. These examples are
used for the automated testing of this Helm chart.
### How to deploy this chart on a specific K8S distribution?

### Default
This chart is designed to run on production scale Kubernetes clusters with
multiple nodes, lots of memory and persistent storage. For that reason it can be
a bit tricky to run them against local Kubernetes environments such as
[Minikube][].

To deploy a cluster with all default values and run the integration tests:
This chart is highly tested with [GKE][], but some K8S distribution also
requires specific configurations.

```
cd examples/default
make
```
We provide examples of configuration for the following K8S providers:

### Multi
- [Docker for Mac][]
- [KIND][]
- [Minikube][]
- [MicroK8S][]
- [OpenShift][]

A cluster with dedicated node types:
### How to deploy dedicated nodes types?

```
cd examples/multi
make
```
All the Elasticsearch pods deployed share the same configuration. If you need to
deploy dedicated [nodes types][] (for example dedicated master and data nodes),
you can deploy multiple releases of this chart with different configurations
while they share the same `clusterName` value.

### Security
For each Helm release, the nodes types can then be defined using `roles` value.

A cluster with node to node security and https enabled. This example uses
autogenerated certificates and password, for a production deployment you want to
generate SSL certificates following the [official docs][node-certificates].
An example of Elasticsearch cluster using 2 different Helm releases for master
and data nodes can be found in [examples/multi][].

Generate the certificates and install Elasticsearch:
#### Clustering and Node Discovery

```
cd examples/security
make
This chart facilitates Elasticsearch node discovery and services by creating two
`Service` definitions in Kubernetes, one with the name `$clusterName-$nodeGroup`
and another named `$clusterName-$nodeGroup-headless`.
Only `Ready` pods are a part of the `$clusterName-$nodeGroup` service, while all
pods ( `Ready` or not) are a part of `$clusterName-$nodeGroup-headless`.

# Run a curl command to interact with the cluster
kubectl exec -ti security-master-0 -- sh -c 'curl -u $ELASTIC_USERNAME:$ELASTIC_PASSWORD -k https://localhost:9200/_cluster/health?pretty'
```
If your group of master nodes has the default `nodeGroup: master` then you can
just add new groups of nodes with a different `nodeGroup` and they will
automatically discover the correct master. If your master nodes have a different
`nodeGroup` name then you will need to set `masterService` to
`$clusterName-$masterNodeGroup`.

The chart value for `masterService` is used to populate
`discovery.zen.ping.unicast.hosts` , which Elasticsearch nodes will use to
contact master nodes and form a cluster.
Therefore, to add a group of nodes to an existing cluster, setting
`masterService` to the desired `Service` name of the related cluster is
sufficient.

## FAQ
### How to deploy clusters with security (authentication and TLS) enabled?

This Helm chart can use existing [Kubernetes secrets][] to setup
credentials or certificates for examples. These secrets should be created
outside of this chart and accessed using [environment variables][] and volumes.

An example of Elasticsearch cluster using security can be found in
[examples/security][].

### How to migrate from helm/charts stable chart?

If you currently have a cluster deployed with the [helm/charts stable][] chart
you can follow the [migration guide][].

### How to install OSS version of Elasticsearch

Deploying OSS version of Elasticsearch can be done by setting `image` value to
[Elasticsearch OSS Docker image][]

An example of Elasticsearch cluster using OSS version can be found in
[examples/oss][].

### How to install plugins?

Expand Down Expand Up @@ -347,92 +368,6 @@ there are plans to have Elasticsearch manage automated snapshots with
[Snapshot Lifecycle Management][].


## Local development environments

This chart is designed to run on production scale Kubernetes clusters with
multiple nodes, lots of memory and persistent storage. For that reason it can be
a bit tricky to run them against local Kubernetes environments such as minikube.
Below are some examples of how to get this working locally.

### Minikube

This chart also works successfully on [minikube][] in addition to typical hosted
Kubernetes environments. An example `values.yaml` file for minikube is provided
under `examples/`.

In order to properly support the required persistent volume claims for the
Elasticsearch `StatefulSet` , the `default-storageclass` and
`storage-provisioner` minikube addons must be enabled:

```
minikube addons enable default-storageclass
minikube addons enable storage-provisioner
cd examples/minikube
make
```

Note that if `helm` or `kubectl` timeouts occur, you may consider creating a
minikube VM with more CPU cores or memory allocated.

### Docker for Mac - Kubernetes

It is also possible to run this chart with the built in Kubernetes cluster that
comes with [docker-for-mac][]:

```
cd examples/docker-for-mac
make
```

### KIND - Kubernetes

It is also possible to run this chart using a Kubernetes [KIND][] (Kubernetes in
Docker) cluster:

```
cd examples/kubernetes-kind
make
```

### MicroK8S

It is also possible to run this chart using [MicroK8S][]:

```
microk8s.enable dns
microk8s.enable helm
microk8s.enable storage
cd examples/microk8s
make
```


## Clustering and Node Discovery

This chart facilitates Elasticsearch node discovery and services by creating two
`Service` definitions in Kubernetes, one with the name `$clusterName-$nodeGroup`
and another named `$clusterName-$nodeGroup-headless`.
Only `Ready` pods are a part of the `$clusterName-$nodeGroup` service, while all
pods ( `Ready` or not) are a part of `$clusterName-$nodeGroup-headless`.

If your group of master nodes has the default `nodeGroup: master` then you can
just add new groups of nodes with a different `nodeGroup` and they will
automatically discover the correct master. If your master nodes have a different
`nodeGroup` name then you will need to set `masterService` to
`$clusterName-$masterNodeGroup`.

The chart value for `masterService` is used to populate
`discovery.zen.ping.unicast.hosts` , which Elasticsearch nodes will use to
contact master nodes and form a cluster.
Therefore, to add a group of nodes to an existing cluster, setting
`masterService` to the desired `Service` name of the related cluster is
sufficient.

For an example of deploying both a group master nodes and data nodes using
multiple releases of this chart, see the accompanying values files in
`examples/multi`.


## Contributing

Please check [CONTRIBUTING.md][] before any contribution or for any questions
Expand All @@ -453,11 +388,16 @@ about our development and testing process.
[custom docker image]: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_c_customized_image
[deploys statefulsets serially]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies
[discovery.zen.minimum_master_nodes]: https://www.elastic.co/guide/en/elasticsearch/reference/current/discovery-settings.html#minimum_master_nodes
[docker-for-mac]: https://docs.docker.com/docker-for-mac/kubernetes/
[docker for mac]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/docker-for-mac
[elasticsearch cluster health status params]: https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html#request-params
[elasticsearch docker image]: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
[elasticsearch oss docker image]: https://www.docker.elastic.co/#elasticsearch-7-6-2-oss
[environment variables]: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/#using-environment-variables-inside-of-your-config
[examples]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/
[examples/multi]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/multi
[examples/oss]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/oss
[examples/security]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/security
[gke]: https://cloud.google.com/kubernetes-engine
[helm]: https://helm.sh
[helm/charts stable]: https://github.com/helm/charts/tree/master/stable/elasticsearch/
[how to install plugins guide]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/README.md#how-to-install-plugins
Expand All @@ -468,20 +408,23 @@ about our development and testing process.
[ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/
[java options]: https://www.elastic.co/guide/en/elasticsearch/reference/current/jvm-options.html
[jvm heap size]: https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
[kind]: https://github.com/kubernetes-sigs/kind
[kind]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/kubernetes-kind
[kubernetes secrets]: https://kubernetes.io/docs/concepts/configuration/secret/
[labels]: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
[loadBalancer annotations]: https://kubernetes.io/docs/concepts/services-networking/service/#ssl-support-on-aws
[loadBalancer]: https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer
[maxUnavailable]: https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
[migration guide]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/migration/README.md
[minikube]: https://kubernetes.io/docs/setup/minikube/
[microk8s]: https://microk8s.io
[minikube]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/minikube
[microk8s]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/microk8s
[multi]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/multi/
[network.host elasticsearch setting]: https://www.elastic.co/guide/en/elasticsearch/reference/current/network.host.html
[node affinity settings]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature
[node-certificates]: https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates
[nodePort]: https://kubernetes.io/docs/concepts/services-networking/service/#nodeport
[nodes types]: https://www.elastic.co/guide/en/elasticsearch/reference/7.6/modules-node.html
[nodeSelector]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
[openshift]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/openshift
[parent readme]: https://github.com/elastic/helm-charts/tree/master/README.md
[priorityClass]: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass
[probe]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
Expand Down
23 changes: 23 additions & 0 deletions elasticsearch/examples/6.x/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# 6.x

This example deploy a 3 nodes Elasticsearch 6.8.8 cluster.


## Usage

* Deploy Elasticsearch chart with the default values: `make install`

* You can now setup a port forward to query Elasticsearch API:

```
kubectl port-forward svc/six-master 9200
curl localhost:9200/_cat/indices
```


## Testing

You can also run [goss integration tests][] using `make test`


[goss integration tests]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/6.x/test/goss.yaml
26 changes: 25 additions & 1 deletion elasticsearch/examples/config/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
# Config

An example testing suite for testing some of the optional features of this chart.
This example deploy a single node Elasticsearch 7.6.2 with authentication and
custom [values][].


## Usage

* Create the required secrets: `make secrets`

* Deploy Elasticsearch chart with the default values: `make install`

* You can now setup a port forward to query Elasticsearch API:

```
kubectl port-forward svc/config-master 9200
curl -u elastic:changeme http://localhost:9200/_cat/indices
```


## Testing

You can also run [goss integration tests][] using `make test`


[goss integration tests]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/config/test/goss.yaml
[values]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/config/values.yaml
25 changes: 25 additions & 0 deletions elasticsearch/examples/default/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Default

This example deploy a 3 nodes Elasticsearch 7.6.2 cluster using
[default values][].


## Usage

* Deploy Elasticsearch chart with the default values: `make install`

* You can now setup a port forward to query Elasticsearch API:

```
kubectl port-forward svc/elasticsearch-master 9200
curl localhost:9200/_cat/indices
```


## Testing

You can also run [goss integration tests][] using `make test`


[goss integration tests]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/default/test/goss.yaml
[default values]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/values.yaml
23 changes: 23 additions & 0 deletions elasticsearch/examples/docker-for-mac/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Docker for Mac

This example deploy a 3 nodes Elasticsearch 7.6.2 cluster on [Docker for Mac][]
using [custom values][].

Note that this configuration should be used for test only and isn't recommended
for production.


## Usage

* Deploy Elasticsearch chart with the default values: `make install`

* You can now setup a port forward to query Elasticsearch API:

```
kubectl port-forward svc/elasticsearch-master 9200
curl localhost:9200/_cat/indices
```


[custom values]: https://github.com/elastic/helm-charts/tree/master/elasticsearch/examples/docker-for-mac/values.yaml
[docker for mac]: https://docs.docker.com/docker-for-mac/kubernetes/
Loading