Add metricbeat chart #127

Crazybus · 2019-05-09T07:02:47Z

The first commit is a copy paste of the filebeat chart. If you want to see all of the changes after the s/filebeat/metricbeat you can look at: https://github.com/elastic/helm-charts/pull/127/files/f8aa4bfaa972ac72abb017acb484ffbf457295d2..8bc136528dfb40ed3fd563edf9322708e6873447

Couple of notes:

I don't really like the add/remove charts/ dance in .helmignore however I had to add it because helm lint --strict will fail because of linting problems with kube-state-metrics child chart. And leaving it in .helmignore will mean that helm can't find the child chart :(
I made the decision for the metricbeat that uses kube-state-metrics to re-use the configmaps, secrets and extra envs from the daemonset. I did this because these secrets are always being used for the output to Elasticsearch. I couldn't imagine a situation where you would want to add a secret to one of them, but not the other. This simplifies the chart and removes a lot of duplication. The real reason we run a seperate deployment is so that we don't have each daemonset pod sending exactly the same information from the Kubernetes API. Since we theoretically could move the kube-state-metrics logic into the daemonset config (with the downside that you would have duplicated data and extra queries to the Kubernetes API) I can't see this causing any issues.
I added in a couple of tests to make sure that the correct data really appears in Elasticsearch. This was to be sure that the daemonset metricbeat data and the deployment were making it in properly.

This is being added to make sure that automated testing for the WIP metricbeat chart is happening in #127

jmlrt

first diff review between filebeat and metribeat charts LGTM

This is being added to make sure that automated testing for the WIP metricbeat chart is happening in elastic#127

Right now this is literally the filebeat chart with a s/filebeat/metricbeat run over it and the default metricbeat.yml changed. Since it was already working in this state I'm opening up a PR so that it is easier to see the changes made versus filebeat and already get CI testing it.

`helm lint` also tries to lint subcharts. So the makefile now adds `charts/` to the .helmignore before running the linting then adds it back when adding the dependency.

Crazybus · 2019-07-01T13:04:36Z

Just realised the CI changes won't work in this PR since they are only deployed from master. I just deployed the changes from this branch to devops-ci.

jenkins test this please

Crazybus · 2019-07-05T06:15:59Z

jenkins test this please

tylerjl

Chart and all the caveats you mention LGTM - I agree about firing some ES queries to confirm data is flowing as expected; I think the queries are generic enough that they shouldn't be brittle or problematic between versions. 👍

* bump helm version to helm 3.4.1 * update doc to make helm 3 the recommended version * update helm commands arguments for helm 3 * remove helm init and tiller install commands * add stable repo for metricbeat dependencies * remove the charts dance in helmignore (related to #127) * optimize docker images * replace deprecated gitversion capability (related to helm/helm#5601) * remove some heritage labels in immutable fields

* bump helm version to helm 3.4.1 * update doc to make helm 3 the recommended version * update helm commands arguments for helm 3 * remove helm init and tiller install commands * add stable repo for metricbeat dependencies * remove the charts dance in helmignore (related to elastic#127) * optimize docker images * replace deprecated gitversion capability (related to helm/helm#5601) * remove some heritage labels in immutable fields

* bump helm version to helm 3.4.1 * update doc to make helm 3 the recommended version * update helm commands arguments for helm 3 * remove helm init and tiller install commands * add stable repo for metricbeat dependencies * remove the charts dance in helmignore (related to #127) * optimize docker images * replace deprecated gitversion capability (related to helm/helm#5601) * remove some heritage labels in immutable fields

* bump helm version to helm 3.4.1 * update doc to make helm 3 the recommended version * update helm commands arguments for helm 3 * remove helm init and tiller install commands * add stable repo for metricbeat dependencies * remove the charts dance in helmignore (related to elastic/helm-charts#127) * optimize docker images * replace deprecated gitversion capability (related to helm/helm#5601) * remove some heritage labels in immutable fields

Crazybus added a commit that referenced this pull request May 9, 2019

Add CI definitions for metricbeat chart

cc3b7c0

This is being added to make sure that automated testing for the WIP metricbeat chart is happening in #127

Crazybus mentioned this pull request May 9, 2019

Add CI definitions for metricbeat chart #128

Merged

jmlrt reviewed May 9, 2019

View reviewed changes

kimxogus pushed a commit to aroundus-inc/elastic-charts that referenced this pull request May 22, 2019

Add CI definitions for metricbeat chart

55d4c94

This is being added to make sure that automated testing for the WIP metricbeat chart is happening in elastic#127

Crazybus added 10 commits July 1, 2019 13:31

Copy in all of the missing fields from the daemonset

bc6a027

Update description and beta disclaimer

0d2fc1f

Fix up linting and tests

4f41c6c

`helm lint` also tries to lint subcharts. So the makefile now adds `charts/` to the .helmignore before running the linting then adds it back when adding the dependency.

Update goss testing to support testing multiple containers

6b5cd0c

Add tests to make sure that data is flowing in

a31fed0

Add 6.x example

fa00f3d

Add oss example

6ac4ffd

Add security example

a71ece6

Rebase on master and bump to latest version

1e90f5e

Crazybus force-pushed the metric_is_the_future branch from b7754fa to 1e90f5e Compare July 1, 2019 11:33

Crazybus added 2 commits July 1, 2019 13:43

Update readme and backport fixes from filebeat chart

8bc8a8b

Update file pattern to include all goss tests

8bc1365

Crazybus changed the title ~~WIP: Add metricbeat chart~~ Add metricbeat chart Jul 1, 2019

Crazybus requested a review from jmlrt July 1, 2019 12:05

Always install helm chart dependencies

39c3c86

Crazybus added 2 commits July 1, 2019 16:37

Always run helm client init to make sure deps can be added

3608b21

Remove data mount from the kube-state-metrics metricbeat

f4ff57e

Crazybus requested review from tylerjl and removed request for jmlrt July 3, 2019 16:20

tylerjl approved these changes Jul 5, 2019

View reviewed changes

Crazybus merged commit cc1de5e into master Jul 8, 2019

Crazybus deleted the metric_is_the_future branch July 8, 2019 09:52

Crazybus mentioned this pull request Jul 8, 2019

ability to add additional labels #212

Closed

This was referenced Nov 17, 2020

[meta] Helm 3 (#516) #902

Merged

Helm 3 (#516) #904

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add metricbeat chart #127

Add metricbeat chart #127

Crazybus commented May 9, 2019 •

edited

Loading

jmlrt left a comment

Crazybus commented Jul 1, 2019

Crazybus commented Jul 5, 2019

tylerjl left a comment

Add metricbeat chart #127

Add metricbeat chart #127

Conversation

Crazybus commented May 9, 2019 • edited Loading

jmlrt left a comment

Choose a reason for hiding this comment

Crazybus commented Jul 1, 2019

Crazybus commented Jul 5, 2019

tylerjl left a comment

Choose a reason for hiding this comment

Crazybus commented May 9, 2019 •

edited

Loading