Description
openedon May 8, 2019
The cmd beat setup offers different flags, to only setup specified components of the environment. This aims to allow for more customized setup, especially in combination with using the beat export xxx command.
At the moment libbeat offers setup --template and setup --ilm-policy commands for Elasticsearch index related setup. What's problematic here is that template and ILM setup are entangled with each other, leading to some unexpected behaviour when running the commands.
Running setup, setup --template and setup --ilm-policy leads to the same result in Elasticsearch (run with metricbeat-7.0.0):
| setup. template. enabled | setup. ilm. enabled | load template | include lifecycle settings | load ilm policies | |
|---|---|---|---|---|---|
| true | true | yes | yes | yes | |
| true | false | yes | no | no | |
| false | true | no | - | yes | |
| false | false | no | - | no |
This is somewhat unexpected behaviour, as the cmd flags suggest that only a template or only an ilm policy is loaded. If the settings were acting completely unrelated from each other, the setup could be in a non-functioning state. Only setting up policies is not enough to apply ILM. Only setting up a template with lifecycle settings but not having a write alias or policy would lead to an exception.
This all comes down to having two different cmd flags for something that is strongly coupled. In my opinion we should not change the current behaviour for existing flags for reasons:
- would be a breaking change for customers that only run one of the two setup cmd flags and have ILM enabled
- could leave customers with a broken setup.
On the other hand this makes it clear that the current options are not the best fit anymore. I suggest to
- deprecate
setup --templateandsetup --ilm-policy - introduce
setup --index-managementthat combines what belongs together - the whole ES index management setup, currently consisting of templates, ilm-policies and write aliases, always considering what is enabled and what not, which still allows for customisation. Suggested behavior:
| setup. template. enabled | setup. ilm. enabled | load template | include lifecycle settings | load ilm policies | note | |
|---|---|---|---|---|---|---|
| true | true | yes | yes | yes | ||
| true | false | yes | no | no | ||
| false | true | no | - | yes | prints warning | |
| false | false | no | - | no | prints warning |
The setup command flags can be used in combination with the beat export template and beat export ilm-policy commands, allowing to print components to stdout or to files in a defined directory (recently introduced). The setup --index-management command in combination with the export command still keeps all the flexibility that users have now, but hopefully makes it more clear what happens during setup.
