[FEATURE REQUEST] Filtering and templating the vault pillar module #60777
Labels
Feature
new functionality including changes to functionality and code refactors, etc.
Pending-Discussion
The issue or pull request needs more discussion before it can be closed or merged
Pillar
Vault
Is your feature request related to a problem? Please describe.
The current vault pillar module is pretty static and only allows filtering by the minion id. #47817 and #47483 are probably related.
I opened this issue to see if this feature would be desired to be merged into the tree before I open a pull request since it would require a bit more work to do some cleanup, additional documentation, making tests work from the salt repository and removing environment specific code.
Describe the solution you'd like
I have written a pillar extension_module for this with tests which allows filtering results by keys and values for a vault lookup which allows filtering by either static values for each key or dynamic values(only grains and pillar lookups currently).
Also templating of the mapping of a vault path -> pillar path with either nested lookups(via jinja2 templating), grains, pillars lookups(using results or exact matches of a single value of the result of the lookup) or wildcards.
Additional context
I could keep support for the old config style since filtering and templates are just an addition and this mapping is just basically the same. I just liked it more explict with using path and pillarpath as keys and iterating over each config as a list instead of adding multiple vault module definitions in master config.
The following examples of possible configurations with the module show the configs from the current implementation.
Single value lookup
Mapping a single value of vault path to a pillarpath
equivalent of the old config
vault lookup:
mapped to pillar:
Wildcards
Example of using wildcards. There must be an equal amount of wildcards for both paths.
For c1.
path wildcard of vault contains
["w1", "w2"]
. For each item a lookup will be done and the result mapped to pillarpathFor c2.
The products of each result will be combined and mapped to the pillarpath
secret/*
returns["s1", "s2"]
Filtering with keys and values
Example of filtering with single lookup
vault lookup of
secret/path
value of the key
host
is in the values of the matchers so the key'private'
from the vault lookup data will not be removedif the vault lookup was
then the mapped pillar would look
Grains and pillars in filtering
Using grains or pillars in values of matchers
vault lookup of c1.
secret/path
if the value of the key
host
matches the result ofG@id
then the pillar would beif the value of
G@id
washost1
then instead the mapped pillar would lookFiltering all or none
Example of adding all keys or none if the matcher is not true.
Matching against a value of a pillar lookup
Example of matching against an exact value of a pillar lookup
Using grains or pillars in the paths
These will expand and create multiple pairs matching their result. A lookup
result which contains a list will iterate over it. For a dictionary only the
keynames will be replaced for it.
If only one or the other of the paths contains a lookup definition its lookup value
must be a single value. If the lookup value contains a list or dict then both
paths need to have the same lookup definition otherwise it will be unable to
create pairs to map against.
Vault lookup for
path: "secret/personal/I@persons"
topillarpath: "private/I@persons"
I@persons
returns a list of["p1", "p2"]
or if it can return a dict of{"p1": {...}, "p2": ...}
Mapped pillar would be merged to
Matching against a value of a lookup result in paths
Using different amounts for each path or multiple this way is fine since it
will map to one path each.
With for example
If
I@cluster
returns["master1", "master2"]
andI@role
returns["server", "backup"]
andI@app
returns["app1", ...]
then the mapping will behosts/master1/webtoken -> myapp/app1/backup/token
. Note the lookup result can be a single value or a dict in which case the keys of the dict will be used to compare.Jinja2 templates
Templates are mainly used to do lookups of a result of a previous lookup(a
template depends on a previous templates result).
The constraint for using the keys from a template definition in paths is that
all keys are of the same root template definition and either
k1 and k2 are from the same template
templates from path
ie is the set of the templates of the keys of path a subset of the set of
the templates of the keys of pillarpath
Practically one has to make sure that the templates can be resolved in order of
the list they are defined in. Currently did not test multiple template
dependency resolves with a different root in the same config(since there was no need for it in my environment).
Example of using jinja2 templates in paths and template definitions which depend
on a previous template definition.
The pillar lookup of template t1 'ext_pillar_vault_ssh_keys:present' returns
Key
'puser'
will be["admin", "root"]
The pillar lookup of template t2 'ext_pillar_vault_ssh_keys:present:admin' returns the list
Keys and value pairs will consist of the pairs in a list
[(id: id_backup, user: backup), (id: intranet, user: root)]
The pillar lookup of template t2 'ext_pillar_vault_ssh_keys:present:root' returns
Keys and value pairs will be
[(id: id_root, user: root)]
mapping of values for each pair:
Mixing of templates, lookups, wildcards.
Order of rendering to get the resolved paths for vault lookup and pillarpath:
Rendering in order
Result of template t1.
Result of template t2.
Salt pillar lookup of
I@appslaves
is["s1", "s2"]
wildcard vault listing
secret/apps/* -> ["app1", "app2"]
Mapping the resolved paths
Result of rendering in order for the paths:
If anything fails the pair will not be considered for pillar merging.
Any feedback welcome
The text was updated successfully, but these errors were encountered: