Add list of raw_vars whose values are auto-escaped

[fullyint]

#548 wrapped WP salts/keys in {% raw %} to avoid templating failures due to values containing opening jinja delimiters {{ and {%. This was not user-configurable (no easy way to opt-out of the magic) and it didn't wrap other variables likely to have randomly generated values, leaving the potential for problematic jinja delimiters.

This PR replaces the #548 method with a new list of raw_vars as a user-configurable method to wrap chosen variable values in {% raw %}.

This doesn't seem to be a very risky use of the potentially evolving Ansible plugin API. The only new API material introduced is the checking of Ansible-specific classes AnsibleMapping, AnsibleSequence, and AnsibleUnicode in the raw_triage method. These classes have been around all of Ansible 2.0.

---

To see it work, pull this branch then add the vars below to the end of group_vars/all/main.yml:

simple_var: no {{ fail

complex_var:
  - name: "{{ admin_user }}"
    password: random_with_{{_jinja_delimiter
    salt: ran{%dom
    some_list:
      - no {{ fail
      - templated -> {{ admin_user }}
  - name: "{{ web_user }}"
    some_dict:
      key_1: no {% fail
      key_2: no {{ fail
      other_key: templated -> {{ web_user }}

raw_vars:
  - simple_var
  - complex_var.*.password
  - complex_var.*.salt
  - complex_var.*.some_list.0
  - complex_var.*.some_dict.key_*

Now add this task to the end of the Determine Remote User play in server.yml:

  tasks:
    - debug:
        msg: |
          {{ simple_var | to_nice_yaml }}
          {{ complex_var | to_nice_yaml }}
      tags: remote-user

Now run server.yml to print the vars, seeing that the stray jinja delimiters don't cause failures:

ansible-playbook server.yml -e env=production --tags remote-user

[swalkinshaw]

This is good but think I'd rather skip all the wildcard stuff for now. It's currently unused by default and not sure it's worth the potential hassle.

[fullyint]

We wouldn't want the wildcard functionality to become a wildcard of its own. Surprises not welcome.

But to explain, the wildcard functionality stemmed from development on #614 with its new var:

vault_users:
  - name: "{{ admin_user }}"
    password: example_password
    salt: "generateme"

This var would require the wildcard, a la:

raw_vars:
  - vault_users.*.password
  - vault_users.*.salt

[swalkinshaw]

I wonder if it's better to move the default vars into here.

Pros: users shouldn't have to worry about these too much so they are kept out of the way and harder to mess up.
Cons: to disable them would require raw_vars: [] and maybe less visibility?

[fullyint]

That "visibility" was my focus. My overarching goal was to transition this functionality from "hidden" -- which for users is possibly surprising and certainly non-configurable -- to "out in the open" and easy to use.

I agree most users won't understand it and won't touch it, in which case they won't mess it up. In that sense, I think it is like neighboring obscure vars in group_vars/all/main.yml, such as wordpress_env_defaults or site_env.

For users who do try to use it, actually adding patterns versus merely disabling it, I think they'll have the easiest time if they readily see the full list of raw_vars default patterns right there in a group_vars file.

Would users really want to adjust patterns? I think a conceivable example would be that someone wants to add an env var in vault_wordpress_sites that should not be wrapped in {% raw %}. They'd need to adjust the raw_vars to be more specific, just wrapping the right parts, such as:

raw_vars:
  - vault_wordpress_sites.*.*_password       # for `admin_password` (dev only) and `db_password`
  - vault_wordpress_sites.*.*_salt
  - vault_wordpress_sites.*.*_key

On the other hand, if we decide that visibility and configurability aren't the goal or aren't worth it, then I suppose moving the defaults into vars.py makes sense. In that case, we could consider whether the python code would be "safer" being more targeted to those defaults, not supporting the wildcard stuff and being more "hard-coded" like it has been for WP keys and salts.

[fullyint]

I incorporated @swalkinshaw's suggested comprehensions; a remarkable streamlining of the code.

---

I added a commit to raise an error if raw_vars is not defined as a list.

---

The vars.py plugin accesses vars before they are templated. This is what enables us to wrap vars in {% raw %} before a templating problem occurs.

However, this means that the raw_vars variable will not be templated/expanded here unless we do it ourselves. I added a commit to template the raw_vars variable before working with it.

some_var:
  - vault_mail_password
  - vault_mysql_root_password
  - vault_wordpress_sites

raw_vars: "{{ some_var }}"

If a user were to create the unlikely scenario above, I think it would be a baffling surprise/mystery why some_var wasn't being templated/expanded in the definition of raw_vars. I introduce some code to avoid such a problem. I don't think the code has too high of a price.

• The Templar params seem relatively stable, unchanged during history of Ansible 2.x
• The Templar.template() params have had only one minor irrelevant change during history of 2.x
• DataLoader has been around all of 2.x

I put self.loader in __init__ so it is accessible to other methods to use for templating in the future.

[swalkinshaw]

Works great 🎉