[BUG] salt-ssh support for extra_filerefs in Saltfile is broken

[edgan]

Description
I use lots of map.jinjas with salt-ssh, as part of many formulas in my states git repo. Since the files are only imported I need to use extra_filerefs to include them all. I do this in my Saltfile. As of 3003 this doesn't work anymore. If I downgrade to 3002.6, it works.

I thought it might relate to this issue below, but reverting the patch didn't seem to fix it for me. Though I do question the conversion of the type of extra_filerefs from a list to a str. The only reasonable use of extra_filerefs is as a list. I have over 70 map.jinjas, and I need them all.

https://github.com/saltstack/salt/pull/59664/files

Setup
./Saltfile:

salt-ssh:
  config_dir: .
  max_procs: 1
  ssh_wipe: True
  extra_filerefs:
    - salt://_grains/map.jinja

salt-formula/top.sls:

{% from '_grains/map.jinja' import grain with context %}

base:
  '*':
{% if grain.fqdn == 'foo.bar.com' %}
    - base
{% endif %}

Steps to Reproduce the behavior

salt-ssh -i foo.bar.com state.highstate
[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/salt/utils/templates.py", line 497, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python3.9/site-packages/salt/utils/jinja.py", line 198, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: _grains/map.jinja

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/salt/utils/templates.py", line 261, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python3.9/site-packages/salt/utils/templates.py", line 542, in render_jinja_tmpl
    raise SaltRenderError(
salt.exceptions.SaltRenderError: Jinja error: _grains/map.jinja
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/salt/utils/templates.py", line 497, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python3.9/site-packages/salt/utils/jinja.py", line 198, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: _grains/map.jinja

; line 1

---
{% from '_grains/map.jinja' import grain with context %}    <======================

base:
  '*':
{% if grain.fqdn == 'foo.bar.com' %}
    - base    
[...]
---
[ERROR   ] Unable to render top file: Jinja error: _grains/map.jinja
foo.bar.com:

Summary for foo.bar.com
-----------
Succeeded: 0
Failed:   0
-----------
Total states run:    0
Total run time:  0.000 ms

Expected behavior
It to include the map.jinja in the tgz, and not throw an error about not finding it.

Versions Report

salt-ssh --versions-report
Salt Version:
          Salt: 3003

Dependency Versions:
          cffi: 1.14.1
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.11.3
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: 1.1.3
       msgpack: 1.0.0
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.20
      pycrypto: 2.6.1
  pycryptodome: 3.10.1
        pygit2: Not Installed
        Python: 3.9.2 (default, Feb 20 2021, 00:00:00)
  python-gnupg: Not Installed
        PyYAML: 5.4.1
         PyZMQ: 22.0.3
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: fedora 33
        locale: utf-8
       machine: x86_64
       release: 5.11.10-200.fc33.x86_64
        system: Linux
       version: Fedora 33

[dmurphy18]

@edgan Can you provide more information to reproduce this is since I have been able to replicate the issue using Salt 3002.2 with Centos 7 for master and minion. I may have a misconfiguration causing similar response.

; line 1

---
{% from '_grains/map.jinja' import grain with context %}    <======================

base:
  '*':
{% if grain.fqdn == 'foo' %}
    - base   
[...]
---
[CRITICAL] Pillar render error: Rendering SLS 'base' failed. Please see master log for details.
foo:
    ----------
root@Unknown:/srv/salt# salt-run --versions-report
Salt Version:
          Salt: 3002.2
 
Dependency Versions:
          cffi: 1.14.4
      cherrypy: Not Installed
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.8.1
       libgit2: Not Installed
      M2Crypto: 0.33.0
          Mako: Not Installed
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.20
      pycrypto: Not Installed
  pycryptodome: Not Installed
        pygit2: Not Installed
        Python: 3.6.8 (default, Apr  2 2020, 13:34:55)
  python-gnupg: Not Installed
        PyYAML: 5.4.1
         PyZMQ: 17.0.0
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.1.4
 
System Versions:
          dist: centos 7 Core
        locale: UTF-8
       machine: x86_64
       release: 3.10.0-1127.18.2.el7.x86_64
        system: Linux
       version: CentOS Linux 7 Core

With top.sls

root@Unknown:/srv/salt# cat /srv/pillar/top.sls 

base:
    '*':
     - base

and /srv/pillar/base.sls

root@Unknown:/srv/salt# cat /srv/pillar/base.sls 
{% from '_grains/map.jinja' import grain with context %}

base:
  '*':
{% if grain.fqdn == 'foo' %}
    - base   
{% endif %}

with roster

root@Unknown:/srv/salt# cat /srv/salt/roster
foo:
  host: 192.168.0.124
  user: david
  passwd: dogkat2015
#  sudo: True

with _grains/map.jinja

root@Unknown:/srv/salt# cat /srv/salt/_grains/map.jinja 

# set version to build
{% set build_version = '3000_2' %}


{% set grain.fqdn = 'foo' %}

root@Unknown:/srv/salt#

and Saltfile

root@Unknown:/srv/salt# cat ~/.salt/Saltfile 
salt-ssh:
  config_dir: .
  ssh_log_file: /var/log/salt/salt-ssh.log
  max_procs: 1
  ssh_wipe: True
  extra_filerefs:
    - salt://_grains/map.jinja

[edgan]

I have attached the bare minimum version of my setup. This fails in the same way with 3003 and works with 3002.6.

Note I do have to run rm -rf /var/tmp/.user_ab4926_salt/ /var/tmp/salt-ssh/* when I switch versions, because it will cache the old version.

No pillars
One map.jinja
Super basic base formula that installs curl
Minimal files

salt-formulas.tar.gz