"salt-ssh --roster dir ..." ignores "roster_defaults" master configuration option

[st-voss]

Description of Issue

salt-ssh ignores the master configuration option "roster_defaults" when using a directory roster (--roster dir).
When using a flat file roster (--roster flat) the master configuration option "roster_defaults" is working.

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)

Flat file roster:

[root@endeavourvm ~]# cat /etc/salt/roster
endeavourvm: 
  host: 192.168.122.193

Directory roster:

[root@endeavourvm ~]# cat /etc/salt/roster.d/endeavourvm 
host: 192.168.122.193

From master configuration file /etc/salt/master

roster_defaults:
  thin_dir: /tmp/salt-thin_dir

Steps to Reproduce Issue

Comment out any roster_defaults in /etc/salt/master

# roster_defaults:
#   thin_dir: /tmp/salt-thin_dir

Call salt-ssh with "--roster flat"

[root@endeavourvm ~]# salt-ssh --roster flat endeavourvm  config.option thin_dir
/usr/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
endeavourvm:
    /var/tmp/.root_d39a06_salt

Call salt-ssh with "--roster dir"

[root@endeavourvm ~]# salt-ssh --roster dir endeavourvm  config.option thin_dir
/usr/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
endeavourvm:
    /var/tmp/.root_d39a06_salt

Note that thin_dir has the default setting with roster_defaults commented out -> Ok

Comment in roster_defaults configuration in /etc/salt/master.
In this case I want to have "thin_dir: /tmp/salt-thin_dir" for all my minions

roster_defaults:
  thin_dir: /tmp/salt-thin_dir

Call salt-ssh with "--roster flat" again

[root@endeavourvm ~]# salt-ssh --roster flat endeavourvm  config.option thin_dir
/usr/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
endeavourvm:
    /tmp/salt-thin_dir

Note that thin_dir now reflects the setting from roster_defaults in /etc/salt/master

Call salt-ssh with "--roster dir" again

[root@endeavourvm ~]# salt-ssh --roster dir endeavourvm  config.option thin_dir
/usr/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
endeavourvm:
    /var/tmp/.root_d39a06_salt

Note that thin_dir still has the default setting and not the setting from roster_defaults in /etc/salt/master

Versions Report

[root@endeavourvm ~]# salt --versions-report
/usr/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Salt Version:
          Salt: 3005.1
 
Dependency Versions:
          cffi: 1.15.1
      cherrypy: Not Installed
      dateutil: Not Installed
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
      M2Crypto: 0.38.0
          Mako: Not Installed
       msgpack: 1.0.4
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.12.0
        pygit2: Not Installed
        Python: 3.10.8 (main, Oct 13 2022, 21:13:48) [GCC 12.2.0]
  python-gnupg: Not Installed
        PyYAML: 6.0
         PyZMQ: 24.0.1
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: endeavouros rolling rolling
        locale: utf-8
       machine: x86_64
       release: 6.0.6-arch1-1
        system: Linux
       version: EndeavourOS rolling rolling

[welcome]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[st-voss]

Possible workaround/solution:

*** /usr/lib/python3.10/site-packages/salt/roster/dir.py.ori	2022-11-08 16:45:38.225960279 +0100
--- /usr/lib/python3.10/site-packages/salt/roster/dir.py	2022-11-08 16:52:13.670567268 +0100
***************
*** 49,54 ****
--- 49,55 ----
  
  import logging
  import os
+ import copy
  
  import salt.loader
  import salt.template
***************
*** 73,79 ****
          target_file = salt.utils.verify.clean_path(roster_dir, minion_id)
          if not os.path.exists(target_file):
              raise CommandExecutionError("{} does not exist".format(target_file))
!         rendered[minion_id] = _render(target_file, **kwargs)
      pruned_rendered = {id_: data for id_, data in rendered.items() if data}
      log.debug(
          "Matched %d minions with tgt=%s and tgt_type=%s."
--- 74,81 ----
          target_file = salt.utils.verify.clean_path(roster_dir, minion_id)
          if not os.path.exists(target_file):
              raise CommandExecutionError("{} does not exist".format(target_file))
!         rendered[minion_id] = copy.deepcopy(__opts__.get("roster_defaults", {}))
!         rendered[minion_id].update(_render(target_file, **kwargs))
      pruned_rendered = {id_: data for id_, data in rendered.items() if data}
      log.debug(
          "Matched %d minions with tgt=%s and tgt_type=%s."