Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Parallel states on spawning platforms are missing some global dunders #66996

Open
2 tasks done
lkubb opened this issue Oct 24, 2024 · 0 comments
Open
2 tasks done
Labels
Bug broken, incorrect, or confusing behavior needs-triage

Comments

@lkubb
Copy link
Contributor

lkubb commented Oct 24, 2024

Description
On a spawning platform, running a state module relying on dunders like __env__ via parallel: true fails with a NameError (this is probably an issue for __low__, __running__, __instance_id__, __lowstate__ and __user__ as well).

Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)

Please be as specific as possible and give set-up details.

  • on-prem machine
  • onedir packaging

On bare-metal macOS with official 3007.1 package.

Note that Windows is a spawning platform as well.

Steps to Reproduce the behavior

# /opt/srv/salt/fail.sls

foo:
  git.latest:
    - name: https://github.com/saltstack/salt.git
    - target: /tmp/salt
    - parallel: true
salt-call state.apply fail

Expected behavior
The clone works.

Screenshots

----------
          ID: foo
    Function: git.latest
        Name: https://github.com/saltstack/salt.git
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/opt/salt/lib/python3.10/site-packages/salt/state.py", line 2161, in _call_parallel_target
                  ret = instance.states[cdata["full"]](*cdata["args"], **cdata["kwargs"])
                File "/opt/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 160, in __call__
                  ret = self.loader.run(run_func, *args, **kwargs)
                File "/opt/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1269, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                File "/opt/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1284, in _run_as
                  return _func_or_method(*args, **kwargs)
                File "/opt/salt/lib/python3.10/site-packages/salt/states/git.py", line 726, in latest
                  saltenv=__env__,
              NameError: name '__env__' is not defined
     Started: 10:46:38.229375
    Duration: 7.989 ms
     Changes:

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
          Salt: 3007.1

Python Version:
        Python: 3.10.14 (main, Apr  3 2024, 21:27:26) [Clang 15.0.0 (clang-1500.0.40.1)]

Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.16.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: darwin 23.6.0
        locale: utf-8
       machine: arm64
       release: 23.6.0
        system: Darwin
       version: 14.7 arm64

Additional context
inject_globals is defined in the parent process:

salt/salt/state.py

Lines 2362 to 2372 in d7b4d10

inject_globals = {
# Pass a copy of the running dictionary, the low state chunks and
# the current state dictionaries.
# We pass deep copies here because we don't want any misbehaving
# state module to change these at runtime.
"__low__": immutabletypes.freeze(low),
"__running__": immutabletypes.freeze(running) if running else {},
"__instance_id__": self.instance_id,
"__lowstate__": immutabletypes.freeze(chunks) if chunks else {},
"__user__": self.opts.get("user", "UNKNOWN"),
}

__env__ is ensured to be defined in there as well:

salt/salt/state.py

Lines 2414 to 2416 in d7b4d10

if "__env__" not in inject_globals:
# Let's use the default environment
inject_globals["__env__"] = "base"

The parent process' State instance sets the inject_globals attribute on its state loader:

self.states.inject_globals = inject_globals

When parallel is True, self.call_parallel is called

salt/salt/state.py

Lines 2435 to 2437 in d7b4d10

elif not low.get("__prereq__") and low.get("parallel"):
# run the state call in parallel, but only if not in a prereq
ret = self.call_parallel(cdata, low)

On spawning platforms, a new State instance has to be created, which also resets the inject_globals attribute on the state module loader:

salt/salt/state.py

Lines 2283 to 2291 in d7b4d10

if salt.utils.platform.spawning_platform():
instance = None
else:
instance = self
proc = salt.utils.process.Process(
target=self._call_parallel_target,
args=(instance, self._init_kwargs, name, cdata, low),
name=f"ParallelState({name})",

And it's never redefined.

@lkubb lkubb added Bug broken, incorrect, or confusing behavior needs-triage labels Oct 24, 2024
lkubb added a commit to lkubb/salt that referenced this issue Oct 24, 2024
lkubb added a commit to lkubb/salt that referenced this issue Oct 24, 2024
lkubb added a commit to lkubb/salt that referenced this issue Oct 24, 2024
dwoz pushed a commit that referenced this issue Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior needs-triage
Projects
None yet
Development

No branches or pull requests

1 participant