DOCS: Suggest always calling exec with a globals argument and no locals argument

[hoodmane]

Many users think they want a locals argument for various reasons but they do not understand that it makes code be treated as a class definition. They do not want their code treated as a class definition and get surprised. The reason not to pass locals specifically is that the following code raises a NameError:

exec("""
def f():
    print("hi")

f()

def g():
    f()
g()
""", {}, {})

The reason not to leave out globals is as follows:

def t():
    exec("""
def f():
    print("hi")

f()

def g():
    f()
g()
    """)

---

📚 Documentation preview 📚: https://cpython-previews--119235.org.readthedocs.build/

[miss-islington-app]

Thanks @hoodmane for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-119239 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-119240 is a backport of this pull request to the 3.12 branch.

[ncoghlan]

Just a heads up that I included a rewording of this note in the PEP 667 docs PR at #119201 (see https://github.com/python/cpython/pull/119201/files#r1607516295 ) after hitting a conflict between this change and that one.

Users pass a separate locals to exec all the time in order to capture writes, and it only breaks if the code being executed defines and calls functions that attempt to access top-level variables (or includes a class definition that needs to access the class by name). The updated wording both specifies what breaks, and also specifies how to use collections.ChainMap to capture writes while still running the code with module semantics:

   .. note::

      When ``exec`` gets two separate objects as *globals* and *locals*, the
      code will be executed as if it were embedded in a class definition. This
      means functions and classes defined in the executed code will not be
      able to access variables assigned at the top level (as the "top level"
      variables are treated as class variables in a class definition).
      Passing a :class:`collections.ChainMap` instance as *globals* allows name
      lookups to be chained across multiple mappings without triggering this
      behaviour. Values assigned to top level names in the executed code can be
      retrieved by passing an empty dictionary as the first entry in the chain.

That PR will only get backported as far as 3.13 (since the rest of the PR is specific to a 3.13 change). I'm not sure it's worth worrying about changing the note in 3.12, though.

[hoodmane]

Thanks @ncoghlan! I wanted to keep to as few words as possible here so that the PR wouldn't get bogged down in discussion. I am always concerned that each added word makes it more likely that the PR leads to an endless discussion, I run out of energy, and it goes stale.

[hoodmane]

Passing a :class:collections.ChainMap instance as globals

Interesting!

[ncoghlan]

I actually need to revisit that updated note, as I was reminded today that globals has to be a real dict, so passing ChainMap doesn't work:

>>> from collections import ChainMap
>>> ns = ChainMap({}, globals())
>>> exec("print(__name__)", globals())
__main__
>>> exec("print(__name__)", ns)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: exec() globals must be a dict, not ChainMap

(and there are unfortunately genuine performance reasons for that restriction, so relaxing it would be easier said than done)

[ncoghlan]

PR fixing the error: #119378 (and tagged @gpshead for a review)

[hoodmane]

I think what people may want when they pass locals is for globals to work like a custom overlay over builtins and locals to work like globals? I wonder if that is possible.

[gpshead]

Thanks Alyssa, I wasn't totally happy with the wording in this PR but felt it started to shift us towards the right thing to express. I like your wording better. On of my key issues is that long existing "the code will be executed as if it were embedded in a class definition" wording (from before this PR) is the kind of thing that most docs readers are not likely to understand as it's "weird" but most people need never understand it. Your new text expands upon that in an accessible descriptive manner.

[CNSeniorious000]

for globals to work like a custom overlay over builtins and locals to work like globals? I wonder if that is possible.

I think the best practise to implement layered context is to use ChainMap.

Let's say you have 2 maps, a Mapping global_namespace (acts as the readonly fallback layer, like __builtins__) and a MutableMapping local_namespace (acts as the globals). And build a map namespace = ChainMap(local_namespace, global_namespace) and getitem will first try local_namespace than try global_namespace, while setitem will never modify local_namespace.

But! eval's globals can only be a dict, not a ChainMap, so as to pyodide.runPython. So you have to use a mixin hack like this:

from collections import ChainMap

class ChainMap(ChainMap, dict):
    pass

Reproduction:

>>> from collections import ChainMap

>>> g, l = {"a": 2}, {}

>>> source = """
... def f():
...     return g() + 1
...
... def g():
...     return a
...
... a = f()
... """

>>> exec(source, ChainMap(l, g))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: globals must be a real dict; try eval(expr, {}, mapping)

>>> class ChainMap(ChainMap, dict):
...     pass
...
>>> exec(source, ChainMap(l, g))

>>> g
{'a': 2}

>>> l
{'f': <function f at 0xa04368>, 'g': <function g at 0x12654d0>, 'a': 3}

I think its a common use case to avoid pollution on the global namespace when using exec and eval, so maybe CPython can make ChainMap to inherit from dict so that we don't need to mixin anymore?