PEP 661: Sentinel Values (#1979)

pep-0661.rst

@@ -0,0 +1,323 @@
+PEP: 661
+Title: Sentinel Values
+Author: Tal Einat <tal@python.org>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 06-Jun-2021
+Post-History: 06-Jun-2021
+
+
+TL;DR: See the `Specification`_ and `Reference Implementation`_.
+
+
+Abstract
+========
+
+Unique placeholder values, widely known as "sentinel values", are useful in
+Python programs for several things, such as default values for function
+arguments where ``None`` is a valid input value.  These cases are common
+enough for several idioms for implementing such "sentinels" to have arisen
+over the years, but uncommon enough that there hasn't been a clear need for
+standardization.  However, the common implementations, including some in the
+stdlib, suffer from several significant drawbacks.
+
+This PEP suggests adding a utility for defining sentinel values, to be used
+in the stdlib and made publicly available as part of the stdlib.
+
+Note: Changing all existing sentinels in the stdlib to be implemented this
+way is not deemed necessary, and whether to do so is left to the discretion
+of each maintainer.
+
+
+Motivation
+==========
+
+In May 2021, a question was brought up on the python-dev mailing list
+[#python-dev-thread]_ about how to better implement a sentinel value for
+``traceback.print_exception``.  The existing implementation used the
+following common idiom:
+
+::
+
+    _sentinel = object()
+
+However, this object has an overly verbose repr, causing the function's
+signature to be overly long and hard to read, as seen e.g. when calling
+``help()``:
+
+::
+
+    >>> help(traceback.print_exception)
+    Help on function print_exception in module traceback:
+
+    print_exception(exc, /, value=<object object at
+    0x000002825DF09650>, tb=<object object at 0x000002825DF09650>,
+    limit=None, file=None, chain=True)
+
+Additionally, two other drawbacks of many existing sentinels were brought up
+in the discussion:
+
+1. Not having a distinct type, hence it being impossible to define strict
+   type signatures functions with sentinels as default values
+2. Incorrect behavior after being copied or unpickled, due to a separate
+   instance being created and thus comparisons using ``is`` failing
+
+In the ensuing discussion, Victor Stinner supplied a list of currently used
+sentinel values in the Python standard library [#list-of-sentinels-in-stdlib]_.
+This showed that the need for sentinels is fairly common, that there are
+various implementation methods used even within the stdlib, and that many of
+these suffer from at least one of the aforementioned drawbacks.
+
+The discussion did not lead to any clear consensus on whether a standard
+implementation method is needed or desirable, whether the drawbacks mentioned
+are significant, nor which kind of implementation would be good.
+
+A poll was created on discuss.python.org [#poll]_to get a clearer sense of
+the community's opinions. The poll's results were not conclusive, with 40%
+voting for "The status-quo is fine / there’s no need for consistency in
+this", but most voters voting for one or more standardized solutions.
+Specifically, 37% of the voters chose "Consistent use of a new, dedicated
+sentinel factory / class / meta-class, also made publicly available in the
+stdlib".
+
+With such mixed opinions, this PEP was created to facilitate making a decision
+on the subject.
+
+
+Rationale
+=========
+
+The criteria guiding the chosen implementation were:
+
+1. The sentinel objects should behave as expected by a sentinel object: When
+   compared using the ``is`` operator, it should always be considered identical
+   to itself but never to any other object.
+2. It should be simple to define as many distinct sentinel values as needed.
+3. The sentinel objects should have a clear and short repr.
+4. The sentinel objects should each have a *distinct* type, usable in type
+   annotations to define *strict* type signatures.
+5. The sentinel objects should behave correctly after copying and/or
+   unpickling.
+6. Creating a sentinel object should be a simple, straightforward one-liner.
+7. Works using CPython and PyPy3.  Will hopefully also work with other
+   implementations.
+
+After researching existing idioms and implementations, and going through many
+different possible implementations, an implementation was written which meets
+all of these criteria (see `Reference Implementation`_).
+
+
+Specification
+=============
+
+A new ``sentinel`` function will be added to a new ``sentinels`` module.
+It will accept a single required argument, the name of the sentinel object,
+and a single optional argument, the repr of the object.
+
+::
+
+    >>> NotGiven = sentinel('NotGiven')
+    >>> NotGiven
+    <NotGiven>
+    >>> MISSING = sentinel('MISSING', repr='mymodule.MISSING')
+    >>> MISSING
+    mymodule.MISSING
+
+
+A third optional argument, the name of the module where the sentinel is
+defined, exists to be used to support cases where the name of the module
+cannot be found by inspecting the stack frame.  This is identical to the
+pattern used by ``collections.namedtuple``.  (The name of the module is
+used to choose a unique name for the class generated for the new sentinel,
+which is set as an attribute of the ``sentinels`` module.)
+
+
+Reference Implementation
+========================
+
+The reference implementation is found in a dedicated GitHub repo
+[#reference-github-repo]_.  A simplified version follows::
+
+    def sentinel(name, repr=None):
+        """Create a unique sentinel object."""
+        repr = repr or f'<{name}>'
+
+        module = _get_parent_frame().f_globals.get('__name__', '__main__')
+        class_name = _get_class_name(name, module)
+        class_namespace = {
+            '__repr__': lambda self: repr,
+        }
+        cls = type(class_name, (), class_namespace)
+        cls.__module__ = __name__
+        globals()[class_name] = cls
+
+        sentinel = cls()
+        cls.__new__ lambda cls: sentinel
+
+        return sentinel
+
+    def _get_class_name(sentinel_qualname, module_name):
+        return '__'.join(['_sentinel_type',
+                          module_name.replace('.', '_'),
+                          sentinel_qualname.replace('.', '_')])
+
+
+Rejected Ideas
+==============
+
+
+Use ``NotGiven = object()``
+---------------------------
+
+This suffers from all of the drawbacks mentioned in the `Rationale`_ section.
+
+
+Add a single new sentinel value, e.g. ``MISSING`` or ``Sentinel``
+-----------------------------------------------------------------
+
+Since such a value could be used for various things in various places, one
+could not always be confident that it would never be a valid value in some use
+cases.  On the other hand, a dedicated and distinct sentinel value can be used
+with confidence without needing to consider potential edge-cases.
+
+Additionally, it is useful to be able to provide a meaningful name and repr
+for a sentinel value, specific to the context where it is used.
+
+Finally, this was a very unpopular option in the poll, with only 12% of
+the votes voting for it.
+
+
+Use the existing ``Ellipsis`` sentinel value
+--------------------------------------------
+
+This is not the original intended use of Ellipsis, though it has become
+increasingly common to use it to define empty class or function blocks instead
+of using ``pass``.
+
+Also, similar to a potential new single sentinel value, ``Ellipsis`` can't be
+as confidently used in all cases, unlike a dedicated, distinct value.
+
+
+Use a single-valued enum
+------------------------
+
+The suggested idiom is:
+
+::
+
+    class NotGivenType(Enum):
+        NotGiven = 'NotGiven'
+    NotGiven = NotGivenType.NotGiven
+
+Besides the excessive repetition, the repr is overly long:
+``<NotGivenType.NotGiven: 'NotGiven'>``.  A shorter repr can be defined, at
+the expense of a bit more code and yet more repetition.
+
+Finally, this option was the least popular among the nine options in the poll
+[#poll]_, being the only option to receive no votes.
+
+
+A sentinel class decorator
+--------------------------
+
+The suggested interface:
+
+::
+
+    @sentinel(repr='<NotGiven>')
+    class NotGivenType: pass
+    NotGiven = NotGivenType()
+
+While this allowed for a very simple and clear implementation, the interface
+is too verbose, repetitive, and difficult to remember.
+
+
+Using class objects
+-------------------
+
+Since classes are inherently singletons, using a class as a sentinel value
+makes sense and allows for a simple implementation.
+
+The simplest version of this idiom is:
+
+::
+
+   class NotGiven: pass
+
+To have a clear repr, one could define ``__repr__``:
+
+::
+
+    class NotGiven:
+        def __repr__(self):
+            return '<NotGiven>'
+
+... or use a meta-class:
+
+::
+
+    class NotGiven(metaclass=SentinelMeta): pass
+
+However, all such implementations don't have a dedicated type for the
+sentinel, which is considered desirable.  A dedicated type could be created
+by a meta-class or class decorator, but at that point the implementation would
+become much more complex and loses its advantages over the chosen
+implementation.
+
+Additionally, using classes this way is unusual and could be confusing.
+
+
+Define a recommended "standard" idiom, without supplying an implementation
+--------------------------------------------------------------------------
+
+Most common exiting idioms have significant drawbacks.  So far, no idiom
+has been found that is clear and concise while avoiding these drawbacks.
+
+Also, in the poll on this subject [#poll]_, the options for recommending an
+idiom were unpopular, with the highest-voted option being voted for by only
+25% of the voters.
+
+
+Additional Notes
+================
+
+* This PEP and the initial implementation are drafted in a dedicated GitHub
+  repo [#reference-github-repo]_.
+
+* The support for copying/unpickling works when defined in a module's scope or
+  a (possibly nested) class's scope.  Note that in the latter case, the name
+  provided as the first parameter must be the fully-qualified name of the
+  variable in the module::
+
+      class MyClass:
+          NotGiven = sentinel('MyClass.NotGiven', repr='<NotGiven>')
+
+
+References
+==========
+
+.. [#reference-github-repo] `Reference implementation at the taleinat/python-stdlib-sentinels GitHub repo <https://github.com/taleinat/python-stdlib-sentinels>`_
+.. [#python-dev-thread] Python-Dev mailing list: `The repr of a sentinel <https://mail.python.org/archives/list/python-dev@python.org/thread/ZLVPD2OISI7M4POMTR2FCQTE6TPMPTO3/>`_
+.. [#list-of-sentinels-in-stdlib] Python-Dev mailing list: `"The stdlib contains tons of sentinels" <https://mail.python.org/archives/list/python-dev@python.org/message/JBYXQH3NV3YBF7P2HLHB5CD6V3GVTY55/>`_
+.. [#poll] discuss.python.org Poll: `Sentinel Values in the Stdlib <https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/>`_
+.. [5] `bpo-44123: Make function parameter sentinel values true singletons <https://bugs.python.org/issue44123>`_
+.. [6] `The "sentinels" package on PyPI <https://pypi.org/project/sentinels/>`_
+.. [7] `The "sentinel" package on PyPI <https://pypi.org/project/sentinel/>`_
+
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End:

pep-8001.rst

@@ -10,7 +10,7 @@ Author: Brett Cannon <brett@python.org>,
         Nathaniel J. Smith <njs@pobox.com>,
         Pablo Galindo Salgado <pablogsal@python.org>,
         Raymond Hettinger <python@rcn.com>,
-        Tal Einat <taleinat@gmail.com>,
+        Tal Einat <tal@python.org>,
         Tim Peters <tim.peters@gmail.com>,
         Zachary Ware <zachary.ware@gmail.com>
 Status: Accepted