python-peps/pep-0661.rst

344 lines
12 KiB
ReStructuredText
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

PEP: 661
Title: Sentinel Values
Author: Tal Einat <tal@python.org>
Discussions-To: https://discuss.python.org/t/pep-661-sentinel-values/9126
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 06-Jun-2021
Post-History: 06-Jun-2021
TL;DR: See the `Specification`_ and `Reference Implementation`_.
Abstract
========
Unique placeholder values, commonly known as "sentinel values", are useful in
Python programs for several things, such as default values for function
arguments where ``None`` is a valid input value. These cases are common
enough for several idioms for implementing such "sentinels" to have arisen
over the years, but uncommon enough that there hasn't been a clear need for
standardization. However, the common implementations, including some in the
stdlib, suffer from several significant drawbacks.
This PEP suggests adding a utility for defining sentinel values, to be used
in the stdlib and made publicly available as part of the stdlib.
Note: Changing all existing sentinels in the stdlib to be implemented this
way is not deemed necessary, and whether to do so is left to the discretion
of each maintainer.
Motivation
==========
In May 2021, a question was brought up on the python-dev mailing list
[#python-dev-thread]_ about how to better implement a sentinel value for
``traceback.print_exception``. The existing implementation used the
following common idiom::
_sentinel = object()
However, this object has an uninformative and overly verbose repr, causing the
function's signature to be overly long and hard to read::
>>> help(traceback.print_exception)
Help on function print_exception in module traceback:
print_exception(exc, /, value=<object object at
0x000002825DF09650>, tb=<object object at 0x000002825DF09650>,
limit=None, file=None, chain=True)
Additionally, two other drawbacks of many existing sentinels were brought up
in the discussion:
1. Not having a distinct type, hence it being impossible to define strict
type signatures functions with sentinels as default values
2. Incorrect behavior after being copied or unpickled, due to a separate
instance being created and thus comparisons using ``is`` failing
In the ensuing discussion, Victor Stinner supplied a list of currently used
sentinel values in the Python standard library [#list-of-sentinels-in-stdlib]_.
This showed that the need for sentinels is fairly common, that there are
various implementation methods used even within the stdlib, and that many of
these suffer from at least one of the aforementioned drawbacks.
The discussion did not lead to any clear consensus on whether a standard
implementation method is needed or desirable, whether the drawbacks mentioned
are significant, nor which kind of implementation would be good.
A poll was created on discuss.python.org [#poll]_ to get a clearer sense of
the community's opinions. The poll's results were not conclusive, with 40%
voting for "The status-quo is fine / theres no need for consistency in
this", but most voters voting for one or more standardized solutions.
Specifically, 37% of the voters chose "Consistent use of a new, dedicated
sentinel factory / class / meta-class, also made publicly available in the
stdlib".
With such mixed opinions, this PEP was created to facilitate making a decision
on the subject.
Rationale
=========
The criteria guiding the chosen implementation were:
1. The sentinel objects should behave as expected by a sentinel object: When
compared using the ``is`` operator, it should always be considered identical
to itself but never to any other object.
2. It should be simple to define as many distinct sentinel values as needed.
3. The sentinel objects should have a clear and short repr.
4. The sentinel objects should each have a *distinct* type, usable in type
annotations to define *strict* type signatures.
5. The sentinel objects should behave correctly after copying and/or
unpickling.
6. Creating a sentinel object should be a simple, straightforward one-liner.
7. Works using CPython and PyPy3. Will hopefully also work with other
implementations.
After researching existing idioms and implementations, and going through many
different possible implementations, an implementation was written which meets
all of these criteria (see `Reference Implementation`_).
Specification
=============
A new ``sentinel`` function will be added to a new ``sentinels`` module.
It will accept a single required argument, the name of the sentinel object,
and a single optional argument, the repr of the object.
::
>>> NotGiven = sentinel('NotGiven')
>>> NotGiven
<NotGiven>
>>> MISSING = sentinel('MISSING', repr='mymodule.MISSING')
>>> MISSING
mymodule.MISSING
Checking if a value is such a sentinel *should* be done using the ``is``
operator, as is recommended for ``None``. Equality checks using ``==`` will
also work as expected, returning ``True`` only when the object is compared
with itself.
The name should be set to the name of the variable used to reference the
object, as in the examples above. Otherwise, the sentinel object won't be
able to survive copying or pickling+unpickling while retaining the above
described behavior. Note, that when defined in a class scope, the name must
be the fully-qualified name of the variable in the module, for example::
class MyClass:
NotGiven = sentinel('MyClass.NotGiven')
Type annotations for sentinel values will use `typing.Literal`_.
For example::
def foo(value: int | Literal[NotGiven]) -> None:
...
.. _typing.Literal: https://docs.python.org/3/library/typing.html#typing.Literal
Reference Implementation
========================
The reference implementation is found in a dedicated GitHub repo
[#reference-github-repo]_. A simplified version follows::
def sentinel(name, repr=None):
"""Create a unique sentinel object."""
repr = repr or f'<{name}>'
module = _get_parent_frame().f_globals.get('__name__', '__main__')
class_name = _get_class_name(name, module)
class_namespace = {
'__repr__': lambda self: repr,
}
cls = type(class_name, (), class_namespace)
cls.__module__ = module
_get_parent_frame().f_globals[class_name] = cls
sentinel = cls()
cls.__new__ = lambda cls_: sentinel
return sentinel
def _get_class_name(sentinel_qualname, module_name):
return '__'.join(['_sentinel_type',
module_name.replace('.', '_'),
sentinel_qualname.replace('.', '_')])
Note that a dedicated class is created automatically for each sentinel object.
This class is assigned to the namespace of the module from which the
``sentinel()`` call was made, or to that of the ``sentinels`` module itself as
a fallback. These classes have long names comprised of several parts to
ensure their uniqueness. However, these names usually wouldn't be used, since
type annotations should use ``Literal[]`` as described above, and identity
checks should be preferred over type checks.
Rejected Ideas
==============
Use ``NotGiven = object()``
---------------------------
This suffers from all of the drawbacks mentioned in the `Rationale`_ section.
Add a single new sentinel value, e.g. ``MISSING`` or ``Sentinel``
-----------------------------------------------------------------
Since such a value could be used for various things in various places, one
could not always be confident that it would never be a valid value in some use
cases. On the other hand, a dedicated and distinct sentinel value can be used
with confidence without needing to consider potential edge-cases.
Additionally, it is useful to be able to provide a meaningful name and repr
for a sentinel value, specific to the context where it is used.
Finally, this was a very unpopular option in the poll [#poll]_, with only 12%
of the votes voting for it.
Use the existing ``Ellipsis`` sentinel value
--------------------------------------------
This is not the original intended use of Ellipsis, though it has become
increasingly common to use it to define empty class or function blocks instead
of using ``pass``.
Also, similar to a potential new single sentinel value, ``Ellipsis`` can't be
as confidently used in all cases, unlike a dedicated, distinct value.
Use a single-valued enum
------------------------
The suggested idiom is:
::
class NotGivenType(Enum):
NotGiven = 'NotGiven'
NotGiven = NotGivenType.NotGiven
Besides the excessive repetition, the repr is overly long:
``<NotGivenType.NotGiven: 'NotGiven'>``. A shorter repr can be defined, at
the expense of a bit more code and yet more repetition.
Finally, this option was the least popular among the nine options in the poll
[#poll]_, being the only option to receive no votes.
A sentinel class decorator
--------------------------
The suggested interface:
::
@sentinel(repr='<NotGiven>')
class NotGivenType: pass
NotGiven = NotGivenType()
While this allowed for a very simple and clear implementation, the interface
is too verbose, repetitive, and difficult to remember.
Using class objects
-------------------
Since classes are inherently singletons, using a class as a sentinel value
makes sense and allows for a simple implementation.
The simplest version of this idiom is:
::
class NotGiven: pass
To have a clear repr, one could define ``__repr__``:
::
class NotGiven:
def __repr__(self):
return '<NotGiven>'
... or use a meta-class:
::
class NotGiven(metaclass=SentinelMeta): pass
However, all such implementations don't have a dedicated type for the
sentinel, which is considered desirable for strict typing. A dedicated type
could be created by a meta-class or class decorator, but at that point the
implementation would become much more complex and loses its advantages over
the chosen implementation.
Additionally, using classes this way is unusual and could be confusing.
Define a recommended "standard" idiom, without supplying an implementation
--------------------------------------------------------------------------
Most common exiting idioms have significant drawbacks. So far, no idiom
has been found that is clear and concise while avoiding these drawbacks.
Also, in the poll on this subject [#poll]_, the options for recommending an
idiom were unpopular, with the highest-voted option being voted for by only
25% of the voters.
Additional Notes
================
* This PEP and the initial implementation are drafted in a dedicated GitHub
repo [#reference-github-repo]_.
* The support for copying/unpickling works when defined in a module's scope or
a (possibly nested) class's scope. Note that in the latter case, the name
provided as the first parameter must be the fully-qualified name of the
variable in the module::
class MyClass:
NotGiven = sentinel('MyClass.NotGiven', repr='<NotGiven>')
References
==========
.. [#python-dev-thread] Python-Dev mailing list: `The repr of a sentinel <https://mail.python.org/archives/list/python-dev@python.org/thread/ZLVPD2OISI7M4POMTR2FCQTE6TPMPTO3/>`_
.. [#list-of-sentinels-in-stdlib] Python-Dev mailing list: `"The stdlib contains tons of sentinels" <https://mail.python.org/archives/list/python-dev@python.org/message/JBYXQH3NV3YBF7P2HLHB5CD6V3GVTY55/>`_
.. [#poll] discuss.python.org Poll: `Sentinel Values in the Stdlib <https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/>`_
.. [#reference-github-repo] `Reference implementation at the taleinat/python-stdlib-sentinels GitHub repo <https://github.com/taleinat/python-stdlib-sentinels>`_
.. [5] `bpo-44123: Make function parameter sentinel values true singletons <https://bugs.python.org/issue44123>`_
.. [6] `The "sentinels" package on PyPI <https://pypi.org/project/sentinels/>`_
.. [7] `The "sentinel" package on PyPI <https://pypi.org/project/sentinel/>`_
.. [8] `Discussion thread about type signatures for these sentinels on the typing-sig mailing list <https://mail.python.org/archives/list/typing-sig@python.org/thread/NDEJ7UCDPINP634GXWDARVMTGDVSNBKV/#LVCPTY26JQJW7NKGKGAZXHQKWVW7GOGL>`_
Copyright
=========
This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: