344 lines
12 KiB
ReStructuredText
344 lines
12 KiB
ReStructuredText
PEP: 661
|
||
Title: Sentinel Values
|
||
Author: Tal Einat <tal@python.org>
|
||
Discussions-To: https://discuss.python.org/t/pep-661-sentinel-values/9126
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 06-Jun-2021
|
||
Post-History: 06-Jun-2021
|
||
|
||
|
||
TL;DR: See the `Specification`_ and `Reference Implementation`_.
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
Unique placeholder values, commonly known as "sentinel values", are useful in
|
||
Python programs for several things, such as default values for function
|
||
arguments where ``None`` is a valid input value. These cases are common
|
||
enough for several idioms for implementing such "sentinels" to have arisen
|
||
over the years, but uncommon enough that there hasn't been a clear need for
|
||
standardization. However, the common implementations, including some in the
|
||
stdlib, suffer from several significant drawbacks.
|
||
|
||
This PEP suggests adding a utility for defining sentinel values, to be used
|
||
in the stdlib and made publicly available as part of the stdlib.
|
||
|
||
Note: Changing all existing sentinels in the stdlib to be implemented this
|
||
way is not deemed necessary, and whether to do so is left to the discretion
|
||
of each maintainer.
|
||
|
||
|
||
Motivation
|
||
==========
|
||
|
||
In May 2021, a question was brought up on the python-dev mailing list
|
||
[#python-dev-thread]_ about how to better implement a sentinel value for
|
||
``traceback.print_exception``. The existing implementation used the
|
||
following common idiom::
|
||
|
||
_sentinel = object()
|
||
|
||
However, this object has an uninformative and overly verbose repr, causing the
|
||
function's signature to be overly long and hard to read::
|
||
|
||
>>> help(traceback.print_exception)
|
||
Help on function print_exception in module traceback:
|
||
|
||
print_exception(exc, /, value=<object object at
|
||
0x000002825DF09650>, tb=<object object at 0x000002825DF09650>,
|
||
limit=None, file=None, chain=True)
|
||
|
||
Additionally, two other drawbacks of many existing sentinels were brought up
|
||
in the discussion:
|
||
|
||
1. Not having a distinct type, hence it being impossible to define strict
|
||
type signatures functions with sentinels as default values
|
||
2. Incorrect behavior after being copied or unpickled, due to a separate
|
||
instance being created and thus comparisons using ``is`` failing
|
||
|
||
In the ensuing discussion, Victor Stinner supplied a list of currently used
|
||
sentinel values in the Python standard library [#list-of-sentinels-in-stdlib]_.
|
||
This showed that the need for sentinels is fairly common, that there are
|
||
various implementation methods used even within the stdlib, and that many of
|
||
these suffer from at least one of the aforementioned drawbacks.
|
||
|
||
The discussion did not lead to any clear consensus on whether a standard
|
||
implementation method is needed or desirable, whether the drawbacks mentioned
|
||
are significant, nor which kind of implementation would be good.
|
||
|
||
A poll was created on discuss.python.org [#poll]_ to get a clearer sense of
|
||
the community's opinions. The poll's results were not conclusive, with 40%
|
||
voting for "The status-quo is fine / there’s no need for consistency in
|
||
this", but most voters voting for one or more standardized solutions.
|
||
Specifically, 37% of the voters chose "Consistent use of a new, dedicated
|
||
sentinel factory / class / meta-class, also made publicly available in the
|
||
stdlib".
|
||
|
||
With such mixed opinions, this PEP was created to facilitate making a decision
|
||
on the subject.
|
||
|
||
|
||
Rationale
|
||
=========
|
||
|
||
The criteria guiding the chosen implementation were:
|
||
|
||
1. The sentinel objects should behave as expected by a sentinel object: When
|
||
compared using the ``is`` operator, it should always be considered identical
|
||
to itself but never to any other object.
|
||
2. It should be simple to define as many distinct sentinel values as needed.
|
||
3. The sentinel objects should have a clear and short repr.
|
||
4. The sentinel objects should each have a *distinct* type, usable in type
|
||
annotations to define *strict* type signatures.
|
||
5. The sentinel objects should behave correctly after copying and/or
|
||
unpickling.
|
||
6. Creating a sentinel object should be a simple, straightforward one-liner.
|
||
7. Works using CPython and PyPy3. Will hopefully also work with other
|
||
implementations.
|
||
|
||
After researching existing idioms and implementations, and going through many
|
||
different possible implementations, an implementation was written which meets
|
||
all of these criteria (see `Reference Implementation`_).
|
||
|
||
|
||
Specification
|
||
=============
|
||
|
||
A new ``sentinel`` function will be added to a new ``sentinels`` module.
|
||
It will accept a single required argument, the name of the sentinel object,
|
||
and a single optional argument, the repr of the object.
|
||
|
||
::
|
||
|
||
>>> NotGiven = sentinel('NotGiven')
|
||
>>> NotGiven
|
||
<NotGiven>
|
||
>>> MISSING = sentinel('MISSING', repr='mymodule.MISSING')
|
||
>>> MISSING
|
||
mymodule.MISSING
|
||
|
||
Checking if a value is such a sentinel *should* be done using the ``is``
|
||
operator, as is recommended for ``None``. Equality checks using ``==`` will
|
||
also work as expected, returning ``True`` only when the object is compared
|
||
with itself.
|
||
|
||
The name should be set to the name of the variable used to reference the
|
||
object, as in the examples above. Otherwise, the sentinel object won't be
|
||
able to survive copying or pickling+unpickling while retaining the above
|
||
described behavior. Note, that when defined in a class scope, the name must
|
||
be the fully-qualified name of the variable in the module, for example::
|
||
|
||
class MyClass:
|
||
NotGiven = sentinel('MyClass.NotGiven')
|
||
|
||
Type annotations for sentinel values will use `typing.Literal`_.
|
||
For example::
|
||
|
||
def foo(value: int | Literal[NotGiven]) -> None:
|
||
...
|
||
|
||
.. _typing.Literal: https://docs.python.org/3/library/typing.html#typing.Literal
|
||
|
||
|
||
Reference Implementation
|
||
========================
|
||
|
||
The reference implementation is found in a dedicated GitHub repo
|
||
[#reference-github-repo]_. A simplified version follows::
|
||
|
||
def sentinel(name, repr=None):
|
||
"""Create a unique sentinel object."""
|
||
repr = repr or f'<{name}>'
|
||
|
||
module = _get_parent_frame().f_globals.get('__name__', '__main__')
|
||
class_name = _get_class_name(name, module)
|
||
class_namespace = {
|
||
'__repr__': lambda self: repr,
|
||
}
|
||
cls = type(class_name, (), class_namespace)
|
||
cls.__module__ = module
|
||
_get_parent_frame().f_globals[class_name] = cls
|
||
|
||
sentinel = cls()
|
||
cls.__new__ = lambda cls_: sentinel
|
||
|
||
return sentinel
|
||
|
||
def _get_class_name(sentinel_qualname, module_name):
|
||
return '__'.join(['_sentinel_type',
|
||
module_name.replace('.', '_'),
|
||
sentinel_qualname.replace('.', '_')])
|
||
|
||
|
||
Note that a dedicated class is created automatically for each sentinel object.
|
||
This class is assigned to the namespace of the module from which the
|
||
``sentinel()`` call was made, or to that of the ``sentinels`` module itself as
|
||
a fallback. These classes have long names comprised of several parts to
|
||
ensure their uniqueness. However, these names usually wouldn't be used, since
|
||
type annotations should use ``Literal[]`` as described above, and identity
|
||
checks should be preferred over type checks.
|
||
|
||
|
||
Rejected Ideas
|
||
==============
|
||
|
||
|
||
Use ``NotGiven = object()``
|
||
---------------------------
|
||
|
||
This suffers from all of the drawbacks mentioned in the `Rationale`_ section.
|
||
|
||
|
||
Add a single new sentinel value, e.g. ``MISSING`` or ``Sentinel``
|
||
-----------------------------------------------------------------
|
||
|
||
Since such a value could be used for various things in various places, one
|
||
could not always be confident that it would never be a valid value in some use
|
||
cases. On the other hand, a dedicated and distinct sentinel value can be used
|
||
with confidence without needing to consider potential edge-cases.
|
||
|
||
Additionally, it is useful to be able to provide a meaningful name and repr
|
||
for a sentinel value, specific to the context where it is used.
|
||
|
||
Finally, this was a very unpopular option in the poll [#poll]_, with only 12%
|
||
of the votes voting for it.
|
||
|
||
|
||
Use the existing ``Ellipsis`` sentinel value
|
||
--------------------------------------------
|
||
|
||
This is not the original intended use of Ellipsis, though it has become
|
||
increasingly common to use it to define empty class or function blocks instead
|
||
of using ``pass``.
|
||
|
||
Also, similar to a potential new single sentinel value, ``Ellipsis`` can't be
|
||
as confidently used in all cases, unlike a dedicated, distinct value.
|
||
|
||
|
||
Use a single-valued enum
|
||
------------------------
|
||
|
||
The suggested idiom is:
|
||
|
||
::
|
||
|
||
class NotGivenType(Enum):
|
||
NotGiven = 'NotGiven'
|
||
NotGiven = NotGivenType.NotGiven
|
||
|
||
Besides the excessive repetition, the repr is overly long:
|
||
``<NotGivenType.NotGiven: 'NotGiven'>``. A shorter repr can be defined, at
|
||
the expense of a bit more code and yet more repetition.
|
||
|
||
Finally, this option was the least popular among the nine options in the poll
|
||
[#poll]_, being the only option to receive no votes.
|
||
|
||
|
||
A sentinel class decorator
|
||
--------------------------
|
||
|
||
The suggested interface:
|
||
|
||
::
|
||
|
||
@sentinel(repr='<NotGiven>')
|
||
class NotGivenType: pass
|
||
NotGiven = NotGivenType()
|
||
|
||
While this allowed for a very simple and clear implementation, the interface
|
||
is too verbose, repetitive, and difficult to remember.
|
||
|
||
|
||
Using class objects
|
||
-------------------
|
||
|
||
Since classes are inherently singletons, using a class as a sentinel value
|
||
makes sense and allows for a simple implementation.
|
||
|
||
The simplest version of this idiom is:
|
||
|
||
::
|
||
|
||
class NotGiven: pass
|
||
|
||
To have a clear repr, one could define ``__repr__``:
|
||
|
||
::
|
||
|
||
class NotGiven:
|
||
def __repr__(self):
|
||
return '<NotGiven>'
|
||
|
||
... or use a meta-class:
|
||
|
||
::
|
||
|
||
class NotGiven(metaclass=SentinelMeta): pass
|
||
|
||
However, all such implementations don't have a dedicated type for the
|
||
sentinel, which is considered desirable for strict typing. A dedicated type
|
||
could be created by a meta-class or class decorator, but at that point the
|
||
implementation would become much more complex and loses its advantages over
|
||
the chosen implementation.
|
||
|
||
Additionally, using classes this way is unusual and could be confusing.
|
||
|
||
|
||
Define a recommended "standard" idiom, without supplying an implementation
|
||
--------------------------------------------------------------------------
|
||
|
||
Most common exiting idioms have significant drawbacks. So far, no idiom
|
||
has been found that is clear and concise while avoiding these drawbacks.
|
||
|
||
Also, in the poll on this subject [#poll]_, the options for recommending an
|
||
idiom were unpopular, with the highest-voted option being voted for by only
|
||
25% of the voters.
|
||
|
||
|
||
Additional Notes
|
||
================
|
||
|
||
* This PEP and the initial implementation are drafted in a dedicated GitHub
|
||
repo [#reference-github-repo]_.
|
||
|
||
* The support for copying/unpickling works when defined in a module's scope or
|
||
a (possibly nested) class's scope. Note that in the latter case, the name
|
||
provided as the first parameter must be the fully-qualified name of the
|
||
variable in the module::
|
||
|
||
class MyClass:
|
||
NotGiven = sentinel('MyClass.NotGiven', repr='<NotGiven>')
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [#python-dev-thread] Python-Dev mailing list: `The repr of a sentinel <https://mail.python.org/archives/list/python-dev@python.org/thread/ZLVPD2OISI7M4POMTR2FCQTE6TPMPTO3/>`_
|
||
.. [#list-of-sentinels-in-stdlib] Python-Dev mailing list: `"The stdlib contains tons of sentinels" <https://mail.python.org/archives/list/python-dev@python.org/message/JBYXQH3NV3YBF7P2HLHB5CD6V3GVTY55/>`_
|
||
.. [#poll] discuss.python.org Poll: `Sentinel Values in the Stdlib <https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810/>`_
|
||
.. [#reference-github-repo] `Reference implementation at the taleinat/python-stdlib-sentinels GitHub repo <https://github.com/taleinat/python-stdlib-sentinels>`_
|
||
.. [5] `bpo-44123: Make function parameter sentinel values true singletons <https://bugs.python.org/issue44123>`_
|
||
.. [6] `The "sentinels" package on PyPI <https://pypi.org/project/sentinels/>`_
|
||
.. [7] `The "sentinel" package on PyPI <https://pypi.org/project/sentinel/>`_
|
||
.. [8] `Discussion thread about type signatures for these sentinels on the typing-sig mailing list <https://mail.python.org/archives/list/typing-sig@python.org/thread/NDEJ7UCDPINP634GXWDARVMTGDVSNBKV/#LVCPTY26JQJW7NKGKGAZXHQKWVW7GOGL>`_
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document is placed in the public domain or under the
|
||
CC0-1.0-Universal license, whichever is more permissive.
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|