364 lines
16 KiB
ReStructuredText
364 lines
16 KiB
ReStructuredText
PEP: 702
|
|
Title: Marking deprecations using the type system
|
|
Author: Jelle Zijlstra <jelle.zijlstra@gmail.com>
|
|
Discussions-To: https://discuss.python.org/t/pep-702-marking-deprecations-using-the-type-system/23036
|
|
Status: Accepted
|
|
Type: Standards Track
|
|
Topic: Typing
|
|
Content-Type: text/x-rst
|
|
Created: 30-Dec-2022
|
|
Python-Version: 3.13
|
|
Post-History: `01-Jan-2023 <https://mail.python.org/archives/list/typing-sig@python.org/thread/AKTFUYW3WDT7R7PGRIJQZMYHMDJNE4QH/>`__,
|
|
`22-Jan-2023 <https://discuss.python.org/t/pep-702-marking-deprecations-using-the-type-system/23036>`__
|
|
Resolution: https://discuss.python.org/t/pep-702-marking-deprecations-using-the-type-system/23036/61
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP adds an ``@warnings.deprecated()`` decorator that marks a class or function
|
|
as deprecated, enabling static checkers to warn when it is used. By default, this
|
|
decorator will also raise a runtime ``DeprecationWarning``.
|
|
|
|
Motivation
|
|
==========
|
|
|
|
As software evolves, new functionality is added and old functionality becomes
|
|
obsolete. Library developers want to work towards removing obsolete code while
|
|
giving their users time to migrate to new APIs. Python provides a mechanism for
|
|
achieving these goals: the :exc:`DeprecationWarning` warning class, which is
|
|
used to show warnings when deprecated functionality is used. This mechanism is
|
|
widely used: as of the writing of this PEP, the CPython main branch contains
|
|
about 150 distinct code paths that raise :exc:`!DeprecationWarning`. Many
|
|
third-party libraries also use :exc:`!DeprecationWarning` to mark deprecations.
|
|
In the `top 5000 PyPI packages <https://dev.to/hugovk/how-to-search-5000-python-projects-31gk>`__,
|
|
there are:
|
|
|
|
- 1911 matches for the regex ``warnings\.warn.*\bDeprecationWarning\b``,
|
|
indicating use of :exc:`!DeprecationWarning` (not including cases where the
|
|
warning is split over multiple lines);
|
|
- 1661 matches for the regex ``^\s*@deprecated``, indicating use of some sort of
|
|
deprecation decorator.
|
|
|
|
However, the current mechanism is often insufficient to ensure that users
|
|
of deprecated functionality update their code in time. For example, the
|
|
removal of various long-deprecated :mod:`unittest` features had to be
|
|
`reverted <https://github.com/python/cpython/commit/b50322d20337ca468f2070eedb051a16ee1eba94>`__
|
|
from Python 3.11 to give users more time to update their code.
|
|
Users may run their test suite with warnings disabled for practical reasons,
|
|
or deprecations may be triggered in code paths that are not covered by tests.
|
|
|
|
Providing more ways for users to find out about deprecated functionality
|
|
can speed up the migration process. This PEP proposes to leverage static type
|
|
checkers to communicate deprecations to users. Such checkers have a thorough
|
|
semantic understanding of user code, enabling them to detect and report
|
|
deprecations that a single ``grep`` invocation could not find. In addition, many type
|
|
checkers integrate with IDEs, enabling users to see deprecation warnings
|
|
right in their editors.
|
|
|
|
Rationale
|
|
=========
|
|
|
|
At first glance, deprecations may not seem like a topic that type checkers should
|
|
touch. After all, type checkers are concerned with checking whether code will
|
|
work as is, not with potential future changes. However, the analysis that type
|
|
checkers perform on code to find type errors is very similar to the analysis
|
|
that would be needed to detect usage of many deprecations. Therefore, type
|
|
checkers are well placed to find and report deprecations.
|
|
|
|
Other languages already have similar functionality:
|
|
|
|
* GCC supports a ``deprecated`` `attribute <https://gcc.gnu.org/onlinedocs/gcc-3.1.1/gcc/Type-Attributes.html>`__
|
|
on function declarations. This powers CPython's ``Py_DEPRECATED`` macro.
|
|
* GraphQL `supports <https://spec.graphql.org/June2018/#sec-Field-Deprecation>`__
|
|
marking fields as ``@deprecated``.
|
|
* Kotlin `supports <https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-deprecated/>`__
|
|
a ``Deprecated`` annotation.
|
|
* Scala `supports <https://www.scala-lang.org/api/2.12.5/scala/deprecated.html>`__
|
|
an ``@deprecated`` annotation.
|
|
* Swift `supports <https://docs.swift.org/swift-book/ReferenceManual/Attributes.html>`__
|
|
using the ``@available`` attribute to mark APIs as deprecated.
|
|
* TypeScript `uses <https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html#deprecated>`__
|
|
the ``@deprecated`` JSDoc tag to issue a hint marking use of
|
|
deprecated functionality.
|
|
|
|
Several users have requested support for such a feature:
|
|
|
|
* `typing-sig thread <https://mail.python.org/archives/list/typing-sig@python.org/thread/E24WTMQUTGKPFKEXVCGGEFFMG7LDF3WT/>`__
|
|
* `Pyright feature request <https://github.com/microsoft/pyright/discussions/2300>`__
|
|
* `mypy feature request <https://github.com/python/mypy/issues/11439>`__
|
|
|
|
There are similar existing third-party tools:
|
|
|
|
* `Deprecated <https://pypi.org/project/Deprecated/>`__ provides a decorator to
|
|
mark classes, functions, or methods as deprecated. Access to decorated objects
|
|
raises a runtime warning, but is not detected by type checkers.
|
|
* `flake8-deprecated <https://pypi.org/project/flake8-deprecated/>`__ is a linter
|
|
plugin that warns about use of deprecated features. However, it is limited to
|
|
a short, hard-coded list of deprecations.
|
|
|
|
Specification
|
|
=============
|
|
|
|
A new decorator ``@deprecated()`` is added to the :mod:`warnings` module. This
|
|
decorator can be used on a class, function or method to mark it as deprecated.
|
|
This includes :class:`typing.TypedDict` and :class:`typing.NamedTuple` definitions.
|
|
With overloaded functions, the decorator may be applied to individual overloads,
|
|
indicating that the particular overload is deprecated. The decorator may also be
|
|
applied to the overload implementation function, indicating that the entire function
|
|
is deprecated.
|
|
|
|
The decorator takes the following arguments:
|
|
|
|
* A required positional-only argument representing the deprecation message.
|
|
* Two keyword-only arguments, ``category`` and ``stacklevel``, controlling
|
|
runtime behavior (see under "Runtime behavior" below).
|
|
|
|
The positional-only argument is of type ``str`` and contains a message that should
|
|
be shown by the type checker when it encounters a usage of the decorated object.
|
|
Tools may clean up the deprecation message for display, for example
|
|
by using :func:`inspect.cleandoc` or equivalent logic.
|
|
The message must be a string literal.
|
|
The content of deprecation messages is up to the user, but it may include the version
|
|
in which the deprecated object is to be removed, and information about suggested
|
|
replacement APIs.
|
|
|
|
Type checkers should produce a diagnostic whenever they encounter a usage of an
|
|
object marked as deprecated. For deprecated overloads, this includes all calls
|
|
that resolve to the deprecated overload.
|
|
For deprecated classes and functions, this includes:
|
|
|
|
* References through module, class, or instance attributes (``module.deprecated_object``,
|
|
``module.SomeClass.deprecated_method``, ``module.SomeClass().deprecated_method``)
|
|
* Any usage of deprecated objects in their defining module
|
|
(``x = deprecated_object()`` in ``module.py``)
|
|
* If ``import *`` is used, usage of deprecated objects from the
|
|
module (``from module import *; x = deprecated_object()``)
|
|
* ``from`` imports (``from module import deprecated_object``)
|
|
* Any syntax that indirectly triggers a call to the function. For example,
|
|
if the ``__add__`` method of a class ``C`` is deprecated, then
|
|
the code ``C() + C()`` should trigger a diagnostic. Similarly, if the
|
|
setter of a property is marked deprecated, attempts to set the property
|
|
should trigger a diagnostic.
|
|
|
|
If a method is marked with the :func:`typing.override` decorator from :pep:`698`
|
|
and the base class method it overrides is deprecated, the type checker should
|
|
produce a diagnostic.
|
|
|
|
There are additional scenarios where deprecations could come into play.
|
|
For example, an object may implement a :class:`typing.Protocol`, but one
|
|
of the methods required for protocol compliance is deprecated.
|
|
As scenarios such as this one appear complex and relatively unlikely to come up in practice,
|
|
this PEP does not mandate that type checkers detect them.
|
|
|
|
Example
|
|
-------
|
|
|
|
As an example, consider this library stub named ``library.pyi``:
|
|
|
|
.. code-block:: python
|
|
|
|
from warnings import deprecated
|
|
|
|
@deprecated("Use Spam instead")
|
|
class Ham: ...
|
|
|
|
@deprecated("It is pining for the fiords")
|
|
def norwegian_blue(x: int) -> int: ...
|
|
|
|
@overload
|
|
@deprecated("Only str will be allowed")
|
|
def foo(x: int) -> str: ...
|
|
@overload
|
|
def foo(x: str) -> str: ...
|
|
|
|
class Spam:
|
|
@deprecated("There is enough spam in the world")
|
|
def __add__(self, other: object) -> object: ...
|
|
|
|
@property
|
|
@deprecated("All spam will be equally greasy")
|
|
def greasy(self) -> float: ...
|
|
|
|
@property
|
|
def shape(self) -> str: ...
|
|
@shape.setter
|
|
@deprecated("Shapes are becoming immutable")
|
|
def shape(self, value: str) -> None: ...
|
|
|
|
Here is how type checkers should handle usage of this library:
|
|
|
|
.. code-block:: python
|
|
|
|
from library import Ham # error: Use of deprecated class Ham. Use Spam instead.
|
|
|
|
import library
|
|
|
|
library.norwegian_blue(1) # error: Use of deprecated function norwegian_blue. It is pining for the fiords.
|
|
map(library.norwegian_blue, [1, 2, 3]) # error: Use of deprecated function norwegian_blue. It is pining for the fiords.
|
|
|
|
library.foo(1) # error: Use of deprecated overload for foo. Only str will be allowed.
|
|
library.foo("x") # no error
|
|
|
|
ham = Ham() # no error (already reported above)
|
|
|
|
spam = library.Spam()
|
|
spam + 1 # error: Use of deprecated method Spam.__add__. There is enough spam in the world.
|
|
spam.greasy # error: Use of deprecated property Spam.greasy. All spam will be equally greasy.
|
|
spam.shape # no error
|
|
spam.shape = "cube" # error: Use of deprecated property setter Spam.shape. Shapes are becoming immutable.
|
|
|
|
The exact wording of the diagnostics is up to the type checker and is not part
|
|
of the specification.
|
|
|
|
Runtime behavior
|
|
----------------
|
|
|
|
In addition to the positional-only ``message`` argument,
|
|
the ``@deprecated`` decorator takes two keyword-only arguments:
|
|
|
|
* ``category``: A warning class. Defaults to :exc:`DeprecationWarning`. If this
|
|
is set to ``None``, no warning is issued at runtime and the decorator returns
|
|
the original object, except for setting the ``__deprecated__`` attribute (see below).
|
|
* ``stacklevel``: The number of stack frames to skip when issuing the warning.
|
|
Defaults to 1, indicating that the warning should be issued at the site where the
|
|
deprecated object is called. Internally, the implementation will add the number of
|
|
stack frames it uses in wrapper code.
|
|
|
|
If the decorated object is a class, the decorator wraps the ``__new__`` method
|
|
such that instantiating the class issues a warning. If the decorated object is a
|
|
callable, the decorator returns a new callable that wraps the original callable but
|
|
raises a warning when called. Otherwise, the decorator raises a ``TypeError``
|
|
(unless ``category=None`` is passed).
|
|
|
|
There are several scenarios where use of the decorated object cannot issue a warning,
|
|
including overloads, ``Protocol`` classes, and abstract methods. Type checkers may show a
|
|
warning if ``@deprecated`` is used without ``category=None`` in these cases.
|
|
|
|
To accommodate runtime introspection, the decorator sets an attribute
|
|
``__deprecated__`` on the object it is passed, as well as on the wrapper
|
|
callables it generates for deprecated classes and functions.
|
|
The value of the attribute is the message passed to the decorator.
|
|
Decorating objects that do not allow setting this attribute is not supported.
|
|
|
|
If a ``Protocol`` with the ``@runtime_checkable`` decorator is marked as deprecated,
|
|
the ``__deprecated__`` attribute should not be considered a member of the protocol,
|
|
so its presence should not affect ``isinstance`` checks.
|
|
|
|
For compatibility with :func:`typing.get_overloads`, the ``@deprecated``
|
|
decorator should be placed after the ``@overload`` decorator.
|
|
|
|
Type checker behavior
|
|
---------------------
|
|
|
|
This PEP does not specify exactly how type checkers should present deprecation
|
|
diagnostics to their users. However, some users (e.g., application developers
|
|
targeting only a specific version of Python) may not care about deprecations,
|
|
while others (e.g., library developers who want their library to remain
|
|
compatible with future versions of Python) would want to catch any use of
|
|
deprecated functionality in their CI pipeline. Therefore, it is recommended
|
|
that type checkers provide configuration options that cover both use cases.
|
|
As with any other type checker error, it is also possible to ignore deprecations
|
|
using ``# type: ignore`` comments.
|
|
|
|
Deprecation policy
|
|
------------------
|
|
|
|
We propose that CPython's deprecation policy (:pep:`387`) is updated to require that new deprecations
|
|
use the functionality in this PEP to alert users
|
|
about the deprecation, if possible. Concretely, this means that new
|
|
deprecations should be accompanied by a change to the ``typeshed`` repo to
|
|
add the ``@deprecated`` decorator in the appropriate place.
|
|
This requirement does not apply to deprecations that cannot be expressed
|
|
using this PEP's functionality.
|
|
|
|
Backwards compatibility
|
|
=======================
|
|
|
|
Creating a new decorator poses no backwards compatibility concerns.
|
|
As with all new typing functionality, the ``@deprecated`` decorator
|
|
will be added to the ``typing_extensions`` module, enabling its use
|
|
in older versions of Python.
|
|
|
|
How to teach this
|
|
=================
|
|
|
|
For users who encounter deprecation warnings in their IDE or type
|
|
checker output, the messages they receive should be clear and self-explanatory.
|
|
Usage of the ``@deprecated`` decorator will be an advanced feature
|
|
mostly relevant to library authors. The decorator should be mentioned
|
|
in relevant documentation (e.g., :pep:`387` and the :exc:`DeprecationWarning`
|
|
documentation) as an additional way to mark deprecated functionality.
|
|
|
|
Reference implementation
|
|
========================
|
|
|
|
A runtime implementation of the ``@deprecated`` decorator is
|
|
available in the `typing-extensions <https://pypi.org/project/typing-extensions/>`_
|
|
library since version 4.5.0.
|
|
The ``pyanalyze`` type checker has
|
|
`prototype support <https://github.com/quora/pyanalyze/pull/578>`__
|
|
for emitting deprecation errors, as does
|
|
`Pyright <https://github.com/microsoft/pyright/issues/4456>`__.
|
|
|
|
Rejected ideas
|
|
==============
|
|
|
|
Deprecation of modules and attributes
|
|
-------------------------------------
|
|
|
|
This PEP covers deprecations of classes, functions and overloads. This
|
|
allows type checkers to detect many but not all possible deprecations.
|
|
To evaluate whether additional functionality would be worthwhile, I
|
|
`examined <https://gist.github.com/JelleZijlstra/ff459edc5ff0918e22b56740bb28eb8b>`__
|
|
all current deprecations in the CPython standard library.
|
|
|
|
I found:
|
|
|
|
* 74 deprecations of functions, methods and classes (supported by this PEP)
|
|
* 28 deprecations of whole modules (largely due to :pep:`594`)
|
|
* 9 deprecations of function parameters (supported by this PEP through
|
|
decorating overloads)
|
|
* 1 deprecation of a constant
|
|
* 38 deprecations that are not easily detectable in the type system (for
|
|
example, for calling :func:`asyncio.get_event_loop` without an active
|
|
event loop)
|
|
|
|
Modules could be marked as deprecated by adding a ``__deprecated__``
|
|
module-level constant. However, the need for this is limited, and it
|
|
is relatively easy to detect usage of deprecated modules simply by
|
|
grepping. Therefore, this PEP omits support for whole-module deprecations.
|
|
As a workaround, users could mark all module-level classes and functions
|
|
with ``@deprecated``.
|
|
|
|
For deprecating module-level constants, object attributes, and function
|
|
parameters, a ``Deprecated[type, message]`` type modifier, similar to
|
|
``Annotated`` could be added. However, this would create a new place
|
|
in the type system where strings are just strings, not forward references,
|
|
complicating the implementation of type checkers. In addition, my data
|
|
show that this feature is not commonly needed.
|
|
|
|
Features for deprecating more kinds of objects could be added in a future
|
|
PEP.
|
|
|
|
Placing the decorator in the ``typing`` module
|
|
----------------------------------------------
|
|
|
|
An earlier version of this PEP proposed placing the ``@deprecated``
|
|
decorator in the :mod:`typing` module. However, there was feedback
|
|
that it would be unexpected for a decorator in the :mod:`typing` module
|
|
to have runtime behavior. Therefore, the PEP now proposes adding the
|
|
decorator the :mod:`warnings` module instead.
|
|
|
|
Acknowledgments
|
|
===============
|
|
|
|
A call with the typing-sig meetup group led to useful feedback on this
|
|
proposal.
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document is placed in the public domain or under the
|
|
CC0-1.0-Universal license, whichever is more permissive.
|