Merge branch 'master' of github.com:python/peps
This commit is contained in:
commit
8fba9ebb22
|
@ -6,8 +6,8 @@ Type: Standards Track
|
||||||
Content-Type: text/x-rst
|
Content-Type: text/x-rst
|
||||||
Created: 2017-09-08
|
Created: 2017-09-08
|
||||||
Python-Version: 3.7
|
Python-Version: 3.7
|
||||||
Resolution: https://mail.python.org/pipermail/python-dev/2017-September/149438.html
|
|
||||||
Post-History: 2017-09-09
|
Post-History: 2017-09-09
|
||||||
|
Resolution: https://mail.python.org/pipermail/python-dev/2017-September/149438.html
|
||||||
|
|
||||||
|
|
||||||
Abstract
|
Abstract
|
||||||
|
@ -61,7 +61,7 @@ you return ``7`` or ``((7,), {})``? And so on.
|
||||||
The author claims that you won't ever need the return value of ``noop()`` so
|
The author claims that you won't ever need the return value of ``noop()`` so
|
||||||
it will always return ``None``.
|
it will always return ``None``.
|
||||||
|
|
||||||
Coghlin's Dialogs (edited for formatting):
|
Coghlan's Dialogs (edited for formatting):
|
||||||
|
|
||||||
My counterargument to this would be ``map(noop, iterable)``,
|
My counterargument to this would be ``map(noop, iterable)``,
|
||||||
``sorted(iterable, key=noop)``, etc. (``filter``, ``max``, and
|
``sorted(iterable, key=noop)``, etc. (``filter``, ``max``, and
|
||||||
|
|
|
@ -0,0 +1,202 @@
|
||||||
|
PEP: 560
|
||||||
|
Title: Core support for generic types
|
||||||
|
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 03-Sep-2017
|
||||||
|
Python-Version: 3.7
|
||||||
|
Post-History: 09-Sep-2017
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
Initially PEP 484 was designed in such way that it would not introduce
|
||||||
|
*any* changes to the core CPython interpreter. Now type hints and
|
||||||
|
the ``typing`` module are extensively used by the community, e.g. PEP 526
|
||||||
|
and PEP 557 extend the usage of type hints, and the backport of ``typing``
|
||||||
|
on PyPI has 1M downloads/moth. Therefore, this restriction can be removed.
|
||||||
|
It is proposed to add two special methods ``__class_getitem__`` and
|
||||||
|
``__subclass_base__`` to the core CPython for better support of
|
||||||
|
generic types.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale
|
||||||
|
=========
|
||||||
|
|
||||||
|
The restriction to not modify the core CPython interpreter lead to some
|
||||||
|
design decisions that became questionable when the ``typing`` module started
|
||||||
|
to be widely used. There are three main points of concerns:
|
||||||
|
performance of the ``typing`` module, metaclass conflicts, and the large
|
||||||
|
number of hacks currently used in ``typing``.
|
||||||
|
|
||||||
|
|
||||||
|
Performance:
|
||||||
|
------------
|
||||||
|
|
||||||
|
The ``typing`` module is one of the heaviest and slowest modules in
|
||||||
|
the standard library even with all the optimizations made. Mainly this is
|
||||||
|
because subscripted generic types (see PEP 484 for definition of terms
|
||||||
|
used in this PEP) are class objects (see also [1]_). The three main ways how
|
||||||
|
the performance can be improved with the help of the proposed special methods:
|
||||||
|
|
||||||
|
- Creation of generic classes is slow since the ``GenericMeta.__new__`` is
|
||||||
|
very slow; we will not need it anymore.
|
||||||
|
|
||||||
|
- Very long MROs for generic classes will be twice shorter; they are present
|
||||||
|
because we duplicate the ``collections.abc`` inheritance chain
|
||||||
|
in ``typing``.
|
||||||
|
|
||||||
|
- Time of instantiation of generic classes will be improved
|
||||||
|
(this is minor however).
|
||||||
|
|
||||||
|
|
||||||
|
Metaclass conflicts:
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
All generic types are instances of ``GenericMeta``, so if a user uses
|
||||||
|
a custom metaclass, then it is hard to make a corresponding class generic.
|
||||||
|
This is particularly hard for library classes that a user doesn't control.
|
||||||
|
A workaround is to always mix-in ``GenericMeta``::
|
||||||
|
|
||||||
|
class AdHocMeta(GenericMeta, LibraryMeta):
|
||||||
|
pass
|
||||||
|
|
||||||
|
class UserClass(LibraryBase, Generic[T], metaclass=AdHocMeta):
|
||||||
|
...
|
||||||
|
|
||||||
|
but this is not always practical or even possible. With the help of the
|
||||||
|
proposed special attributes the ``GenericMeta`` metaclass will not be needed.
|
||||||
|
|
||||||
|
|
||||||
|
Hacks and bugs that will be removed by this proposal:
|
||||||
|
-----------------------------------------------------
|
||||||
|
|
||||||
|
- ``_generic_new`` hack that exists since ``__init__`` is not called on
|
||||||
|
instances with a type differing form the type whose ``__new__`` was called,
|
||||||
|
``C[int]().__class__ is C``.
|
||||||
|
|
||||||
|
- ``_next_in_mro`` speed hack will be not necessary since subscription will
|
||||||
|
not create new classes.
|
||||||
|
|
||||||
|
- Ugly ``sys._getframe`` hack, this one is particularly nasty, since it looks
|
||||||
|
like we can't remove it without changes outside ``typing``.
|
||||||
|
|
||||||
|
- Currently generics do dangerous things with private ABC caches
|
||||||
|
to fix large memory consumption that grows at least as O(N\ :sup:`2`),
|
||||||
|
see [2]_. This point is also important because it was recently proposed to
|
||||||
|
re-implement ``ABCMeta`` in C.
|
||||||
|
|
||||||
|
- Problems with sharing attributes between subscripted generics,
|
||||||
|
see [3]_. Current solution already uses ``__getattr__`` and ``__setattr__``,
|
||||||
|
but it is still incomplete, and solving this without the current proposal
|
||||||
|
will be hard and will need ``__getattribute__``.
|
||||||
|
|
||||||
|
- ``_no_slots_copy`` hack, where we clean-up the class dictionary on every
|
||||||
|
subscription thus allowing generics with ``__slots__``.
|
||||||
|
|
||||||
|
- General complexity of the ``typing`` module, the new proposal will not
|
||||||
|
only allow to remove the above mentioned hacks/bugs, but also simplify
|
||||||
|
the implementation, so that it will be easier to maintain.
|
||||||
|
|
||||||
|
|
||||||
|
Specification
|
||||||
|
=============
|
||||||
|
|
||||||
|
The idea of ``__class_getitem__`` is simple: it is an exact analog of
|
||||||
|
``__getitem__`` with an exception that it is called on a class that
|
||||||
|
defines it, not on its instances, this allows us to avoid
|
||||||
|
``GenericMeta.__getitem__`` for things like ``Iterable[int]``.
|
||||||
|
The ``__class_getitem__`` is automatically a class method and
|
||||||
|
does not require ``@classmethod`` decorator (similar to
|
||||||
|
``__init_subclass__``) and is inherited like normal attributes.
|
||||||
|
For example::
|
||||||
|
|
||||||
|
class MyList:
|
||||||
|
def __getitem__(self, index):
|
||||||
|
return index + 1
|
||||||
|
def __class_getitem__(cls, item):
|
||||||
|
return f"{cls.__name__}[{item.__name__}]"
|
||||||
|
|
||||||
|
class MyOtherList(MyList):
|
||||||
|
pass
|
||||||
|
|
||||||
|
assert MyList()[0] == 1
|
||||||
|
assert MyList[int] == "MyList[int]"
|
||||||
|
|
||||||
|
assert MyOtherList()[0] == 1
|
||||||
|
assert MyOtherList[int] == "MyOtherList[int]"
|
||||||
|
|
||||||
|
Note that this method is used as a fallback, so if a metaclass defines
|
||||||
|
``__getitem__``, then that will have the priority.
|
||||||
|
|
||||||
|
If an object that is not a class object appears in the bases of a class
|
||||||
|
definition, the ``__subclass_base__`` is searched on it. If found,
|
||||||
|
it is called with the original tuple of bases as an argument. If the result
|
||||||
|
of the call is not ``None``, then it is substituted instead of this object.
|
||||||
|
Otherwise (if the result is ``None``), the base is just removed. This is
|
||||||
|
necessary to avoid inconsistent MRO errors, that are currently prevented by
|
||||||
|
manipulations in ``GenericMeta.__new__``. After creating the class,
|
||||||
|
the original bases are saved in ``__orig_bases__`` (currently this is also
|
||||||
|
done by the metaclass).
|
||||||
|
|
||||||
|
NOTE: These two method names are reserved for exclusive use by
|
||||||
|
the ``typing`` module and the generic types machinery, and any other use is
|
||||||
|
strongly discouraged. The reference implementation (with tests) can be found
|
||||||
|
in [4]_, the proposal was originally posted and discussed on
|
||||||
|
the ``typing`` tracker, see [5]_.
|
||||||
|
|
||||||
|
|
||||||
|
Backwards compatibility and impact on users who don't use ``typing``:
|
||||||
|
=====================================================================
|
||||||
|
|
||||||
|
This proposal may break code that currently uses the names
|
||||||
|
``__class_getitem__`` and ``__subclass_base__``.
|
||||||
|
|
||||||
|
This proposal will support almost complete backwards compatibility with
|
||||||
|
the current public generic types API; moreover the ``typing`` module is still
|
||||||
|
provisional. The only two exceptions are that currently
|
||||||
|
``issubclass(List[int], List)`` returns True, with this proposal it will raise
|
||||||
|
``TypeError``. Also ``issubclass(collections.abc.Iterable, typing.Iterable)``
|
||||||
|
will return ``False``, which is probably desirable, since currently we have
|
||||||
|
a (virtual) inheritance cycle between these two classes.
|
||||||
|
|
||||||
|
With the reference implementation I measured negligible performance effects
|
||||||
|
(under 1% on a micro-benchmark) for regular (non-generic) classes.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [1] Discussion following Mark Shannon's presentation at Language Summit
|
||||||
|
(https://github.com/python/typing/issues/432)
|
||||||
|
|
||||||
|
.. [2] Pull Request to implement shared generic ABC caches
|
||||||
|
(https://github.com/python/typing/pull/383)
|
||||||
|
|
||||||
|
.. [3] An old bug with setting/accessing attributes on generic types
|
||||||
|
(https://github.com/python/typing/issues/392)
|
||||||
|
|
||||||
|
.. [4] The reference implementation
|
||||||
|
(https://github.com/ilevkivskyi/cpython/pull/2/files)
|
||||||
|
|
||||||
|
.. [5] Original proposal
|
||||||
|
(https://github.com/python/typing/issues/468)
|
||||||
|
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
|
@ -0,0 +1,179 @@
|
||||||
|
PEP: 561
|
||||||
|
Title: Distributing and Packaging Type Information
|
||||||
|
Author: Ethan Smith <ethan@ethanhs.me>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 09-Sep-2017
|
||||||
|
Python-Version: 3.7
|
||||||
|
Post-History:
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
PEP 484 introduced type hints to Python, with goals of making typing
|
||||||
|
gradual and easy to adopt. Currently, typing information must be distributed
|
||||||
|
manually. This PEP provides a standardized means to package and distribute
|
||||||
|
type information and an ordering for type checkers to resolve modules and
|
||||||
|
collect this information for type checking using existing packaging
|
||||||
|
architecture.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale
|
||||||
|
=========
|
||||||
|
|
||||||
|
PEP 484 has a brief section on distributing typing information. In this
|
||||||
|
section [1]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
|
||||||
|
shipping stub files. However, manually adding a path to stub files for each
|
||||||
|
third party library does not scale. The simplest approach people have taken
|
||||||
|
is to add ``site-packages`` to their ``PYTHONPATH``, but this causes type
|
||||||
|
checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
|
||||||
|
and Django).
|
||||||
|
|
||||||
|
Furthermore, package authors are wishing to distribute code that has
|
||||||
|
inline type information, and there currently is no standard method to
|
||||||
|
distribute packages with inline type annotations or syntax that can
|
||||||
|
simultaneously be used at runtime and in type checking.
|
||||||
|
|
||||||
|
|
||||||
|
Specification
|
||||||
|
=============
|
||||||
|
|
||||||
|
There are several motivations and methods of supporting typing in a package. This PEP recognizes three (3) types of packages that may be created:
|
||||||
|
|
||||||
|
1. The package maintainer would like to add type information inline.
|
||||||
|
|
||||||
|
2. The package maintainer would like to add type information via stubs.
|
||||||
|
|
||||||
|
3. A third party would like to share stub files for a package, but the
|
||||||
|
maintainer does not want to include them in the source of the package.
|
||||||
|
|
||||||
|
This PEP aims to support these scenarios and make them simple to add to
|
||||||
|
packaging and deployment.
|
||||||
|
|
||||||
|
The two major parts of this specification are the packaging specifications
|
||||||
|
and the resolution order for resolving module type information. This spec
|
||||||
|
is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [1]_.
|
||||||
|
|
||||||
|
Packaging Type Information
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
Packages must opt into supporting typing. This will be done though a distutils
|
||||||
|
extension [2]_, providing a ``typed`` keyword argument to the distutils
|
||||||
|
``setup()`` command. The argument value will depend on the kind of type
|
||||||
|
information the package provides. The distutils extension will be added to the
|
||||||
|
``typing`` package. Therefore a package maintainer may write
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
setup(
|
||||||
|
...
|
||||||
|
setup_requires=["typing"],
|
||||||
|
typed="inline",
|
||||||
|
...
|
||||||
|
)
|
||||||
|
|
||||||
|
Inline Typed Packages
|
||||||
|
'''''''''''''''''''''
|
||||||
|
|
||||||
|
Packages that have inline type annotations simply have to pass the value
|
||||||
|
``"inline"`` to the ``typed`` argument in ``setup()``.
|
||||||
|
|
||||||
|
Stub Only Packages
|
||||||
|
''''''''''''''''''
|
||||||
|
|
||||||
|
For package maintainers wishing to ship stub files containing all of their
|
||||||
|
type information, it is prefered that the ``*.pyi`` stubs are alongside the
|
||||||
|
corresponding ``*.py`` files. However, the stubs may be put in a sub-folder
|
||||||
|
of the Python sources, with the same name the ``*.py`` files are in. For
|
||||||
|
example, the ``flyingcircus`` package would have its stubs in the folder
|
||||||
|
``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are
|
||||||
|
not found in ``flyingcircus/`` the type checker may treat the subdirectory as
|
||||||
|
a normal package. The normal resolution order of checking ``*.pyi`` before
|
||||||
|
``*.py`` will be maintained. The value of the ``typed`` argument to
|
||||||
|
``setup()`` is ``"stubs"`` for this type of distribution. The author of the
|
||||||
|
package is suggested to use ``package_data`` to assure the stub files are
|
||||||
|
installed alongside the runtime Python code.
|
||||||
|
|
||||||
|
Third Party Stub Packages
|
||||||
|
'''''''''''''''''''''''''
|
||||||
|
|
||||||
|
Third parties seeking to distribute stub files are encouraged to contact the
|
||||||
|
maintainer of the package about distribution alongside the package. If the
|
||||||
|
maintainer does not wish to maintain or package stub files or type information
|
||||||
|
inline, then a "third party stub package" should be created. The structure is
|
||||||
|
similar, but slightly different from that of stub only packages. If the stubs
|
||||||
|
are for the library ``flyingcircus`` then the package should be named
|
||||||
|
``flyingcircus-stubs`` and the stub files should be put in a sub-directory
|
||||||
|
named ``flyingcircus``. This allows the stubs to be checked as if they were in
|
||||||
|
a regular package. These packages should also pass ``"stubs"`` as the value
|
||||||
|
of ``typed`` argument in ``setup()``. These packages are suggested to use
|
||||||
|
``package_data`` to package stub files.
|
||||||
|
|
||||||
|
The version of the ``flyingcircus-stubs`` package should match the version of
|
||||||
|
the ``flyingcircus`` package it is providing types for.
|
||||||
|
|
||||||
|
Type Checker Module Resolution Order
|
||||||
|
------------------------------------
|
||||||
|
|
||||||
|
The following is the order that type checkers supporting this PEP should
|
||||||
|
resolve modules containing type information:
|
||||||
|
|
||||||
|
1. User code - the files the type checker is running on.
|
||||||
|
|
||||||
|
2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user
|
||||||
|
complete control of which stubs to use, and patch broken stubs/inline
|
||||||
|
types from packages.
|
||||||
|
|
||||||
|
3. Third party stub packages - these packages can supersede the installed
|
||||||
|
untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
|
||||||
|
however it is encouraged to check their metadata to confirm that they opt
|
||||||
|
into type checking.
|
||||||
|
|
||||||
|
4. Inline packages - finally, if there is nothing overriding the installed
|
||||||
|
package, and it opts into type checking.
|
||||||
|
|
||||||
|
5. Typeshed (if used) - Provides the stdlib types and several third party libraries
|
||||||
|
|
||||||
|
When resolving step (3) type checkers should assure the version of the stubs
|
||||||
|
match the installed runtime package.
|
||||||
|
|
||||||
|
Type checkers that check a different Python version than the version they run
|
||||||
|
on must find the type information in the ``site-packages``/``dist-packages``
|
||||||
|
of that Python version. This can be queried e.g.
|
||||||
|
``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended
|
||||||
|
that the type checker allow for the user to point to a particular Python
|
||||||
|
binary, in case it is not in the path.
|
||||||
|
|
||||||
|
To check if a package has opted into type checking, type checkers are
|
||||||
|
recommended to use the ``pkg_resources`` module to query the package
|
||||||
|
metadata. If the ``typed`` package metadata has ``None`` as its value, the
|
||||||
|
package has not opted into type checking, and the type checker should skip that
|
||||||
|
package.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [1] PEP 484, Storing and Distributing Stub Files
|
||||||
|
(https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files)
|
||||||
|
|
||||||
|
.. [2] Distutils Extensions, Adding setup() arguments
|
||||||
|
(http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arguments)
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
|
@ -0,0 +1,105 @@
|
||||||
|
PEP: 562
|
||||||
|
Title: Module __getattr__
|
||||||
|
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 09-Sep-2017
|
||||||
|
Python-Version: 3.7
|
||||||
|
Post-History: 09-Sep-2017
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
It is proposed to support ``__getattr__`` function defined on modules to
|
||||||
|
provide basic customization of module attribute access.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale
|
||||||
|
=========
|
||||||
|
|
||||||
|
It is sometimes convenient to customize or otherwise have control over
|
||||||
|
access to module attributes. A typical example is managing deprecation
|
||||||
|
warnings. Typical workarounds are assigning ``__class__`` of a module object
|
||||||
|
to a custom subclass of ``types.ModuleType`` or substituting ``sys.modules``
|
||||||
|
item with a custom wrapper instance. It would be convenient to simplify this
|
||||||
|
procedure by recognizing ``__getattr__`` defined directly in a module that
|
||||||
|
would act like a normal ``__getattr__`` method, except that it will be defined
|
||||||
|
on module *instances*. For example::
|
||||||
|
|
||||||
|
# lib.py
|
||||||
|
|
||||||
|
from warnings import warn
|
||||||
|
|
||||||
|
deprecated_names = ["old_function", ...]
|
||||||
|
|
||||||
|
def _deprecated_old_function(arg, other):
|
||||||
|
...
|
||||||
|
|
||||||
|
def __getattr__(name):
|
||||||
|
if name in deprecated_names:
|
||||||
|
warn(f"{name} is deprecated", DeprecationWarning)
|
||||||
|
return globals()[f"_deprecated_{name}"]
|
||||||
|
raise AttributeError(f"module {__name__} has no attribute {name}")
|
||||||
|
|
||||||
|
# main.py
|
||||||
|
|
||||||
|
from lib import old_function # Works, but emits the warning
|
||||||
|
|
||||||
|
There is a related proposal PEP 549 that proposes to support instance
|
||||||
|
properties for a similar functionality. The difference is this PEP proposes
|
||||||
|
a faster and simpler mechanism, but provides more basic customization.
|
||||||
|
An additional motivation for this proposal is that PEP 484 already defines
|
||||||
|
the use of module ``__getattr__`` for this purpose in Python stub files,
|
||||||
|
see [1]_.
|
||||||
|
|
||||||
|
|
||||||
|
Specification
|
||||||
|
=============
|
||||||
|
|
||||||
|
The ``__getattr__`` function at the module level should accept one argument
|
||||||
|
which is a name of an attribute and return the computed value or raise
|
||||||
|
an ``AttributeError``::
|
||||||
|
|
||||||
|
def __getattr__(name: str) -> Any: ...
|
||||||
|
|
||||||
|
This function will be called only if ``name`` is not found in the module
|
||||||
|
through the normal attribute lookup.
|
||||||
|
|
||||||
|
The reference implementation for this PEP can be found in [2]_.
|
||||||
|
|
||||||
|
|
||||||
|
Backwards compatibility and impact on performance
|
||||||
|
=================================================
|
||||||
|
|
||||||
|
This PEP may break code that uses module level (global) name ``__getattr__``.
|
||||||
|
The performance implications of this PEP are minimal, since ``__getattr__``
|
||||||
|
is called only for missing attributes.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [1] PEP 484 section about ``__getattr__`` in stub files
|
||||||
|
(https://www.python.org/dev/peps/pep-0484/#stub-files)
|
||||||
|
|
||||||
|
.. [2] The reference implementation
|
||||||
|
(https://github.com/ilevkivskyi/cpython/pull/3/files)
|
||||||
|
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
|
@ -0,0 +1,264 @@
|
||||||
|
PEP: 563
|
||||||
|
Title: Postponed Evaluation of Annotations
|
||||||
|
Version: $Revision$
|
||||||
|
Last-Modified: $Date$
|
||||||
|
Author: Łukasz Langa <lukasz@langa.pl>
|
||||||
|
Discussions-To: Python-Dev <python-dev@python.org>
|
||||||
|
Status: Draft
|
||||||
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
|
Created: 8-Sep-2017
|
||||||
|
Python-Version: 3.7
|
||||||
|
Post-History:
|
||||||
|
Resolution:
|
||||||
|
|
||||||
|
|
||||||
|
Abstract
|
||||||
|
========
|
||||||
|
|
||||||
|
PEP 3107 introduced syntax for function annotations, but the semantics
|
||||||
|
were deliberately left undefined. PEP 484 introduced a standard meaning
|
||||||
|
to annotations: type hints. PEP 526 defined variable annotations,
|
||||||
|
explicitly tying them with the type hinting use case.
|
||||||
|
|
||||||
|
This PEP proposes changing function annotations and variable annotations
|
||||||
|
so that they are no longer evaluated at function definition time.
|
||||||
|
Instead, they are preserved in ``__annotations__`` in string form.
|
||||||
|
|
||||||
|
This change is going to be introduced gradually, starting with a new
|
||||||
|
``__future__`` import in Python 3.7.
|
||||||
|
|
||||||
|
|
||||||
|
Rationale and Goals
|
||||||
|
===================
|
||||||
|
|
||||||
|
PEP 3107 added support for arbitrary annotations on parts of a function
|
||||||
|
definition. Just like default values, annotations are evaluated at
|
||||||
|
function definition time. This creates a number of issues for the type
|
||||||
|
hinting use case:
|
||||||
|
|
||||||
|
* forward references: when a type hint contains names that have not been
|
||||||
|
defined yet, that definition needs to be expressed as a string
|
||||||
|
literal;
|
||||||
|
|
||||||
|
* type hints are executed at module import time, which is not
|
||||||
|
computationally free.
|
||||||
|
|
||||||
|
Postponing the evaluation of annotations solves both problems.
|
||||||
|
|
||||||
|
Non-goals
|
||||||
|
---------
|
||||||
|
|
||||||
|
Just like in PEP 484 and PEP 526, it should be emphasized that **Python
|
||||||
|
will remain a dynamically typed language, and the authors have no desire
|
||||||
|
to ever make type hints mandatory, even by convention.**
|
||||||
|
|
||||||
|
Annotations are still available for arbitrary use besides type checking.
|
||||||
|
Using ``@typing.no_type_hints`` in this case is recommended to
|
||||||
|
disambiguate the use case.
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
In a future version of Python, function and variable annotations will no
|
||||||
|
longer be evaluated at definition time. Instead, a string form will be
|
||||||
|
preserved in the respective ``__annotations__`` dictionary. Static type
|
||||||
|
checkers will see no difference in behavior, whereas tools using
|
||||||
|
annotations at runtime will have to perform postponed evaluation.
|
||||||
|
|
||||||
|
If an annotation was already a string, this string is preserved
|
||||||
|
verbatim. In other cases, the string form is obtained from the AST
|
||||||
|
during the compilation step, which means that the string form preserved
|
||||||
|
might not preserve the exact formatting of the source.
|
||||||
|
|
||||||
|
Annotations need to be syntactically valid Python expressions, also when
|
||||||
|
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
|
||||||
|
Annotations can only use names present in the module scope as postponed
|
||||||
|
evaluation using local names is not reliable.
|
||||||
|
|
||||||
|
Note that as per PEP 526, local variable annotations are not evaluated
|
||||||
|
at all since they are not accessible outside of the function's closure.
|
||||||
|
|
||||||
|
Enabling the future behavior in Python 3.7
|
||||||
|
------------------------------------------
|
||||||
|
|
||||||
|
The functionality described above can be enabled starting from Python
|
||||||
|
3.7 using the following special import::
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
Resolving Type Hints at Runtime
|
||||||
|
===============================
|
||||||
|
|
||||||
|
To resolve an annotation at runtime from its string form to the result
|
||||||
|
of the enclosed expression, user code needs to evaluate the string.
|
||||||
|
|
||||||
|
For code that uses type hints, the ``typing.get_type_hints()`` function
|
||||||
|
correctly evaluates expressions back from its string form. Note that
|
||||||
|
all valid code currently using ``__annotations__`` should already be
|
||||||
|
doing that since a type annotation can be expressed as a string literal.
|
||||||
|
|
||||||
|
For code which uses annotations for other purposes, a regular
|
||||||
|
``eval(ann, globals, locals)`` call is enough to resolve the
|
||||||
|
annotation. The trick here is to get the correct value for globals.
|
||||||
|
Fortunately, in the case of functions, they hold a reference to globals
|
||||||
|
in an attribute called ``__globals__``. To get the correct module-level
|
||||||
|
context to resolve class variables, use::
|
||||||
|
|
||||||
|
cls_globals = sys.modules[SomeClass.__module__].__dict__
|
||||||
|
|
||||||
|
Runtime annotation resolution and class decorators
|
||||||
|
--------------------------------------------------
|
||||||
|
|
||||||
|
Metaclasses and class decorators that need to resolve annotations for
|
||||||
|
the current class will fail for annotations that use the name of the
|
||||||
|
current class. Example::
|
||||||
|
|
||||||
|
def class_decorator(cls):
|
||||||
|
annotations = get_type_hints(cls) # raises NameError on 'C'
|
||||||
|
print(f'Annotations for {cls}: {annotations}')
|
||||||
|
return cls
|
||||||
|
|
||||||
|
@class_decorator
|
||||||
|
class C:
|
||||||
|
singleton: 'C' = None
|
||||||
|
|
||||||
|
This was already true before this PEP. The class decorator acts on
|
||||||
|
the class before it's assigned a name in the current definition scope.
|
||||||
|
|
||||||
|
The situation is made somewhat stricter when class-level variables are
|
||||||
|
considered. Previously, when the string form wasn't used in annotations,
|
||||||
|
a class decorator would be able to cover situations like::
|
||||||
|
|
||||||
|
@class_decorator
|
||||||
|
class Restaurant:
|
||||||
|
class MenuOption(Enum):
|
||||||
|
SPAM = 1
|
||||||
|
EGGS = 2
|
||||||
|
|
||||||
|
default_menu: List[MenuOption] = []
|
||||||
|
|
||||||
|
This is no longer possible.
|
||||||
|
|
||||||
|
Runtime annotation resolution and ``TYPE_CHECKING``
|
||||||
|
---------------------------------------------------
|
||||||
|
|
||||||
|
Sometimes there's code that must be seen by a type checker but should
|
||||||
|
not be executed. For such situations the ``typing`` module defines a
|
||||||
|
constant, ``TYPE_CHECKING``, that is considered ``True`` during type
|
||||||
|
checking but ``False`` at runtime. Example::
|
||||||
|
|
||||||
|
import typing
|
||||||
|
|
||||||
|
if typing.TYPE_CHECKING:
|
||||||
|
import expensive_mod
|
||||||
|
|
||||||
|
def a_func(arg: expensive_mod.SomeClass) -> None:
|
||||||
|
a_var: expensive_mod.SomeClass = arg
|
||||||
|
...
|
||||||
|
|
||||||
|
This approach is also useful when handling import cycles.
|
||||||
|
|
||||||
|
Trying to resolve annotations of ``a_func`` at runtime using
|
||||||
|
``typing.get_type_hints()`` will fail since the name ``expensive_mod``
|
||||||
|
is not defined (``TYPE_CHECKING`` variable being ``False`` at runtime).
|
||||||
|
This was already true before this PEP.
|
||||||
|
|
||||||
|
|
||||||
|
Backwards Compatibility
|
||||||
|
=======================
|
||||||
|
|
||||||
|
This is a backwards incompatible change. Applications depending on
|
||||||
|
arbitrary objects to be directly present in annotations will break
|
||||||
|
if they are not using ``typing.get_type_hints()`` or ``eval()``.
|
||||||
|
|
||||||
|
Annotations that depend on locals at the time of the function/class
|
||||||
|
definition are now invalid. Example::
|
||||||
|
|
||||||
|
def generate_class():
|
||||||
|
some_local = datetime.datetime.now()
|
||||||
|
class C:
|
||||||
|
field: some_local = 1 # NOTE: INVALID ANNOTATION
|
||||||
|
def method(self, arg: some_local.day) -> None: # NOTE: INVALID ANNOTATION
|
||||||
|
...
|
||||||
|
|
||||||
|
Annotations using nested classes and their respective state are still
|
||||||
|
valid, provided they use the fully qualified name. Example::
|
||||||
|
|
||||||
|
class C:
|
||||||
|
field = 'c_field'
|
||||||
|
def method(self, arg: C.field) -> None: # this is OK
|
||||||
|
...
|
||||||
|
|
||||||
|
class D:
|
||||||
|
field2 = 'd_field'
|
||||||
|
def method(self, arg: C.field -> C.D.field2: # this is OK
|
||||||
|
...
|
||||||
|
|
||||||
|
In the presence of an annotation that cannot be resolved using the
|
||||||
|
current module's globals, a NameError is raised at compile time.
|
||||||
|
|
||||||
|
|
||||||
|
Deprecation policy
|
||||||
|
------------------
|
||||||
|
|
||||||
|
In Python 3.7, a ``__future__`` import is required to use the described
|
||||||
|
functionality and a ``PendingDeprecationWarning`` is raised by the
|
||||||
|
compiler in the presence of type annotations in modules without the
|
||||||
|
``__future__`` import. In Python 3.8 the warning becomes a
|
||||||
|
``DeprecationWarning``. In the next version this will become the
|
||||||
|
default behavior.
|
||||||
|
|
||||||
|
|
||||||
|
Rejected Ideas
|
||||||
|
==============
|
||||||
|
|
||||||
|
Keep the ability to use local state when defining annotations
|
||||||
|
-------------------------------------------------------------
|
||||||
|
|
||||||
|
With postponed evaluation, this is impossible for function locals. For
|
||||||
|
classes, it would be possible to keep the ability to define annotations
|
||||||
|
using the local scope. However, when using ``eval()`` to perform the
|
||||||
|
postponed evaluation, we need to provide the correct globals and locals
|
||||||
|
to the ``eval()`` call. In the face of nested classes, the routine to
|
||||||
|
get the effective "globals" at definition time would have to look
|
||||||
|
something like this::
|
||||||
|
|
||||||
|
def get_class_globals(cls):
|
||||||
|
result = {}
|
||||||
|
result.update(sys.modules[cls.__module__].__dict__)
|
||||||
|
for child in cls.__qualname__.split('.'):
|
||||||
|
result.update(result[child].__dict__)
|
||||||
|
return result
|
||||||
|
|
||||||
|
This is brittle and doesn't even cover slots. Requiring the use of
|
||||||
|
module-level names simplifies runtime evaluation and provides the
|
||||||
|
"one obvious way" to read annotations. It's the equivalent of absolute
|
||||||
|
imports.
|
||||||
|
|
||||||
|
|
||||||
|
Acknowledgements
|
||||||
|
================
|
||||||
|
|
||||||
|
This document could not be completed without valuable input,
|
||||||
|
encouragement and advice from Guido van Rossum, Jukka Lehtosalo, and
|
||||||
|
Ivan Levkivskyi.
|
||||||
|
|
||||||
|
|
||||||
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
|
Local Variables:
|
||||||
|
mode: indented-text
|
||||||
|
indent-tabs-mode: nil
|
||||||
|
sentence-end-double-space: t
|
||||||
|
fill-column: 70
|
||||||
|
coding: utf-8
|
||||||
|
End:
|
Loading…
Reference in New Issue