225 lines
8.5 KiB
ReStructuredText
225 lines
8.5 KiB
ReStructuredText
PEP: 561
|
||
Title: Distributing and Packaging Type Information
|
||
Author: Ethan Smith <ethan@ethanhs.me>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 09-Sep-2017
|
||
Python-Version: 3.7
|
||
Post-History:
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
PEP 484 introduced type hinting to Python, with goals of making typing
|
||
gradual and easy to adopt. Currently, typing information must be distributed
|
||
manually. This PEP provides a standardized means to package and distribute
|
||
type information and an ordering for type checkers to resolve modules and
|
||
collect this information for type checking using existing packaging
|
||
architecture.
|
||
|
||
|
||
Rationale
|
||
=========
|
||
|
||
Currently, package authors wish to distribute code that has
|
||
inline type information. However, there is no standard method to distribute
|
||
packages with inline type annotations or syntax that can simultaneously
|
||
be used at runtime and in type checking. Additionally, if one wished to
|
||
ship typing information privately the only method would be via setting
|
||
``MYPYPATH`` or the equivalent to manually point to stubs. If the package
|
||
can be released publicly, it can be added to typeshed [1]_. However, this
|
||
does not scale and becomes a burden on the maintainers of typeshed.
|
||
Additionally, it ties bugfixes to releases of the tool using typeshed.
|
||
|
||
PEP 484 has a brief section on distributing typing information. In this
|
||
section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
|
||
shipping stub files. However, manually adding a path to stub files for each
|
||
third party library does not scale. The simplest approach people have taken
|
||
is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
|
||
checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
|
||
and Django).
|
||
|
||
|
||
Specification
|
||
=============
|
||
|
||
There are several motivations and methods of supporting typing in a package.
|
||
This PEP recognizes three (3) types of packages that may be created:
|
||
|
||
1. The package maintainer would like to add type information inline.
|
||
|
||
2. The package maintainer would like to add type information via stubs.
|
||
|
||
3. A third party would like to share stub files for a package, but the
|
||
maintainer does not want to include them in the source of the package.
|
||
|
||
This PEP aims to support these scenarios and make them simple to add to
|
||
packaging and deployment.
|
||
|
||
The two major parts of this specification are the packaging specifications
|
||
and the resolution order for resolving module type information. The packaging
|
||
spec is based on and extends PEP 345 metadata. The type checking spec is
|
||
meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [2]_.
|
||
|
||
New third party stub libraries are encouraged to distribute stubs via the
|
||
third party packaging proposed in this PEP in place of being added to
|
||
typeshed. Typeshed will remain in use, but if maintainers are found, third
|
||
party stubs in typeshed are encouraged to be split into their own package.
|
||
|
||
Packaging Type Information
|
||
--------------------------
|
||
In order to make packaging and distributing type information as simple and
|
||
easy as possible, the distribution of type information, and typed Python code
|
||
is done through existing packaging frameworks. This PEP adds a new item to the
|
||
``*.distinfo/METADATA`` file to contain metadata about a package's support for
|
||
typing. The new item is optional, but must have a name of ``Typed`` and have a
|
||
value of either ``inline`` or ``stubs``, if present.
|
||
|
||
Metadata Examples::
|
||
|
||
Typed: inline
|
||
Typed: stubs
|
||
|
||
|
||
Stub Only Packages
|
||
''''''''''''''''''
|
||
|
||
For package maintainers wishing to ship stub files containing all of their
|
||
type information, it is prefered that the ``*.pyi`` stubs are alongside the
|
||
corresponding ``*.py`` files. However, the stubs may be put in a sub-folder
|
||
of the Python sources, with the same name the ``*.py`` files are in. For
|
||
example, the ``flyingcircus`` package would have its stubs in the folder
|
||
``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are
|
||
not found in ``flyingcircus/`` the type checker may treat the subdirectory as
|
||
a normal package. The normal resolution order of checking ``*.pyi`` before
|
||
``*.py`` will be maintained.
|
||
|
||
Third Party Stub Packages
|
||
'''''''''''''''''''''''''
|
||
|
||
Third parties seeking to distribute stub files are encouraged to contact the
|
||
maintainer of the package about distribution alongside the package. If the
|
||
maintainer does not wish to maintain or package stub files or type information
|
||
inline, then a "third party stub package" should be created. The structure is
|
||
similar, but slightly different from that of stub only packages. If the stubs
|
||
are for the library ``flyingcircus`` then the package should be named
|
||
``flyingcircus-stubs`` and the stub files should be put in a sub-directory
|
||
named ``flyingcircus``. This allows the stubs to be checked as if they were in
|
||
a regular package.
|
||
|
||
In addition, the third party stub package should indicate which version(s)
|
||
of the runtime package are supported by indicating the runtime package's
|
||
version(s) through the normal dependency data. For example, if there was a
|
||
stub package ``flyingcircus-stubs``, it can indicate the versions of the
|
||
runtime ``flyingcircus`` package supported through ``install_requires``
|
||
in distutils based tools, or the equivalent in other packaging tools.
|
||
|
||
Type Checker Module Resolution Order
|
||
------------------------------------
|
||
|
||
The following is the order that type checkers supporting this PEP should
|
||
resolve modules containing type information:
|
||
|
||
1. User code - the files the type checker is running on.
|
||
|
||
2. Stubs or Python source manually put in the beginning of the path. Type
|
||
checkers should provide this to allow the user complete control of which
|
||
stubs to use, and patch broken stubs/inline types from packages.
|
||
|
||
3. Third party stub packages - these packages can supersede the installed
|
||
untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
|
||
however it is encouraged to check the package's metadata using packaging
|
||
query APIs such as ``pkg_resources`` to assure that the package is meant
|
||
for type checking, and is compatible with the installed version.
|
||
|
||
4. Inline packages - finally, if there is nothing overriding the installed
|
||
package, and it opts into type checking.
|
||
|
||
5. Typeshed (if used) - Provides the stdlib types and several third party
|
||
libraries
|
||
|
||
Type checkers that check a different Python version than the version they run
|
||
on must find the type information in the ``site-packages``/``dist-packages``
|
||
of that Python version. This can be queried e.g.
|
||
``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also recommended
|
||
that the type checker allow for the user to point to a particular Python
|
||
binary, in case it is not in the path.
|
||
|
||
To check if a package has opted into type checking, type checkers are
|
||
recommended to use the ``pkg_resources`` module to query the package
|
||
metadata. If the ``typed`` package metadata has ``None`` as its value, the
|
||
package has not opted into type checking, and the type checker should skip
|
||
that package.
|
||
|
||
|
||
Implementation
|
||
==============
|
||
|
||
A CPython branch with a modified distutils supporting the ``typed`` setup
|
||
keyword lives here: [impl]_. In addition, a sample package with inline types is
|
||
available [typed_pkg]_, as well as a sample package [pkg_checker]_ which reads the metadata
|
||
of installed packages and reports on their status as either not typed, inline
|
||
typed, or a stub package.
|
||
|
||
|
||
Acknowledgements
|
||
================
|
||
|
||
This PEP would not have been possible without the ideas, feedback, and support
|
||
of Ivan Levkivskyi, Jelle Zijlstra, Nick Coghlan, Daniel F Moisset, and
|
||
Guido van Rossum.
|
||
|
||
|
||
Version History
|
||
===============
|
||
|
||
* 2017-10-26
|
||
|
||
* Added implementation references.
|
||
* Added acknowledgements and version history.
|
||
|
||
* 2017-10-06
|
||
|
||
* Rewritten to use .distinfo/METADATA over a distutils specific command.
|
||
* Clarify versioning of third party stub packages.
|
||
|
||
* 2017-09-11
|
||
|
||
* Added information about current solutions and typeshed.
|
||
* Clarify rationale.
|
||
|
||
|
||
References
|
||
==========
|
||
.. [1] Typeshed (https://github.com/python/typeshed)
|
||
|
||
.. [2] PEP 484, Storing and Distributing Stub Files
|
||
(https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files)
|
||
|
||
.. [impl] CPython sample implementation
|
||
(https://github.com/ethanhs/cpython/tree/typeddist)
|
||
|
||
.. [typed_pkg] Sample typed package
|
||
(https://github.com/ethanhs/sample-typed-package)
|
||
|
||
.. [pkg_checker] Sample package checker
|
||
(https://github.com/ethanhs/check_typedpkg)
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|