PEP 461: Distributing and Packaging Type Information (#415)

pep-0561.rst

@@ -0,0 +1,179 @@
+PEP: 561 
+Title: Distributing and Packaging Type Information
+Author: Ethan Smith <ethan@ethanhs.me>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 09-Sep-2017
+Python-Version: 3.7
+Post-History: 
+
+
+Abstract
+========
+
+PEP 484 introduced type hints to Python, with goals of making typing
+gradual and easy to adopt. Currently, typing information must be distributed 
+manually. This PEP provides a standardized means to package and distribute
+type information and an ordering for type checkers to resolve modules and 
+collect this information for type checking using existing packaging
+architecture.
+
+
+Rationale
+=========
+
+PEP 484 has a brief section on distributing typing information. In this
+section [1]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
+shipping stub files. However, manually adding a path to stub files for each
+third party library does not scale. The simplest approach people have taken
+is to add ``site-packages`` to their ``PYTHONPATH``, but this causes type
+checkers to fail on packages that are highly dynamic (e.g. sqlalchemy 
+and Django).
+
+Furthermore, package authors are wishing to distribute code that has
+inline type information, and there currently is no standard method to
+distribute packages with inline type annotations or syntax that can
+simultaneously be used at runtime and in type checking.
+
+
+Specification
+=============
+
+There are several motivations and methods of supporting typing in a package.                                                 This PEP recognizes three (3) types of packages that may be created:
+
+1. The package maintainer would like to add type information inline.
+
+2. The package maintainer would like to add type information via stubs.
+
+3. A third party would like to share stub files for a package, but the
+   maintainer does not want to include them in the source of the package.
+   
+This PEP aims to support these scenarios and make them simple to add to
+packaging and deployment.
+
+The two major parts of this specification are the packaging specifications
+and the resolution order for resolving module type information. This spec
+is meant to replace the ``shared/typehints/pythonX.Y/`` spec of PEP 484 [1]_. 
+
+Packaging Type Information
+--------------------------
+
+Packages must opt into supporting typing. This will be done though a distutils
+extension [2]_, providing a ``typed`` keyword argument to the distutils
+``setup()`` command. The argument value will depend on the kind of type
+information the package provides. The distutils extension will be added to the
+``typing`` package. Therefore a package maintainer may write
+
+::
+
+    setup(
+        ...
+        setup_requires=["typing"],
+        typed="inline",
+        ...
+    )
+
+Inline Typed Packages
+'''''''''''''''''''''
+
+Packages that have inline type annotations simply have to pass the value
+``"inline"`` to the ``typed`` argument in ``setup()``.
+
+Stub Only Packages
+''''''''''''''''''
+
+For package maintainers wishing to ship stub files containing all of their
+type information, it is prefered that the ``*.pyi`` stubs are alongside the
+corresponding ``*.py`` files. However, the stubs may be put in a sub-folder
+of the Python sources, with the same name the ``*.py`` files are in. For 
+example, the ``flyingcircus`` package would have its stubs in the folder
+``flyingcircus/flyingcircus/``. This path is chosen so that if stubs are
+not found in ``flyingcircus/`` the type checker may treat the subdirectory as
+a normal package. The normal resolution order of checking ``*.pyi`` before
+``*.py`` will be maintained. The value of the ``typed`` argument to 
+``setup()`` is ``"stubs"`` for this type of distribution. The author of the
+package is suggested to use ``package_data`` to assure the stub files are
+installed alongside the runtime Python code.
+
+Third Party Stub Packages
+'''''''''''''''''''''''''
+
+Third parties seeking to distribute stub files are encouraged to contact the
+maintainer of the package about distribution alongside the package. If the
+maintainer does not wish to maintain or package stub files or type information
+inline, then a "third party stub package" should be created. The structure is
+similar, but slightly different from that of stub only packages. If the stubs
+are for the library ``flyingcircus`` then the package should be named 
+``flyingcircus-stubs`` and the stub files should be put in a sub-directory
+named ``flyingcircus``. This allows the stubs to be checked as if they were in
+a regular package. These packages should also pass ``"stubs"`` as the value 
+of ``typed`` argument in ``setup()``. These packages are suggested to use
+``package_data`` to package stub files.
+
+The version of the ``flyingcircus-stubs`` package should match the version of
+the ``flyingcircus`` package it is providing types for.
+
+Type Checker Module Resolution Order
+------------------------------------
+
+The following is the order that type checkers supporting this PEP should
+resolve modules containing type information:
+
+1. User code - the files the type checker is running on.
+
+2. Stubs or Python source in ``PYTHONPATH``. This is to allow the user
+   complete control of which stubs to use, and patch broken stubs/inline
+   types from packages.
+
+3. Third party stub packages - these packages can supersede the installed
+   untyped packages. They can be found at ``pkg-stubs`` for package ``pkg``,
+   however it is encouraged to check their metadata to confirm that they opt
+   into type checking.
+
+4. Inline packages - finally, if there is nothing overriding the installed
+   package, and it opts into type checking.
+
+5. Typeshed (if used) - Provides the stdlib types and several third party libraries
+
+When resolving step (3) type checkers should assure the version of the stubs
+match the installed runtime package.
+
+Type checkers that check a different Python version than the version they run
+on must find the type information in the ``site-packages``/``dist-packages``
+of that Python version. This can be queried e.g.
+``pythonX.Y -c 'import sys; print(sys.exec_prefix)'``. It is also recommended
+that the type checker allow for the user to point to a particular Python
+binary, in case it is not in the path.
+
+To check if a package has opted into type checking, type checkers are
+recommended to use the ``pkg_resources`` module to query the package
+metadata. If the ``typed`` package metadata has ``None`` as its value, the
+package has not opted into type checking, and the type checker should skip that
+package.
+
+
+References
+==========
+
+.. [1] PEP 484, Storing and Distributing Stub Files
+   (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files)
+
+.. [2] Distutils Extensions, Adding setup() arguments
+   (http://setuptools.readthedocs.io/en/latest/setuptools.html#adding-setup-arguments)
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End: