318 lines
13 KiB
Plaintext
318 lines
13 KiB
Plaintext
PEP: 408
|
|
Title: Standard library __preview__ package
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Nick Coghlan <ncoghlan@gmail.com>,
|
|
Eli Bendersky <eliben@gmail.com>
|
|
Status: Rejected
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 07-Jan-2012
|
|
Python-Version: 3.3
|
|
Post-History: 2012-01-27
|
|
Resolution: https://mail.python.org/pipermail/python-dev/2012-January/115962.html
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
The process of including a new module into the Python standard library is
|
|
hindered by the API lock-in and promise of backward compatibility implied by
|
|
a module being formally part of Python. This PEP proposes a transitional
|
|
state for modules - inclusion in a special ``__preview__`` package for the
|
|
duration of a minor release (roughly 18 months) prior to full acceptance into
|
|
the standard library. On one hand, this state provides the module with the
|
|
benefits of being formally part of the Python distribution. On the other hand,
|
|
the core development team explicitly states that no promises are made with
|
|
regards to the module's eventual full inclusion into the standard library,
|
|
or to the stability of its API, which may change for the next release.
|
|
|
|
|
|
PEP Rejection
|
|
=============
|
|
|
|
Based on his experience with a similar "labs" namespace in Google App Engine,
|
|
Guido has rejected this PEP [3]_ in favour of the simpler alternative of
|
|
explicitly marking provisional modules as such in their documentation.
|
|
|
|
If a module is otherwise considered suitable for standard library inclusion,
|
|
but some concerns remain regarding maintainability or certain API details,
|
|
then the module can be accepted on a provisional basis. While it is considered
|
|
an unlikely outcome, such modules *may* be removed from the standard library
|
|
without a deprecation period if the lingering concerns prove well-founded.
|
|
|
|
As part of the same announcement, Guido explicitly accepted Matthew
|
|
Barnett's 'regex' module [4]_ as a provisional addition to the standard
|
|
library for Python 3.3 (using the 'regex' name, rather than as a drop-in
|
|
replacement for the existing 're' module).
|
|
|
|
|
|
Proposal - the __preview__ package
|
|
==================================
|
|
|
|
Whenever the Python core development team decides that a new module should be
|
|
included into the standard library, but isn't entirely sure about whether the
|
|
module's API is optimal, the module can be placed in a special package named
|
|
``__preview__`` for a single minor release.
|
|
|
|
In the next minor release, the module may either be "graduated" into the
|
|
standard library (and occupy its natural place within its namespace, leaving the
|
|
``__preview__`` package), or be rejected and removed entirely from the Python
|
|
source tree. If the module ends up graduating into the standard library after
|
|
spending a minor release in ``__preview__``, its API may be changed according
|
|
to accumulated feedback. The core development team explicitly makes no
|
|
guarantees about API stability and backward compatibility of modules in
|
|
``__preview__``.
|
|
|
|
Entry into the ``__preview__`` package marks the start of a transition of the
|
|
module into the standard library. It means that the core development team
|
|
assumes responsibility of the module, similarly to any other module in the
|
|
standard library.
|
|
|
|
|
|
Which modules should go through ``__preview__``
|
|
-----------------------------------------------
|
|
|
|
We expect most modules proposed for addition into the Python standard library
|
|
to go through a minor release in ``__preview__``. There may, however, be some
|
|
exceptions, such as modules that use a pre-defined API (for example ``lzma``,
|
|
which generally follows the API of the existing ``bz2`` module), or modules
|
|
with an API that has wide acceptance in the Python development community.
|
|
|
|
In any case, modules that are proposed to be added to the standard library,
|
|
whether via ``__preview__`` or directly, must fulfill the acceptance conditions
|
|
set by :pep:`2`.
|
|
|
|
It is important to stress that the aim of this proposal is not to make the
|
|
process of adding new modules to the standard library more difficult. On the
|
|
contrary, it tries to provide a means to add *more* useful libraries. Modules
|
|
which are obvious candidates for entry can be added as before. Modules which
|
|
due to uncertainties about the API could be stalled for a long time now have
|
|
a means to still be distributed with Python, via an incubation period in the
|
|
``__preview__`` package.
|
|
|
|
|
|
Criteria for "graduation"
|
|
-------------------------
|
|
|
|
In principle, most modules in the ``__preview__`` package should eventually
|
|
graduate to the stable standard library. Some reasons for not graduating are:
|
|
|
|
* The module may prove to be unstable or fragile, without sufficient developer
|
|
support to maintain it.
|
|
* A much better alternative module may be found during the preview release
|
|
|
|
Essentially, the decision will be made by the core developers on a per-case
|
|
basis. The point to emphasize here is that a module's appearance in the
|
|
``__preview__`` package in some release does not guarantee it will continue
|
|
being part of Python in the next release.
|
|
|
|
|
|
Example
|
|
-------
|
|
|
|
Suppose the ``example`` module is a candidate for inclusion in the standard
|
|
library, but some Python developers aren't convinced that it presents the best
|
|
API for the problem it intends to solve. The module can then be added to the
|
|
``__preview__`` package in release ``3.X``, importable via::
|
|
|
|
from __preview__ import example
|
|
|
|
Assuming the module is then promoted to the standard library proper in
|
|
release ``3.X+1``, it will be moved to a permanent location in the library::
|
|
|
|
import example
|
|
|
|
And importing it from ``__preview__`` will no longer work.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
Benefits for the core development team
|
|
--------------------------------------
|
|
|
|
Currently, the core developers are really reluctant to add new interfaces to
|
|
the standard library. This is because as soon as they're published in a
|
|
release, API design mistakes get locked in due to backward compatibility
|
|
concerns.
|
|
|
|
By gating all major API additions through some kind of a preview mechanism
|
|
for a full release, we get one full release cycle of community feedback
|
|
before we lock in the APIs with our standard backward compatibility guarantee.
|
|
|
|
We can also start integrating preview modules with the rest of the standard
|
|
library early, so long as we make it clear to packagers that the preview
|
|
modules should not be considered optional. The only difference between preview
|
|
APIs and the rest of the standard library is that preview APIs are explicitly
|
|
exempted from the usual backward compatibility guarantees.
|
|
|
|
Essentially, the ``__preview__`` package is intended to lower the risk of
|
|
locking in minor API design mistakes for extended periods of time. Currently,
|
|
this concern can block new additions, even when the core development team
|
|
consensus is that a particular addition is a good idea in principle.
|
|
|
|
|
|
Benefits for end users
|
|
----------------------
|
|
|
|
For future end users, the broadest benefit lies in a better "out-of-the-box"
|
|
experience - rather than being told "oh, the standard library tools for task X
|
|
are horrible, download this 3rd party library instead", those superior tools
|
|
are more likely to be just be an import away.
|
|
|
|
For environments where developers are required to conduct due diligence on
|
|
their upstream dependencies (severely harming the cost-effectiveness of, or
|
|
even ruling out entirely, much of the material on PyPI), the key benefit lies
|
|
in ensuring that anything in the ``__preview__`` package is clearly under
|
|
python-dev's aegis from at least the following perspectives:
|
|
|
|
* Licensing: Redistributed by the PSF under a Contributor Licensing Agreement.
|
|
* Documentation: The documentation of the module is published and organized via
|
|
the standard Python documentation tools (i.e. ReST source, output generated
|
|
with Sphinx and published on http://docs.python.org).
|
|
* Testing: The module test suites are run on the python.org buildbot fleet
|
|
and results published via http://www.python.org/dev/buildbot.
|
|
* Issue management: Bugs and feature requests are handled on
|
|
http://bugs.python.org
|
|
* Source control: The master repository for the software is published
|
|
on http://hg.python.org.
|
|
|
|
|
|
Candidates for inclusion into __preview__
|
|
=========================================
|
|
|
|
For Python 3.3, there are a number of clear current candidates:
|
|
|
|
* ``regex`` (http://pypi.python.org/pypi/regex)
|
|
* ``daemon`` (:pep:`3143`)
|
|
* ``ipaddr`` (:pep:`3144`)
|
|
|
|
Other possible future use cases include:
|
|
|
|
* Improved HTTP modules (e.g. ``requests``)
|
|
* HTML 5 parsing support (e.g. ``html5lib``)
|
|
* Improved URL/URI/IRI parsing
|
|
* A standard image API (:pep:`368`)
|
|
* Encapsulation of the import state (:pep:`368`)
|
|
* Standard event loop API (:pep:`3153`)
|
|
* A binary version of WSGI for Python 3 (e.g. :pep:`444`)
|
|
* Generic function support (e.g. ``simplegeneric``)
|
|
|
|
|
|
Relationship with PEP 407
|
|
=========================
|
|
|
|
:pep:`407` proposes a change to the core Python release cycle to permit interim
|
|
releases every 6 months (perhaps limited to standard library updates). If
|
|
such a change to the release cycle is made, the following policy for the
|
|
``__preview__`` namespace is suggested:
|
|
|
|
* For long-term support releases, the ``__preview__`` namespace would always
|
|
be empty.
|
|
* New modules would be accepted into the ``__preview__`` namespace only in
|
|
interim releases that immediately follow a long-term support release.
|
|
* All modules added will either be migrated to their final location in the
|
|
standard library or dropped entirely prior to the next long-term support
|
|
release.
|
|
|
|
|
|
Rejected alternatives and variations
|
|
====================================
|
|
|
|
|
|
Using ``__future__``
|
|
--------------------
|
|
|
|
Python already has a "forward-looking" namespace in the form of the
|
|
``__future__`` module, so it's reasonable to ask why that can't be re-used for
|
|
this new purpose.
|
|
|
|
There are two reasons why doing so not appropriate:
|
|
|
|
1. The ``__future__`` module is actually linked to a separate compiler
|
|
directives feature that can actually change the way the Python interpreter
|
|
compiles a module. We don't want that for the preview package - we just want
|
|
an ordinary Python package.
|
|
|
|
2. The ``__future__`` module comes with an express promise that names will be
|
|
maintained in perpetuity, long after the associated features have become the
|
|
compiler's default behaviour. Again, this is precisely the opposite of what is
|
|
intended for the preview package - it is almost certain that all names added to
|
|
the preview will be removed at some point, most likely due to their being moved
|
|
to a permanent home in the standard library, but also potentially due to their
|
|
being reverted to third party package status (if community feedback suggests the
|
|
proposed addition is irredeemably broken).
|
|
|
|
|
|
Versioning the package
|
|
----------------------
|
|
|
|
One proposed alternative [1]_ was to add explicit versioning to the
|
|
``__preview__`` package, i.e. ``__preview34__``. We think that it's better to
|
|
simply define that a module being in ``__preview__`` in Python 3.X will either
|
|
graduate to the normal standard library namespace in Python 3.X+1 or will
|
|
disappear from the Python source tree altogether. Versioning the ``_preview__``
|
|
package complicates the process and does not align well with the main intent of
|
|
this proposal.
|
|
|
|
|
|
Using a package name without leading and trailing underscores
|
|
-------------------------------------------------------------
|
|
|
|
It was proposed [1]_ to use a package name like ``preview`` or ``exp``, instead
|
|
of ``__preview__``. This was rejected in the discussion due to the special
|
|
meaning a "dunder" package name (that is, a name *with* leading and
|
|
trailing double-underscores) conveys in Python. Besides, a non-dunder name
|
|
would suggest normal standard library API stability guarantees, which is not
|
|
the intention of the ``__preview__`` package.
|
|
|
|
|
|
Preserving pickle compatibility
|
|
-------------------------------
|
|
|
|
A pickled class instance based on a module in ``__preview__`` in release 3.X
|
|
won't be unpickle-able in release 3.X+1, where the module won't be in
|
|
``__preview__``. Special code may be added to make this work, but this goes
|
|
against the intent of this proposal, since it implies backward compatibility.
|
|
Therefore, this PEP does not propose to preserve pickle compatibility.
|
|
|
|
|
|
Credits
|
|
=======
|
|
|
|
Dj Gilcrease initially proposed the idea of having a ``__preview__`` package
|
|
in Python [2]_. Although his original proposal uses the name
|
|
``__experimental__``, we feel that ``__preview__`` conveys the meaning of this
|
|
package in a better way.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] Discussed in this thread:
|
|
https://mail.python.org/pipermail/python-ideas/2012-January/013246.html
|
|
|
|
.. [2] https://mail.python.org/pipermail/python-ideas/2011-August/011278.html
|
|
|
|
.. [3] Guido's decision:
|
|
https://mail.python.org/pipermail/python-dev/2012-January/115962.html
|
|
|
|
.. [4] Proposal for inclusion of regex: http://bugs.python.org/issue2636
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
coding: utf-8
|
|
End:
|