322 lines
13 KiB
Plaintext
322 lines
13 KiB
Plaintext
PEP: 497
|
|
Title: A standard mechanism for backward compatibility
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Ed Schofield <ed at pythoncharmers.com>
|
|
PEP-Delegate: Brett Cannon <brett@python.org>
|
|
Status: Rejected
|
|
Type: Process
|
|
Content-Type: text/x-rst
|
|
Created: 04-Aug-2015
|
|
|
|
|
|
Rejection Notice
|
|
================
|
|
The steering council decided that the ``__past__`` aspect of this proposal
|
|
was too complicated for the potential benefit. The other aspect of stronger
|
|
requirements for backwards-compatibility should be addressed by :pep:`387`.
|
|
|
|
|
|
Scope
|
|
=====
|
|
|
|
This PEP is complementary to PEPs 5, 236, and 387, and shares similar
|
|
goals.
|
|
|
|
This PEP explains the need for an additional compatibility mechanism
|
|
in support of :pep:`5`, "Guidelines for Language Evolution". :pep:`236`,
|
|
"Back to the ``__future__``", introduced a mechanism for forward
|
|
compatibility in support of :pep:`5` but noted that a new mechanism for
|
|
backward compatibility was outside the scope of that PEP. A related
|
|
PEP (in progress) introduces such a mechanism for backward
|
|
compatibility.
|
|
|
|
:pep:`5`, "Guidelines for Language Evolution", notes that "This PEP [:pep:`5`]
|
|
does not replace or preclude other compatibility strategies such as
|
|
dynamic loading of backwards-compatible parsers."
|
|
|
|
|
|
Context
|
|
=======
|
|
|
|
From :pep:`236`: "From time to time, Python makes an incompatible change
|
|
to the advertised semantics of core language constructs, or changes
|
|
their accidental (implementation-dependent) behavior in some way.
|
|
While this is never done capriciously, and is always done with the aim
|
|
of improving the language over the long term, over the short term it's
|
|
contentious and disrupting. :pep:`5`, Guidelines for Language Evolution,
|
|
suggests ways to ease the pain, and this PEP [:pep:`236`] introduces some
|
|
machinery in support of that."
|
|
|
|
Also from :pep:`236`: "The purpose of future_statement is to make life
|
|
easier for people who keep current with the latest release in a timely
|
|
fashion. We don't hate you if you don't, but your problems are much
|
|
harder to solve, and somebody with those problems will need to write a
|
|
PEP addressing them. future_statement is aimed at a different
|
|
audience."
|
|
|
|
|
|
The current situation
|
|
=====================
|
|
|
|
When an incompatible change to core language syntax or semantics is
|
|
being made, Python currently provides the future_statement mechanism
|
|
for providing forward compatibility until the release that enforces
|
|
the new syntax or semantics, but provides no corresponding standard
|
|
mechanism for providing backward compatibility after this release.
|
|
|
|
|
|
Problem
|
|
=======
|
|
|
|
A consequence of this asymmetry is that, with respect to a breaking
|
|
change, the older (pre-breaking) version of the Python interpreter is
|
|
more capable than the newer (breaking) version; the older interpreter
|
|
can use both code designed prior to the change and newer code, whereas
|
|
the newer interpreter is only capable of using code that has been
|
|
upgraded to support the changed feature.
|
|
|
|
As an example, consider the changes to the division operator
|
|
introduced in :pep:`238` in 2001, soon after :pep:`236` introduced the
|
|
future_statement mechanism. :pep:`238` outlines a suite of useful
|
|
forward-compatibility mechanisms for "true division" in the Python 2.x
|
|
series but omits to include any backward-compatibility mechanisms for
|
|
after "true division" was first enforced in Python 3.0. Python versions
|
|
since 3.0 do not provide a backward compatibility mechanism such as
|
|
``from __past__ import division`` for code that expects the old
|
|
"classic division" semantics, whereas Python versions prior to 3.0 do
|
|
support both "classic division" code and also forward compatibility
|
|
with code expecting "true division". A further consequence of this is
|
|
that the "most compatible" interpreter with respect to the variety of
|
|
division-related Python code in the wild is Python 2.7, the version
|
|
before the breaking change was first enforced.
|
|
|
|
|
|
Backward compatibility as enabler for "downhill upgrades"
|
|
=========================================================
|
|
|
|
In contrast to this situation, newer versions of application software
|
|
such as office suites tend to be more capable than earlier versions
|
|
with respect to their support for loading different versions of their
|
|
data file formats. The pattern is usually that the newer application
|
|
versions can transparently load data from either their newer or their
|
|
older data formats, and that the newer version defaults to saving data
|
|
in the newer format. Newer application software versions tend to be
|
|
backward-compatible by default. Forward compatibility is relatively
|
|
rare.
|
|
|
|
This policy puts the user of the newer application software at an
|
|
advantage over the user of the older software, which is usually
|
|
incapable of loading data in the newer format. Sometimes it is
|
|
possible for a user of a newer software application version to export
|
|
data in an older version by choosing this option explicitly. In these
|
|
cases, the forward-compatibility this enables may or may not be
|
|
perfect; some features may be missing or the results may be otherwise
|
|
suboptimal. Upgrading is therefore easy, whereas downgrading is
|
|
harder.
|
|
|
|
The emergent behaviour over many users from such a policy of new
|
|
attractive features plus backward compatibility features is that a
|
|
natural pressure builds up on each individual user to upgrade his or
|
|
her own application version, and, the more other users an individual
|
|
exchanges data files with, the more acute this pressure becomes.
|
|
|
|
|
|
Proposal - part 1
|
|
=================
|
|
|
|
This PEP makes two specific, related proposals. The first is that:
|
|
|
|
:pep:`5` be augmented with a 6th step in the section "Steps for
|
|
Introducing Backwards-Incompatible Features" to indicate that, when an
|
|
incompatible change to core language syntax or semantics is being
|
|
made, Python-dev's policy is to prefer and expect that, wherever
|
|
possible, a mechanism for backward compatibility be considered and
|
|
provided for future Python versions after the breaking change is
|
|
adopted by default, in addition to any mechanisms proposed for forward
|
|
compatibility such as new future_statements. Furthermore, :pep:`387`,
|
|
"Backwards Compatibility Policy" (if accepted) would be
|
|
augmented with the same 6th step.
|
|
|
|
|
|
Example
|
|
~~~~~~~
|
|
|
|
As an example of how this PEP is to be applied, if the latest revision
|
|
of the "true division" PEP (238) were proposed today, it would be
|
|
considered incomplete. :pep:`238` notes the "severe backwards
|
|
compatibility issues" raised by the proposal and describes several
|
|
measures for forward compatibility in the Abstract and API Changes
|
|
sections. It also mentions some backward compatibility ideas raised on
|
|
c.l.py, including "Use ``from __past__ import division`` to use
|
|
classic division semantics in a module", but it does not put forward
|
|
any backward compatibility plan as part of the proposal.
|
|
|
|
If this PEP is accepted, it would be expected that a proposal such as
|
|
:pep:`238`, because of its large-scale compatibility implications, would
|
|
also be accompanied by a backward compatibility plan that enables
|
|
users of future Python versions after the breaking change has come
|
|
into effect to re-enable the classic division behaviour easily in
|
|
their code.
|
|
|
|
|
|
Proposal - part 2
|
|
=================
|
|
|
|
The second proposal is that:
|
|
|
|
Python provide a standard backward compatibility mechanism in
|
|
parallel to the ``__future__`` module mechanism for forward
|
|
compatibility.
|
|
|
|
For reference, this document will refer to this as a "``__past__``"
|
|
mechanism hereon, although it need not have all the characteristics
|
|
of the ``__future__`` module and ``future_statement`` mechanism.
|
|
|
|
The specific form and implementation of the ``__past__`` mechanism is
|
|
the subject of a separate PEP (in progress). However, this PEP
|
|
recommends that this ``__past__`` mechanism be designed to meet
|
|
similar criteria to those outlined in :pep:`296` for ``__future__``.
|
|
Specifically:
|
|
|
|
a. It should enable individual modules to specify obsolete behaviours
|
|
to re-enable from older Python versions on a module-by-module basis.
|
|
|
|
b. It should be flexible enough for both Python 3.6+ and point
|
|
releases of earlier versions to reintroduce backward compatibility
|
|
with older Python syntax or semantics for user modules that invoke the
|
|
``__past__`` mechanism.
|
|
|
|
c. It should be possible to run older code augmented to invoke
|
|
``__past__`` behaviours on older Python versions such as 2.x that have
|
|
no knowledge of the specific ``__past__`` features invoked, or even
|
|
that the ``__past__`` mechanism for backward-compatibility exists.
|
|
|
|
|
|
Counter-examples
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
Some implementations of ``__past__`` mechanisms that would violate
|
|
these criteria are:
|
|
|
|
a. Import hooks. These would normally fail to work on a
|
|
module-by-module basis; instead they apply recursively to all new
|
|
modules imported from within a module.
|
|
|
|
b. A new piece of syntax or new semantics for Python 3.6 that is
|
|
incompatible with prior versions.
|
|
|
|
c. A function added in Python 3.6 to a module in the Python standard
|
|
library that exists under the same name in prior Python versions.
|
|
|
|
|
|
Benefits
|
|
========
|
|
|
|
The benefit to Python-dev of adopting this proposal is that future
|
|
backward-incompatible changes can be less disruptive if these changes
|
|
each have a corresponding ``__past__`` feature that has been
|
|
implemented and can be invoked easily by users of future Python
|
|
versions. This can help the language to evolve more quickly and more
|
|
effectively to correct for design mistakes.
|
|
|
|
The benefit to conservative users is obvious: they can add support for
|
|
the latest shiny compatibility-breaking Python version to their code
|
|
merely by adding a ``__past__`` incantation (perhaps a single line) to
|
|
each module, and that this can be automated. They can then upgrade
|
|
their interpreter to the latest version and gain access to the latest
|
|
shiny Python features.
|
|
|
|
The benefit to the community is that, if ten thousand users rely on
|
|
package XYZ, and package XYZ can trivially add support for the latest
|
|
Python version, those ten thousand users can also upgrade to the
|
|
latest Python version quickly, without being held back waiting for
|
|
package XYZ to do this.
|
|
|
|
|
|
Questions and answers
|
|
=====================
|
|
|
|
Q1: Does this PEP require that Python keep two possible sets of semantics
|
|
for each backward-incompatible feature forever?
|
|
|
|
A1: Definitely not. Legacy features can still be phased out when
|
|
appropriate -- that is, when the majority of the user-base has
|
|
migrated to the newer Python version. This PEP merely proposes to
|
|
shift the emphasis of the development effort directed at compatibility
|
|
from 100% forwards to at least 50% backwards. Backwards compatibility
|
|
is the more powerful of the two concepts for allowing a user-base to
|
|
adopt the latest Python interpreter version.
|
|
|
|
Notice that it has been a long time since most users have cared about
|
|
backwards compatibility for non-nested scopes, because most users have
|
|
moved comfortably past Python 2.1.
|
|
|
|
Q2: But Python-dev is already overwhelmed and doesn't have the
|
|
bandwidth to implement / maintain the additional complexity!
|
|
|
|
A2: Python-dev can ask the community of developers to step up and
|
|
maintain backward compatibility in Python for legacy language features
|
|
they care about. When the community stops caring about a particular
|
|
obsolete behaviour, Python-dev can stop caring too.
|
|
|
|
The ``__past__`` mechanism could possibly be designed to be extensible
|
|
by the community, e.g. as a standard but "blessed" PyPI package, to
|
|
reduce the load on the core developers.
|
|
|
|
Q3: Won't backward compatibility features lead to lots of cruft and
|
|
bloat and baggage in Python?
|
|
|
|
A3: Not necessarily. First, proposals for new compatibility-breaking
|
|
features in Python could be evaluated partly on the simplicity and
|
|
maintainability of the implementation of their associated ``__past__``
|
|
feature up-front.
|
|
|
|
Second, some old features are simple to provide backward compatibility
|
|
for. Consider the "classic division" behaviour before Python 3.0. The
|
|
``python-future`` project contains a compatible implementation of
|
|
classic division in the function ``future.utils.old_div``:
|
|
|
|
::
|
|
|
|
def old_div(a, b):
|
|
"""
|
|
Equivalent to ``a / b`` on Python 2 without ``from __future__ import
|
|
division``.
|
|
"""
|
|
if isinstance(a, numbers.Integral) and isinstance(b, numbers.Integral):
|
|
return a // b
|
|
else:
|
|
return a / b
|
|
|
|
|
|
Bundling such a function with Python 3.x versions, together with
|
|
a simple mechanism to invoke it for every appearance of ``a
|
|
/ b`` after an appropriate ``__past__`` invocation, need not be
|
|
onerous.
|
|
|
|
|
|
Q4: What about performance? Won't the performance of newer Python
|
|
versions suffer under the weight of legacy features?
|
|
|
|
A4: This can be evaluated on a case-by-case basis. The major potential
|
|
concern is that the performance with the new default behaviour does
|
|
not suffer unduly because of the presence of the legacy option. The
|
|
performance under the influence of the ``__past__`` invocation is of
|
|
secondary importance.
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
coding: utf-8
|