python-peps/pep-3002.txt

PEP: 3002
Title: Procedure for Backwards-Incompatible Changes
Version: $Revision$
Last-Modified: $Date$
Author: Steven Bethard <steven.bethard@gmail.com>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 27-Mar-2006
Post-History: 27-Mar-2006, 13-Apr-2006


Abstract
========

This PEP describes the procedure for changes to Python that are
backwards-incompatible between the Python 2.X series and Python 3000.
All such changes must be documented by an appropriate Python 3000 PEP
and must be accompanied by code that can identify when pieces of
Python 2.X code may be problematic in Python 3000.


Rationale
=========

Python 3000 will introduce a number of backwards-incompatible changes
to Python, mainly to streamline the language and to remove some
previous design mistakes.  But Python 3000 is not intended to be a new
and completely different language from the Python 2.X series, and it
is expected that much of the Python user community will make the
transition to Python 3000 when it becomes available.

To encourage this transition, it is crucial to provide a clear and
complete guide on how to upgrade Python 2.X code to Python 3000 code.
Thus, for any backwards-incompatible change, two things are required:

* An official Python Enhancement Proposal (PEP)
* Code that can identify pieces of Python 2.X code that may be
  problematic in Python 3000


Python Enchancement Proposals
=============================

Every backwards-incompatible change must be accompanied by a PEP.
This PEP should follow the usual PEP guidelines and explain the
purpose and reasoning behind the backwards incompatible change.  In
addition to the usual PEP sections, all PEPs proposing
backwards-incompatible changes must include an additional section:
Compatibility Issues.  This section should describe what is backwards
incompatible about the proposed change to Python, and the major sorts
of breakage to be expected.

While PEPs must still be evaluated on a case-by-case basis, a PEP may
be inappropriate for Python 3000 if its Compatibility Issues section
implies any of the following:

* Most or all instances of a Python 2.X construct are incorrect in
  Python 3000, and most or all instances of the Python 3000 construct
  are incorrect in Python 2.X.

  So for example, changing the meaning of the for-loop else-clause
  from "executed when the loop was not broken out of" to "executed
  when the loop had zero iterations" would mean that all Python 2.X
  for-loop else-clauses would be broken, and there would be no way to
  use a for-loop else-clause in a Python-3000-appropriate manner.
  Thus a PEP for such an idea would likely be rejected.

* Many instances of a Python 2.X construct are incorrect in Python
  3000 and the PEP fails to demonstrate real-world use-cases for the
  changes.

  Backwards incompatible changes are allowed in Python 3000, but not
  to excess.  A PEP that proposes backwards-incompatible changes
  should provide good examples of code that visibly benefits from the
  changes.

PEP-writing is time-consuming, so when a number of
backwards-incompatible changes are closely related, they should be
proposed in the same PEP.  Such PEPs will likely have longer
Compatibility Issues sections however, since they must now describe
the sorts of breakage expected from *all* the proposed changes.


Identifying Problematic Code
============================

In addition to the PEP requirement, backwards incompatible changes to
Python must also be accompanied by code that can identify pieces of
Python 2.X code that may be problematic in Python 3.0.

This PEP proposes to house this code in tools/scripts/python3warn.py.
Thus PEPs for backwards incompatible changes should include a patch to
this file that produces the appropriate warnings.  Code in
python3warn.py should be written to the latest version of Python 2.X
(not Python 3000) so that Python 2.X users will be able to run the
program without having Python 3000 installed.

Currently, it seems too stringent to require that the code in
python3warn.py identify all changes perfectly.  Thus it is permissable
if a backwards-incompatible PEP's python3warn.py code produces a
number of false-positives (warning that a piece of code might be
invalid in Python 3000 when it's actually still okay).  However,
false-negatives (not issuing a warning for code that will do the
wrong thing in Python 3000) should be avoided whenever possible --
users of python3warn.py should be reasonably confident that they have
been warned about the vast majority of incompatibilities.

So for example, a PEP proposing that ``dict.items()`` be modified to
return an iterator instead of a list might add code like the following
to python3warn.py::

    items_in_for = re.compile(r'for\s+\w+\s+in\s+\w+\.items\(\):')
    ...
    for i, line in enumerate(file_lines):
        ...
        if '.items()' in line and not items_in_for.search(line):
            message = 'dict.items() call may expect list at line %i'
            warnings.warn(message % i)

This would issue a warning any time a ``.items()`` method was called
and not immediately iterated over in a for-loop.  Clearly this will
issue a number of false-positive warnings (e.g. ``d2 =
dict(d.items())``), but the number of false-negative warnings should
be relatively low.


Optional Extensions
===================

Instead of the python3warn.py script, a branch of Python 3000 could be
maintained that added warnings at all the appropriate points in the
code-base.  PEPs proposing backwards-incompatible changes would then
provide patches to the Python-3000-warn branch instead of to
python3warn.py.  With such a branch, the warnings issued could be
near-perfect and Python users could be confident that their code was
correct Python 3000 code by first running it on the Python-3000-warn
branch and fixing all the warnings.

At the moment, however, this PEP opts for the weaker measure
(python3warn.py) as it is expected that maintaining a Python-3000-warn
branch will be too much of a time drain.


References
==========

TBD


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: