python-peps/pep-0236.txt

PEP: 236
Title: Back to the __future__
Version: $Revision$
Author: Tim Peters <tim@digicool.com>
Python-Version: 2.1
Status: Active
Type: Standards Track
Created: 26-Feb-2001
Post-History: 26-Feb-2001


Motivation

    From time to time, Python makes an incompatible change to the
    advertised semantics of core language constructs, or changes their
    accidental (implementation-dependent) behavior in some way.  While this
    is never done capriciously, and is always done with the aim of
    improving the language over the long term, over the short term it's
    contentious and disrupting.

    The "Guidelines for Language Evolution" PEP [1] suggests ways to ease
    the pain, and this PEP introduces some machinery in support of that.

    The "Statically Nested Scopes" PEP [2] is the first application, and
    will be used as an example here.


Intent

    [Note:  This is policy, and so should eventually move into PEP 5[1]]

    When an incompatible change to core language syntax or semantics is
    being made:

    1. The release C that introduces the change does not change the
       syntax or semantics by default.

    2. A future release R is identified in which the new syntax or semantics
       will be enforced.

    3. The mechanisms described in the "Warning Framework" PEP [3] are used
       to generate warnings, whenever possible, about constructs or
       operations whose meaning may[4] change in release R.

    4. The new future_statement (see below) can be explicitly included in a
       module M to request that the code in module M use the new syntax or
       semantics in the current release C.

    So old code continues to work by default, for at least one release,
    although it may start to generate new warning messages.  Migration to
    the new syntax or semantics can proceed during that time, using the
    future_statement to make modules containing it act as if the new syntax
    or semantics were already being enforced.

    Note that there is no need to involve the future_statement machinery
    in new features unless they can break existing code; fully backward-
    compatible additions can-- and should --be introduced without a
    corresponding future_statement.


Syntax

    A future_statement is simply a from/import statement using the reserved
    module name __future__:

        future_statement: "from" "__future__" "import" feature ["as" name]
                          ("," feature ["as" name])*

        feature: identifier
        name: identifier

    In addition, all future_statments must appear near the top of the
    module.  The only lines that can appear before a future_statement are:

    + The module docstring (if any).
    + Comments.
    + Blank lines.
    + Other future_statements.

    Example:
        """This is a module docstring."""

        # This is a comment, preceded by a blank line and followed by
        # a future_statement.
        from __future__ import nested_scopes

        from math import sin
        from __future__ import alabaster_weenoblobs  # compile-time error!
        # That was an error because preceded by a non-future_statement.


Semantics

    A future_statement is recognized and treated specially at compile time:
    changes to the semantics of core constructs are often implemented by
    generating different code.  It may even be the case that a new feature
    introduces new incompatible syntax (such as a new reserved word), in
    which case the compiler may need to parse the module differently.  Such
    decisions cannot be pushed off until runtime.

    For any given release, the compiler knows which feature names have been
    defined, and raises a compile-time error if a future_statement contains
    a feature not known to it[5].

    The direct runtime semantics are the same as for any import statement:
    there is a standard module __future__.py, described later, and it will
    be imported in the usual way at the time the future_statement is
    executed.

    The *interesting* runtime semantics depend on the specific feature(s)
    "imported" by the future_statement(s) appearing in the module.

    Note that there is nothing special about the statement:

        import __future__ [as name]

    That is not a future_statement; it's an ordinary import statement, with
    no special semantics or syntax restrictions.


Example

    Consider this code, in file scope.py:

        x = 42
        def f():
            x = 666
            def g():
                print "x is", x
            g()
        f()

    Under 2.0, it prints:

        x is 42

    Nested scopes[2] are being introduced in 2.1.  But under 2.1, it still
    prints

        x is 42

    and also generates a warning.

    In 2.2, and also in 2.1 *if* "from __future__ import nested_scopes" is
    included at the top of scope.py, it prints

        x is 666


Standard Module __future__.py

    Lib/__future__.py is a real module, and serves three purposes:

    1. To avoid confusing existing tools that analyze import statements and
       expect to find the modules they're importing.

    2. To ensure that future_statements run under releases prior to 2.1
       at least yield runtime exceptions (the import of __future__ will
       fail, because there was no module of that name prior to 2.1).

    3. To document when incompatible changes were introduced, and when they
       will be-- or were --made mandatory.  This is a form of executable
       documentation, and can be inspected programatically via importing
       __future__ and examining its contents.

    Each statment in __future__.py is of the form:

        FeatureName = "_Feature(" OptionalRelease "," MandatoryRelease ")"

    where, normally, OptionalRelease <  MandatoryRelease, and both are
    5-tuples of the same form as sys.version_info:

    (PY_MAJOR_VERSION, # the 2 in 2.1.0a3; an int
     PY_MINOR_VERSION, # the 1; an int
     PY_MICRO_VERSION, # the 0; an int
     PY_RELEASE_LEVEL, # "alpha", "beta", "candidate" or "final"; string
     PY_RELEASE_SERIAL # the 3; an int
    )

    OptionalRelease records the first release in which

        from __future__ import FeatureName

    was accepted.

    In the case of MandatoryReleases that have not yet occurred,
    MandatoryRelease predicts the release in which the feature will become
    part of the language.

    Else MandatoryRelease records when the feature became part of the
    language; in releases at or after that, modules no longer need

        from __future__ import FeatureName

    to use the feature in question, but may continue to use such imports.

    MandatoryRelease may also be None, meaning that a planned feature got
    dropped.

    Instances of class _Feature have two corresponding methods,
    .getOptionalRelease() and .getMandatoryRelease().

    No feature line will ever be deleted from __future__.py.

    Example line:

      nested_scopes = _Feature((2, 1, 0, "beta", 1), (2, 2, 0, "final", 0))

    This means that

        from __future__ import nested_scopes

    will work in all releases at or after 2.1b1, and that nested_scopes are
    intended to be enforced starting in release 2.2.


Unresolved Problems:  Runtime Compilation

    Several Python features can compile code during a module's runtime:

    1. The exec statement.
    2. The execfile() function.
    3. The compile() function.
    4. The eval() function.
    5. The input() function.

    Since a module M containing a future_statement naming feature F
    explicitly requests that the current release act like a future release
    with respect to F, any code compiled dynamically from text passed to
    one of these from within M should probably also use the new syntax or
    semantics associated with F.

    This isn't always desired, though.  For example, doctest.testmod(M)
    compiles examples taken from strings in M, and those examples should
    use M's choices, not necessarily doctest module's choices.

    It's unclear what to do about this.  The initial release (2.1b1) is
    likely to ignore these issues, saying that each dynamic compilation
    starts over from scratch (i.e., as if no future_statements had been
    specified).

    In any case, a future_statement appearing "near the top" (see Syntax
    above) of text compiled dynamically by an exec, execfile() or compile()
    applies to the code block generated, but has no further effect on the
    module that executes such an exec, execfile() or compile().  This
    can't be used to affect eval() or input(), however, because they only
    allow expression input, and a future_statement is not an expression.


Unresolved Problems:  Interactive Shells

    An interactive shell can be seen as an extreme case of runtime
    compilation (see above):  in effect, each statement typed at an
    interactive shell prompt runs a new instance of exec, compile() or
    execfile().  The initial release (2.1b1) is likely to be such that
    future_statements typed at an interactive shell have no effect beyond
    their runtime meaning as ordinary import statements.

    It would make more sense if a future_statement typed at an interactive
    shell applied to the rest of the shell session's life, as if the
    future_statement had appeared at the top of a module.  Again, it's
    unclear what to do about this.


Questions and Answers

    Q:  What about a "from __past__" version, to get back *old* behavior?

    A:  Outside the scope of this PEP.  Seems unlikely to the author,
        though.  Write a PEP if you want to pursue it.

    Q:  What about incompatibilites due to changes in the Python virtual
        machine?

    A:  Outside the scope of this PEP, although PEP 5[1] suggests a grace
        period there too, and the future_statement may also have a role to
        play there.

    Q:  What about incompatibilites due to changes in Python's C API?

    A:  Outside the scope of this PEP.

    Q:  I want to wrap future_statements in try/except blocks, so I can
        use different code depending on which version of Python I'm running.
        Why can't I?

    A:  Sorry!  try/except is a runtime feature; future_statements are
        primarily compile-time gimmicks, and your try/except happens long
        after the compiler is done.  That is, by the time you do
        try/except, the semantics in effect for the module are already a
        done deal.  Since the try/except wouldn't accomplish what it
        *looks* like it should accomplish, it's simply not allowed.  We
        also want to keep these special statements very easy to find and to
        recognize.

        Note that you *can* import __future__ directly, and use the
        information in it, along with sys.version_info, to figure out where
        the release you're running under stands in relation to a given
        feature's status.

     Q: Going back to the nested_scopes example, what if release 2.2 comes
        along and I still haven't changed my code?  How can I keep the 2.1
        behavior then?

     A: By continuing to use 2.1, and not moving to 2.2 until you do change
        your code.  The purpose of future_statement is to make life easier
        for people who keep keep current with the latest release in a timely
        fashion.  We don't hate you if you don't, but your problems are
        much harder to solve, and somebody with those problems will need to
        write a PEP addressing them.  future_statement is aimed at a
        different audience.

     Q: Overloading "import" sucks.  Why not introduce a new statement for
        this?

     A: Like maybe "lambda lambda nested_scopes"?  That is, unless we
        introduce a new keyword, we can't introduce an entirely new
        statement.  But if we introduce a new keyword, that in itself
        would break old code.  That would be too ironic to bear.  Yes,
        overloading "import" does suck, but not as energeticallly as the
        alternatives -- as is, future_statements are 100% backward
        compatible.


Copyright

    This document has been placed in the public domain.


References and Footnotes

    [1] http://python.sourceforge.net/peps/pep-0005.html

    [2] http://python.sourceforge.net/peps/pep-0227.html

    [3] http://python.sourceforge.net/peps/pep-0230.html

    [4] Note that this is "may" and not "will":  better safe than sorry.  Of
        course spurious warnings won't be generated when avoidable with
        reasonable cost.

    [5] This ensures that a future_statement run under a release prior to
        the first one in which a given feature is known (but >= 2.1) will
        raise a compile-time error rather than silently do a wrong thing.
        If transported to a release prior to 2.1, a runtime error will be
        raised because of the failure to import __future__ (no such module
        existed in the standard distribution before the 2.1 release, and
        the double underscores make it a reserved name).


Local Variables:
mode: indented-text
indent-tabs-mode: nil
End: