python-peps/pep-0394.txt

PEP: 394
Title: The "python" command on Unix-Like Systems
Version: $Revision$
Last-Modified: $Date$
Author: Kerrick Staley <mail@kerrickstaley.com>,
        Nick Coghlan <ncoghlan@gmail.com>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 02-Mar-2011
Post-History: 04-Mar-2011


Abstract
========

This PEP provides a convention to ensure that Python scripts can continue to
be portable across ``*nix`` systems, regardless of the default version of the
Python interpreter (i.e. the version invoked by the ``python`` command).

* ``python2`` will refer to some version of Python 2.x
* ``python3`` will refer to some version of Python 3.x
* ``python`` will refer to the same target as either ``python2`` or
  ``python3``, depending on the specific distribution and system


Recommendation
==============

* Unix-like software distributions (including systems like Mac OS X and
  Cygwin) should install the ``python2`` command into the default path
  whenever a version of the Python 2 interpreter is installed, and the same
  for ``python3`` and the Python 3 interpreter.
* When  invoked, ``python2`` should run some version of the Python 2
  interpreter, and ``python3`` should run some version of the Python 3
  interpreter.
* Similarly, the more general ``python`` command should be installed whenever
  any version of Python is installed and should invoke the same version of
  Python as either ``python2`` or ``python3``.
* For the time being, it is recommended that ``python`` should refer to
  ``python2``, except on distributions which include only ``python3`` in their
  base install, or those that wish to push strongly for migration of user
  scripts to Python 3.
* In order to tolerate differences across platforms, all new code that needs
  to invoke the Python interpreter should not specify ``python``, but rather
  should specify either ``python2`` or ``python3`` (or the more specific
  ``python2.x`` and ``python3.x`` versions; see the Notes).
  This distinction should be made in shebangs, when invoking from a shell
  script, when invoking via the system() call, or when invoking in any other
  context.
* One exception to this is scripts that are deliberately written to be source
  compatible with both Python 2.x and 3.x. Such scripts may continue to use
  ``python`` on their shebang line without affecting their portability.
* When reinvoking the interpreter from a Python script, querying
  ``sys.executable`` to avoid hardcoded assumptions regarding the
  interpreter location remains the preferred approach.

These recommendations are the outcome of the relevant python-dev discussion in
March 2011 [1] (NOTE: More accurately, they will be such once that "Draft"
status disappears from the PEP header, it has been moved into the "Other
Informational PEP" section in PEP 0 and this note has been deleted)


Rationale
=========

This is needed as, even though the majority of distributions still alias the
``python`` command to Python 2, some now alias it to Python 3. Some of
the former also do not provide a ``python2`` command; hence, there is
currently no way for Python 2 code (or any code that invokes the Python 2
interpreter directly rather than via ``sys.executable``) to reliably run on
all systems without modification, as the ``python`` command will invoke the
wrong interpreter version on some systems, and the ``python2`` command will
fail completely on others. The recommendations in this PEP provide a very
simple mechanism to restore cross-platform support, with minimal additional
work required on the part of distribution maintainers.


Notes
=====

* Distributions can alias the ``python`` command to whichever version of the
  Python interpreter they choose (noting that, in the near term, most 3rd
  party scripts will still expect this command to refer to Python 2.x). The
  version chosen should also be available via either the ``python2`` or
  ``python3`` command as appropriate.
* The ``pythonX.X`` (e.g. ``python2.6``) commands exist on some systems, on
  which they invoke specific minor versions of the Python interpreter. It
  would be wise for distribution-specific packages to take advantage of these
  utilities if they exist, since it will prevent code breakage if the default
  minor version of a given major version is changed. However, scripts
  intending to be cross-platform should not rely on the presence of these
  utilities, but rather should be tested on several recent minor versions of
  the target major version, compensating, if necessary, for the small
  differences that exist between minor versions. This prevents the need for
  sysadmins to install many very similar versions of the interpreter.
* When the ``pythonX.X`` binaries are provided by a distribution, the
  ``python2`` and ``python3`` commands should refer to one of those files
  rather being provided as a separate binary file.
* It is suggested that even distribution-specific packages follow the
  ``python2``/``python3`` convention, even in code that is not intended to
  operate on other distributions. This will prevent problems if the
  distribution later decides to change the version of the Python interpreter
  that the ``python`` command invokes, or if a sysadmin installs a custom
  ``python`` command with a different major version than the distribution
  default. Distributions can test whether they are fully following this
  convention by changing the ``python`` interpreter on a test box and checking
  to see if anything breaks.
* If the above point is adhered to and sysadmins are permitted to change the
  ``python`` command, then the ``python`` command should always be implemented
  as a link to the interpreter binary (or a link to a link) and not vice
  versa. That way, if a sysadmin does decide to replace the installed
  ``python`` file, they can do so without inadvertently deleting the
  previously installed binary.
* As an alternative to the recommendation presented above, some distributions
  may choose to leave the ``python`` command itself undefined, leaving
  sysadmins and users with the responsibility to choose their own preferred
  version to be made available as the ``python`` command.
* If the Python 2 interpreter becomes uncommon, scripts should nevertheless
  continue to use the ``python3`` convention rather that just ``python``. This
  will ease transition in the event that yet another major version of Python
  is released.
* If these conventions are adhered to, it will be the case that the ``python``
  command is only executed in an interactive manner.


Backwards Compatibility
=========================

A potential problem can arise if a script adhering to the
``python2``/``python3`` convention is executed on a system not supporting
these commands. This is mostly a non-issue, since the sysadmin can simply
create these symbolic links and avoid further problems.


Application to the CPython Reference Interpreter
================================================

While technically a new feature, the ``make install`` command and the Mac OS
X installer in the 2.7 version of CPython will be adjusted to create the
new ``python2`` command in addition to the existing ``python`` and
``python2.7`` commands. This feature will first appear in CPython 2.7.2.

The ``make install`` command in the CPython 3.x series will continue to
install only the ``python3`` symlink for the foreseeable future.


Impact on PYTHON* Environment Variables
=======================================

The choice of target for the ``python`` command implicitly affects a
distribution's expected interpretation of the various Python related
environment variables. The use of ``*.pth`` files in the relevant
``site-packages`` folder, the "per-user site packages" feature (see
``python -m site``) or more flexible tools such as ``virtualenv`` are all more
tolerant of the presence of multiple versions of Python on a system than the
direct use of ``PYTHONPATH``.


Exclusions of MS Windows
========================

This PEP deliberately excludes any proposals relating to Microsoft Windows.
The use of parallel installs on Windows suffers from numerous issues,
including the "last installed wins" behaviour for handling of file
associations, a lack of universal robust symlink support for easy aliasing of
commands, the fact that the Python executable is not available on ``PATH`` by
default, the fact that the ``python.exe`` and ``pythonw.exe`` names are
used for both Python 2 and Python 3 binaries and the lack of distinction
between the different Python versions when the Start menu shortcuts are
divorced from their containing folders (e.g. when they appear in the
"Recently Used" list.

While these are questions well worth addressing, they do not have easy
answers. The authors of this particular PEP aren't even inclined to try to
begin answering them, but anyone that wants to tackle them should feel free
to start working on their own PEP :)


References
==========

[1] Support the /usr/bin/python2 symlink upstream (with bonus grammar class!)
    (http://mail.python.org/pipermail/python-dev/2011-March/108491.html)


Copyright
===========
This document has been placed in the public domain.