PEP 394: Allow for more flexibility in handling /usr/bin/python (GH-989)
* PEP 394: Distributions can choose what does python mean Co-Authored-By: Petr Viktorin <encukou@gmail.com> * fixup! PEP 394: Distributions can choose what does python mean * fixup! PEP 394: Distributions can choose what does python mean * fixup! PEP 394: Distributions can choose what does python mean * PEP 394: Keep version agnostic shebang recommendation This update reverts back to the version agnostic "python" invocation as the default recommendation for developers, and rewords the rest of the PEP accordingly. In particular, it makes it clear that publishers are free to adopt the attitude of "we assume you are using a virtual environment", while at the same time granting the distributors the freedom they need to make software with the expectation work correctly when run directly against a system Python installation. * Apply suggestions from code review Co-Authored-By: Carol Willing <carolcode@willingconsulting.com> * Moving letters around * Change the abstract * Remove a paragraph * ValueError: some authors have more than one email address listed * Apply suggestions from code review Co-Authored-By: Petr Viktorin <encukou@gmail.com> * Address review comments * Rename section headers * Push script publishers and users to virtual environments * Rewording * Remove "preferred" shebang line wording * Change post date * Little tweaks * Address review comments - Clarify wording on "older" Linux distros - Remove discussion aroung Python 4.0
This commit is contained in:
Miro Hrončok
Barry Warsaw
parent
72c1c4151f
commit
ae932bd6fd
272
pep-0394.txt
272
pep-0394.txt
|
|
@ -5,104 +5,195 @@ Last-Modified: $Date$
|
|||
Author: Kerrick Staley <mail@kerrickstaley.com>,
|
||||
Nick Coghlan <ncoghlan@gmail.com>,
|
||||
Barry Warsaw <barry@python.org>,
|
||||
Petr Viktorin <encukou@gmail.com>
|
||||
Petr Viktorin <encukou@gmail.com>,
|
||||
Miro Hrončok <miro@hroncok.cz>,
|
||||
Carol Willing <willingc@gmail.com>,
|
||||
Status: Active
|
||||
Type: Informational
|
||||
Content-Type: text/x-rst
|
||||
Created: 02-Mar-2011
|
||||
Post-History: 04-Mar-2011, 20-Jul-2011, 16-Feb-2012, 30-Sep-2014, 28-Apr-2018
|
||||
Post-History: 04-Mar-2011, 20-Jul-2011, 16-Feb-2012, 30-Sep-2014, 28-Apr-2018,
|
||||
26-Jun-2019
|
||||
Resolution: https://mail.python.org/pipermail/python-dev/2012-February/116594.html
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This PEP provides a convention to ensure that Python scripts can continue to
|
||||
be portable across ``*nix`` systems, regardless of the default version of the
|
||||
Python interpreter (i.e. the version invoked by the ``python`` command).
|
||||
This PEP outlines the behavior of Python scripts when the ``python`` command
|
||||
is invoked.
|
||||
Depending on a distribution or system configuration,
|
||||
``python`` may or may not be installed.
|
||||
If ``python`` is installed its target interpreter may refer to ``python2``
|
||||
or ``python3``.
|
||||
End users may be unaware of this inconsistency across Unix-like systems.
|
||||
This PEP's goal is to reduce user confusion about what ``python`` references
|
||||
and what will be the script's behavior.
|
||||
|
||||
* ``python2`` will refer to some version of Python 2.x.
|
||||
* ``python3`` will refer to some version of Python 3.x.
|
||||
* for the time being, all distributions *should* ensure that ``python``,
|
||||
if installed, refers to the same target as ``python2``, unless the user
|
||||
deliberately overrides this or a virtual environment is active.
|
||||
* however, end users should be aware that ``python`` refers to ``python3``
|
||||
on at least Arch Linux (that change is what prompted the creation of this
|
||||
PEP), so ``python`` should be used in the shebang line only for scripts
|
||||
that are source compatible with both Python 2 and 3.
|
||||
* in preparation for an eventual change in the default version of Python,
|
||||
Python 2 only scripts should either be updated to be source compatible
|
||||
with Python 3 or else to use ``python2`` in the shebang line.
|
||||
The recommendations in the next section of this PEP will outline the behavior
|
||||
when:
|
||||
|
||||
* using virtual environments
|
||||
* writing cross-platform scripts with shebangs for either ``python2`` or ``python3``
|
||||
|
||||
The PEP's goal is to clarify the behavior for script end users, distribution
|
||||
providers, and script maintainers / authors.
|
||||
|
||||
|
||||
Recommendation
|
||||
==============
|
||||
|
||||
* Unix-like software distributions (including systems like Mac OS X and
|
||||
Cygwin) should install the ``python2`` command into the default path
|
||||
|
||||
Our recommendations are detailed below.
|
||||
We call out any expectations that these recommendations are based upon.
|
||||
|
||||
For Python runtime distributors
|
||||
-------------------------------
|
||||
|
||||
* We expect Unix-like software distributions (including systems like macOS and
|
||||
Cygwin) to install the ``python2`` command into the default path
|
||||
whenever a version of the Python 2 interpreter is installed, and the same
|
||||
for ``python3`` and the Python 3 interpreter.
|
||||
* When invoked, ``python2`` should run some version of the Python 2
|
||||
interpreter, and ``python3`` should run some version of the Python 3
|
||||
interpreter.
|
||||
* If the ``python`` command is installed, it should invoke the same version of
|
||||
Python as the ``python2`` command (however, note that some distributions
|
||||
have already chosen to have ``python`` implement the ``python3``
|
||||
command; see the `Rationale`_ and `Migration Notes`_ below).
|
||||
* The Python 2.x ``idle``, ``pydoc``, and ``python-config`` commands should
|
||||
likewise be available as ``idle2``, ``pydoc2``, and ``python2-config``,
|
||||
with the original commands invoking these versions by default, but possibly
|
||||
invoking the Python 3.x versions instead if configured to do so by the
|
||||
system administrator.
|
||||
* In order to tolerate differences across platforms, all new code that needs
|
||||
to invoke the Python interpreter should not specify ``python``, but rather
|
||||
should specify either ``python2`` or ``python3`` (or the more specific
|
||||
``python2.x`` and ``python3.x`` versions; see the `Migration Notes`_).
|
||||
This distinction should be made in shebangs, when invoking from a shell
|
||||
script, when invoking via the system() call, or when invoking in any other
|
||||
context.
|
||||
* One exception to this is scripts that are deliberately written to be source
|
||||
compatible with both Python 2.x and 3.x. Such scripts may continue to use
|
||||
``python`` on their shebang line.
|
||||
* When packaging software that is source compatible with both versions,
|
||||
distributions may change such ``python`` shebangs to ``python3``.
|
||||
This ensures software is used with the latest version of
|
||||
Python available, and it can remove a dependency on Python 2.
|
||||
* If the ``python`` command is installed, it is expected to invoke either
|
||||
the same version of Python as the ``python3`` command or as the ``python2``
|
||||
command.
|
||||
* Distributors may choose to set the behavior of the ``python`` command
|
||||
as follows:
|
||||
|
||||
* ``python2``,
|
||||
* ``python3``,
|
||||
* not provide ``python`` command,
|
||||
* allow ``python`` to be configurable by an end user or
|
||||
a system administrator.
|
||||
|
||||
* The Python 3.x ``idle``, ``pydoc``, and ``python-config`` commands should
|
||||
likewise be available as ``idle3``, ``pydoc3``, and ``python3-config``;
|
||||
Python 2.x versions as ``idle2``, ``pydoc2``, and ``python2-config``.
|
||||
The commands with no version number should either invoke the same version
|
||||
of Python as the ``python`` command, or not be available at all.
|
||||
* When packaging third party Python scripts, distributors are encouraged to
|
||||
change less specific shebangs to more specific ones.
|
||||
This ensures software is used with the latest version of Python available,
|
||||
and it can remove a dependency on Python 2.
|
||||
The details on what specifics to set are left to the distributors;
|
||||
though. Example specifics could include:
|
||||
|
||||
* Changing ``python`` shebangs to ``python3`` when Python 3.x is supported.
|
||||
* Changing ``python`` shebangs to ``python2`` when Python 3.x is not yet
|
||||
supported.
|
||||
* Changing ``python3`` shebangs to ``python3.8`` if the software is built
|
||||
with Python 3.8.
|
||||
|
||||
* When a virtual environment (created by the PEP 405 ``venv`` package or a
|
||||
similar tool such as ``virtualenv`` or ``conda``) is active, the ``python``
|
||||
command should refer to the virtual environment's interpreter and should
|
||||
always be available.
|
||||
The ``python3`` or ``python2`` command (according to the environment's
|
||||
interpreter version) should also be available.
|
||||
|
||||
For Python script publishers
|
||||
----------------------------
|
||||
|
||||
* When reinvoking the interpreter from a Python script, querying
|
||||
``sys.executable`` to avoid hardcoded assumptions regarding the
|
||||
interpreter location remains the preferred approach.
|
||||
* In controlled environments aimed at expert users, where being explicit
|
||||
is valued over user experience (for example, in test environments and
|
||||
package build systems), distributions may choose to not provide the
|
||||
``python`` command even if ``python2`` is available.
|
||||
(All software in such a controlled environment must use ``python3`` or
|
||||
``python2`` rather than ``python``, which means scripts that deliberately
|
||||
use ``python`` need to be modified for such environments.)
|
||||
* When a virtual environment (created by the PEP 405 ``venv`` package or a
|
||||
similar tool) is active, the ``python`` command should refer to the
|
||||
virtual environment's interpreter. In other words, activating a virtual
|
||||
environment counts as deliberate user action to change the default
|
||||
``python`` interpreter.
|
||||
* Encourage your end users to use a virtual environment.
|
||||
This makes the user's environment more predictable (possibly resulting
|
||||
in fewer issues), and helps avoid disrupting their system.
|
||||
* For scripts that are only expected to be run in an activated virtual
|
||||
environment, shebang lines can be written as ``#!/usr/bin/env python``,
|
||||
as this instructs the script to respect the active virtual environment.
|
||||
* In cases where the script is expected to be executed outside virtual
|
||||
environments, developers will need to be aware of the following
|
||||
discrepancies across platforms and installation methods:
|
||||
|
||||
* Older Linux distributions will provide a ``python`` command that
|
||||
refers to Python 2, and will likely not provide a ``python2`` command.
|
||||
* Some newer Linux distributions will provide a ``python`` command that
|
||||
refers to Python 3.
|
||||
* Some Linux distributions will not provide a ``python`` command at
|
||||
all by default, but will provide a ``python3`` command by default.
|
||||
|
||||
* When potentially targeting these environments, developers may either
|
||||
use a Python package installation tool that rewrites shebang lines for
|
||||
the installed environment, provide instructions on updating shebang lines
|
||||
interactively, or else use more specific shebang lines that are
|
||||
tailored to the target environment.
|
||||
* Scripts targeting both “*old systems*” and systems without the default
|
||||
``python`` command need to make a compromise and document this situation.
|
||||
Avoiding shebangs (via the console_scripts Entry Points ([9]_) or similar
|
||||
means) is the recommended workaround for this problem.
|
||||
* Applications designed exclusively for a specific environment (such as
|
||||
a container or virtual environment) may continue to use the ``python``
|
||||
command name.
|
||||
|
||||
For end users of Python
|
||||
-----------------------
|
||||
|
||||
* While far from being universally available, ``python`` remains the
|
||||
preferred spelling for explicitly invoking Python, as this is the
|
||||
spelling that virtual environments make consistently available
|
||||
across different platforms and Python installations.
|
||||
* For software that is not distributed with (or developed for) your system,
|
||||
we recommend using a virtual environment, possibly with an environment
|
||||
manager like ``conda`` or ``pipenv``, to help avoid disrupting your system
|
||||
Python installation.
|
||||
|
||||
|
||||
These recommendations are the outcome of the relevant python-dev discussions
|
||||
in March and July 2011 ([1]_, [2]_), February 2012 ([4]_),
|
||||
September 2014 ([6]_), and discussion on GitHub in April 2018 ([7]_).
|
||||
September 2014 ([6]_), discussion on GitHub in April 2018 ([7]_),
|
||||
on python-dev in February 2019 ([8]_), and during the PEP update review
|
||||
in May/June 2019 ([10]_).
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
History of this PEP
|
||||
===================
|
||||
|
||||
This recommendation is needed as, even though the majority of distributions
|
||||
still alias the ``python`` command to Python 2, some now alias it to
|
||||
In 2011, the majority of distributions
|
||||
aliased the ``python`` command to Python 2, but some started switching it to
|
||||
Python 3 ([5]_). As some of the former distributions did not provide a
|
||||
``python2`` command by default, there was previously no way for Python 2 code
|
||||
(or any code that invokes the Python 2 interpreter directly rather than via
|
||||
``sys.executable``) to reliably run on all Unix-like systems without
|
||||
modification, as the ``python`` command would invoke the wrong interpreter
|
||||
version on some systems, and the ``python2`` command would fail completely
|
||||
on others. The recommendations in this PEP provide a very simple mechanism
|
||||
on others. This PEP originally provided a very simple mechanism
|
||||
to restore cross-platform support, with minimal additional work required
|
||||
on the part of distribution maintainers.
|
||||
on the part of distribution maintainers. Simplified, the recommendation was:
|
||||
|
||||
1. The ``python`` command was preferred for code compatible with both
|
||||
Python 2 and 3 (since it was available on all systems, even those that
|
||||
already aliased it to Python 3).
|
||||
2. The ``python`` command should always invoke Python 2 (to prevent
|
||||
hard-to-diagnose errors when Python 2 code is run on Python 3).
|
||||
3. The ``python2`` and ``python3`` commands should be available to specify
|
||||
the version explicitly.
|
||||
|
||||
However, these recommendations implicitly assumed that Python 2 would always be
|
||||
available. As Python 2 is nearing its end of life in 2020 (PEP 373, PEP 404),
|
||||
distributions are making Python 2 optional or removing it entirely.
|
||||
This means either removing the ``python`` command or switching it to invoke
|
||||
Python 3. Some distributors also decided that their users were better served by
|
||||
ignoring the PEP's original recommendations, and provided system
|
||||
administrators with the freedom to configure their systems based on
|
||||
the needs of their particular environment.
|
||||
|
||||
|
||||
.. _rationale:
|
||||
|
||||
Current Rationale
|
||||
=================
|
||||
|
||||
As of 2019, activating a Python virtual environment (or its functional
|
||||
equivalent) prior to script execution is one way to obtain a consistent
|
||||
cross-platform and cross-distribution experience.
|
||||
|
||||
Accordingly, publishers can expect users of the software to provide a suitable
|
||||
execution environment.
|
||||
|
||||
|
||||
Future Changes to this Recommendation
|
||||
|
|
@ -111,7 +202,7 @@ Future Changes to this Recommendation
|
|||
This recommendation will be periodically reviewed over the next few years,
|
||||
and updated when the core development team judges it appropriate. As a
|
||||
point of reference, regular maintenance releases for the Python 2.7 series
|
||||
will continue until at least 2020.
|
||||
will continue until January 2020.
|
||||
|
||||
|
||||
Migration Notes
|
||||
|
|
@ -129,27 +220,25 @@ making such a change.
|
|||
and other users. Updating the ``python`` command to invoke ``python3``
|
||||
by default indicates that a distribution is willing to break such scripts
|
||||
with errors that are potentially quite confusing for users that aren't
|
||||
yet familiar with the backwards incompatible changes in Python 3. For
|
||||
familiar with the backwards incompatible changes in Python 3. For
|
||||
example, while the change of ``print`` from a statement to a builtin
|
||||
function is relatively simple for automated converters to handle, the
|
||||
SyntaxError from attempting to use the Python 2 notation in versions of
|
||||
Python 3 prior to 3.4.2 is thoroughly confusing if you aren't already
|
||||
aware of the change::
|
||||
SyntaxError from attempting to use the Python 2 notation in Python 3
|
||||
may be confusing for users that are not aware of the change::
|
||||
|
||||
$ python3 -c 'print "Hello, world!"'
|
||||
File "<string>", line 1
|
||||
print "Hello, world!"
|
||||
^
|
||||
SyntaxError: invalid syntax
|
||||
^
|
||||
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Hello, world!")?
|
||||
|
||||
(In Python 3.4.2+, that generic error message has been replaced with the
|
||||
more explicit "SyntaxError: Missing parentheses in call to 'print'")
|
||||
* Avoiding breakage of such third party scripts is the key reason this
|
||||
PEP recommends that ``python`` continue to refer to ``python2`` for the
|
||||
time being. Until the conventions described in this PEP are more widely
|
||||
adopted, having ``python`` invoke ``python2`` will remain the recommended
|
||||
option.
|
||||
* The ``pythonX.X`` (e.g. ``python2.6``) commands exist on some systems, on
|
||||
While this might be obvious for experienced Pythonistas, such scripts
|
||||
might even be run by people who are not familiar with Python at all.
|
||||
Avoiding breakage of such third party scripts was the key reason this
|
||||
PEP used to recommend that ``python`` continue to refer to ``python2``.
|
||||
* The error message ``python: command not found`` tends to be surprisingly
|
||||
actionable, even for people unfamiliar with Python.
|
||||
* The ``pythonX.X`` (e.g. ``python3.6``) commands exist on modern systems, on
|
||||
which they invoke specific minor versions of the Python interpreter. It
|
||||
can be useful for distribution-specific packages to take advantage of these
|
||||
utilities if they exist, since it will prevent code breakage if the default
|
||||
|
|
@ -162,8 +251,8 @@ making such a change.
|
|||
* When the ``pythonX.X`` binaries are provided by a distribution, the
|
||||
``python2`` and ``python3`` commands should refer to one of those files
|
||||
rather than being provided as a separate binary file.
|
||||
* It is strongly encouraged that distribution-specific packages use ``python2``
|
||||
or ``python3`` rather than ``python``, even in code that is not intended to
|
||||
* It is strongly encouraged that distribution-specific packages use ``python3``
|
||||
(or ``python2``) rather than ``python``, even in code that is not intended to
|
||||
operate on other distributions. This will reduce problems if the
|
||||
distribution later decides to change the version of the Python interpreter
|
||||
that the ``python`` command invokes, or if a sysadmin installs a custom
|
||||
|
|
@ -175,14 +264,12 @@ making such a change.
|
|||
versa. That way, if a sysadmin does decide to replace the installed
|
||||
``python`` file, they can do so without inadvertently deleting the
|
||||
previously installed binary.
|
||||
* If the Python 2 interpreter becomes uncommon, scripts should nevertheless
|
||||
continue to use the ``python3`` convention rather that just ``python``. This
|
||||
will ease transition in the event that yet another major version of Python
|
||||
is released.
|
||||
* Even as the Python 2 interpreter becomes less common, it remains reasonable
|
||||
for scripts to continue to use the ``python3`` convention, rather than just
|
||||
``python``.
|
||||
* If these conventions are adhered to, it will become the case that the
|
||||
``python`` command is only executed in an interactive manner as a user
|
||||
convenience, or to run scripts that are source compatible with both Python
|
||||
2 and Python 3.
|
||||
convenience, or else when using a virtual environment or similar mechanism.
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
|
|
@ -194,7 +281,7 @@ these commands. This is mostly a non-issue, since the sysadmin can simply
|
|||
create these symbolic links and avoid further problems. It is a significantly
|
||||
more obvious breakage than the sometimes cryptic errors that can arise when
|
||||
attempting to execute a script containing Python 2 specific syntax with a
|
||||
Python 3 interpreter.
|
||||
Python 3 interpreter or vice versa.
|
||||
|
||||
|
||||
Application to the CPython Reference Interpreter
|
||||
|
|
@ -209,7 +296,7 @@ items are relative symbolic links)::
|
|||
python -> python2 -> python2.7
|
||||
python-config -> python2-config -> python2.7-config
|
||||
|
||||
Similar adjustments were made to the Mac OS X binary installer.
|
||||
Similar adjustments were made to the macOS binary installer.
|
||||
|
||||
This feature first appeared in the default installation process in
|
||||
CPython 2.7.3.
|
||||
|
|
@ -252,8 +339,7 @@ Exclusion of MS Windows
|
|||
This PEP deliberately excludes any proposals relating to Microsoft Windows, as
|
||||
devising an equivalent solution for Windows was deemed too complex to handle
|
||||
here. PEP 397 and the related discussion on the python-dev mailing list
|
||||
address this issue (like this PEP, the PEP 397 launcher invokes Python 2 by
|
||||
default if versions of both Python 2 and 3 are installed on the system).
|
||||
address this issue.
|
||||
|
||||
|
||||
References
|
||||
|
|
@ -281,6 +367,16 @@ References
|
|||
minor edits
|
||||
(https://github.com/python/peps/pull/630)
|
||||
|
||||
.. [8] Another update for PEP 394 -- The "python" Command on Unix-Like Systems
|
||||
(https://mail.python.org/pipermail/python-dev/2019-February/156272.html)
|
||||
|
||||
.. [9] The console_scripts Entry Point
|
||||
(https://python-packaging.readthedocs.io/en/latest/command-line-scripts.html#the-console-scripts-entry-point)
|
||||
|
||||
.. [10] May 2019 PEP update review
|
||||
(https://github.com/python/peps/pull/989)
|
||||
|
||||
|
||||
Copyright
|
||||
===========
|
||||
This document has been placed in the public domain.
|
||||
|
|
|
|||
Loading…
Reference in New Issue