434 lines
20 KiB
Plaintext
434 lines
20 KiB
Plaintext
PEP: 397
|
|
Title: Python launcher for Windows
|
|
Version: $Revision: a57419aee37d $
|
|
Last-Modified: $Date: 2012/06/19 15:13:49 $
|
|
Author: Mark Hammond <mhammond@skippinet.com.au>,
|
|
Martin v. Löwis <martin@v.loewis.de>
|
|
Status: Final
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 15-Mar-2011
|
|
Post-History: 21-July-2011, 17-May-2011, 15-Mar-2011
|
|
Resolution: https://mail.python.org/pipermail/python-dev/2012-June/120505.html
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP describes a Python launcher for the Windows platform. A
|
|
Python launcher is a single executable which uses a number of
|
|
heuristics to locate a Python executable and launch it with a
|
|
specified command line.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
Windows provides "file associations" so an executable can be associated
|
|
with an extension, allowing for scripts to be executed directly in some
|
|
contexts (eg., double-clicking the file in Windows Explorer.) Until now,
|
|
a strategy of "last installed Python wins" has been used and while not
|
|
ideal, has generally been workable due to the conservative changes in
|
|
Python 2.x releases. As Python 3.x scripts are often syntactically
|
|
incompatible with Python 2.x scripts, a different strategy must be used
|
|
to allow files with a '.py' extension to use a different executable based
|
|
on the Python version the script targets. This will be done by borrowing
|
|
the existing practices of another operating system - scripts will be able
|
|
to nominate the version of Python they need by way of a "shebang" line, as
|
|
described below.
|
|
|
|
Unix-like operating systems (referred to simply as "Unix" in this
|
|
PEP) allow scripts to be executed as if they were executable images
|
|
by examining the script for a "shebang" line which specifies the
|
|
actual executable to be used to run the script. This is described in
|
|
detail in the ``evecve(2)`` man page [1]_ and while user documentation will
|
|
be created for this feature, for the purposes of this PEP that man
|
|
page describes a valid shebang line.
|
|
|
|
Additionally, these operating systems provide symbolic-links to
|
|
Python executables in well-known directories. For example, many
|
|
systems will have a link /usr/bin/python which references a
|
|
particular version of Python installed under the operating-system.
|
|
These symbolic links allow Python to be executed without regard for
|
|
where Python it actually installed on the machine (eg., without
|
|
requiring the path where Python is actually installed to be
|
|
referenced in the shebang line or in the ``PATH``.) PEP 394 'The "python"
|
|
command on Unix-Like Systems' [2]_ describes additional conventions
|
|
for more fine-grained specification of a particular Python version.
|
|
|
|
These 2 facilities combined allow for a portable and somewhat
|
|
predictable way of both starting Python interactively and for allowing
|
|
Python scripts to execute. This PEP describes an implementation of a
|
|
launcher which can offer the same benefits for Python on the Windows
|
|
platform and therefore allows the launcher to be the executable
|
|
associated with '.py' files to support multiple Python versions
|
|
concurrently.
|
|
|
|
While this PEP offers the ability to use a shebang line which should
|
|
work on both Windows and Unix, this is not the primary motivation for
|
|
this PEP - the primary motivation is to allow a specific version to be
|
|
specified without inventing new syntax or conventions to describe
|
|
it.
|
|
|
|
Specification
|
|
=============
|
|
|
|
This PEP specifies features of the launcher; a prototype
|
|
implementation is provided in [3]_ which will be distributed
|
|
together with the Windows installer of Python, but will also be
|
|
available separately (but released along with the Python
|
|
installer). New features may be added to the launcher as
|
|
long as the features prescribed here continue to work.
|
|
|
|
Installation
|
|
------------
|
|
|
|
The launcher comes in 2 versions - one which is a console program and
|
|
one which is a "windows" (ie., GUI) program. These 2 launchers correspond
|
|
to the 'python.exe' and 'pythonw.exe' executables which currently ship
|
|
with Python. The console launcher will be named 'py.exe' and the Windows
|
|
one named 'pyw.exe'. The "windows" (ie., GUI) version of the launcher
|
|
will attempt to locate and launch pythonw.exe even if a virtual shebang
|
|
line nominates simply "python" - in fact, the trailing 'w' notation is
|
|
not supported in the virtual shebang line at all.
|
|
|
|
The launcher is installed into the Windows directory (see
|
|
discussion below) if installed by a privileged user. The
|
|
stand-alone installer asks for an alternative location of the
|
|
installer, and adds that location to the user's ``PATH``.
|
|
|
|
The installation in the Windows directory is a 32-bit executable
|
|
(see discussion); the standalone installer may also offer to install
|
|
64-bit versions of the launcher.
|
|
|
|
The launcher installation is registered in
|
|
``HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CurrentVersion\SharedDLLs``
|
|
with a reference counter.
|
|
It contains a version resource matching the version number of the
|
|
pythonXY.dll with which it is distributed. Independent
|
|
installations will overwrite older version
|
|
of the launcher with newer versions. Stand-alone releases use
|
|
a release level of ``0x10`` in ``FIELD3`` of the CPython release on which
|
|
they are based.
|
|
|
|
Once installed, the "console" version of the launcher is
|
|
associated with .py files and the "windows" version associated with .pyw
|
|
files.
|
|
|
|
The launcher is not tied to a specific version of Python - eg., a
|
|
launcher distributed with Python 3.3 should be capable of locating and
|
|
executing any Python 2.x and Python 3.x version. However, the
|
|
launcher binaries have a version resource that is the same as the
|
|
version resource in the Python binaries that they are released with.
|
|
|
|
Python Script Launching
|
|
-----------------------
|
|
|
|
The launcher is restricted to launching Python scripts.
|
|
It is not intended as a general-purpose script launcher or
|
|
shebang processor.
|
|
|
|
The launcher supports the syntax of shebang lines as described
|
|
in [1]_, including all restrictions listed.
|
|
|
|
The launcher supports shebang lines referring to Python
|
|
executables with any of the (regex) prefixes "/usr/bin/", "/usr/local/bin"
|
|
and "/usr/bin/env \*", as well as binaries specified without
|
|
|
|
For example, a shebang line of '#! /usr/bin/python' should work even
|
|
though there is unlikely to be an executable in the relative Windows
|
|
directory "\\usr\\bin". This means that many scripts can use a single
|
|
shebang line and be likely to work on both Unix and Windows without
|
|
modification.
|
|
|
|
The launcher will support fully-qualified paths to executables.
|
|
While this will make the script inherently non-portable, it is a
|
|
feature offered by Unix and would be useful for Windows users in
|
|
some cases.
|
|
|
|
The launcher will be capable of supporting implementations other than
|
|
CPython, such as jython and IronPython, but given both the absence of
|
|
common links on Unix (such as "/usr/bin/jython") and the inability for the
|
|
launcher to automatically locate the installation location of these
|
|
implementations on Windows, the launcher will support this via
|
|
customization options. Scripts taking advantage of this will not be
|
|
portable (as these customization options must be set to reflect the
|
|
configuration of the machine on which the launcher is running) but this
|
|
ability is nonetheless considered worthwhile.
|
|
|
|
On Unix, the user can control which specific version of Python is used
|
|
by adjusting the links in /usr/bin to point to the desired version. As
|
|
the launcher on Windows will not use Windows links, customization options
|
|
(exposed via both environment variables and INI files) will be used to
|
|
override the semantics for determining what version of Python will be
|
|
used. For example, while a shebang line of "/usr/bin/python2" will
|
|
automatically locate a Python 2.x implementation, an environment variable
|
|
can override exactly which Python 2.x implementation will be chosen.
|
|
Similarly for "/usr/bin/python" and "/usr/bin/python3". This is
|
|
specified in detail later in this PEP.
|
|
|
|
Shebang line parsing
|
|
--------------------
|
|
|
|
If the first command-line argument does not start with a dash ('-')
|
|
character, an attempt will be made to open that argument as a file
|
|
and parsed for a shebang line according to the rules in [1]_::
|
|
|
|
#! interpreter [optional-arg]
|
|
|
|
Once parsed, the command will be categorized according to the following rules:
|
|
|
|
* If the command starts with the definition of a customized command
|
|
followed by a whitespace character (including a newline), the customized
|
|
command will be used. See below for a description of customized
|
|
commands.
|
|
|
|
* The launcher will define a set of prefixes which are considered Unix
|
|
compatible commands to launch Python, namely "/usr/bin/python",
|
|
"/usr/local/bin/python", "/usr/bin/env python", and "python".
|
|
If a command starts with one of these strings will be treated as a
|
|
'virtual command' and the rules described in Python Version Qualifiers
|
|
(below) will be used to locate the executable to use.
|
|
|
|
* Otherwise the command is assumed to be directly ready to execute - ie.
|
|
a fully-qualified path (or a reference to an executable on the ``PATH``)
|
|
optionally followed by arguments. The contents of the string will not
|
|
be parsed - it will be passed directly to the Windows CreateProcess
|
|
function after appending the name of the script and the launcher
|
|
command-line arguments. This means that the rules used by
|
|
CreateProcess will be used, including how relative path names and
|
|
executable references without extensions are treated. Notably, the
|
|
Windows command processor will not be used, so special rules used by the
|
|
command processor (such as automatic appending of extensions other than
|
|
'.exe', support for batch files, etc) will not be used.
|
|
|
|
The use of 'virtual' shebang lines is encouraged as this should
|
|
allow for portable shebang lines to be specified which work on
|
|
multiple operating systems and different installations of the same
|
|
operating system.
|
|
|
|
If the first argument can not be opened as a file or if no valid
|
|
shebang line can be found, the launcher will act as if a shebang line of
|
|
'#!python' was found - ie., a default Python interpreter will be
|
|
located and the arguments passed to that. However, if a valid
|
|
shebang line is found but the process specified by that line can not
|
|
be started, the default interpreter will not be started - the error
|
|
to create the specified child process will cause the launcher to display
|
|
an appropriate message and terminate with a specific exit code.
|
|
|
|
Configuration file
|
|
------------------
|
|
|
|
Two .ini files will be searched by the launcher - ``py.ini`` in the
|
|
current user's "application data" directory (i.e. the directory returned
|
|
by calling the Windows function ``SHGetFolderPath`` with ``CSIDL_LOCAL_APPDATA``,
|
|
``%USERPROFILE%\AppData\Local`` on Vista+,
|
|
``%USERPROFILE%\Local Settings\Application Data`` on XP)
|
|
and ``py.ini`` in the same directory as the launcher. The same .ini
|
|
files are used for both the 'console' version of the launcher (i.e.
|
|
py.exe) and for the 'windows' version (i.e. pyw.exe)
|
|
|
|
|
|
Customization specified in the "application directory" will have
|
|
precedence over the one next to the executable, so a user, who may not
|
|
have write access to the .ini file next to the launcher, can override
|
|
commands in that global .ini file)
|
|
|
|
Virtual commands in shebang lines
|
|
---------------------------------
|
|
|
|
Virtual Commands are shebang lines which start with strings which would
|
|
be expected to work on Unix platforms - examples include
|
|
'/usr/bin/python', '/usr/bin/env python' and 'python'. Optionally, the
|
|
virtual command may be suffixed with a version qualifier (see below),
|
|
such as '/usr/bin/python2' or '/usr/bin/python3.2'. The command executed
|
|
is based on the rules described in Python Version Qualifiers
|
|
below.
|
|
|
|
Customized Commands
|
|
-------------------
|
|
|
|
The launcher will support the ability to define "Customized Commands" in a
|
|
Windows .ini file (ie, a file which can be parsed by the Windows function
|
|
GetPrivateProfileString). A section called '[commands]' can be created
|
|
with key names defining the virtual command and the value specifying the
|
|
actual command-line to be used for this virtual command.
|
|
|
|
For example, if an INI file has the contents::
|
|
|
|
[commands]
|
|
vpython=c:\bin\vpython.exe -foo
|
|
|
|
Then a shebang line of '#! vpython' in a script named 'doit.py' will
|
|
result in the launcher using the command-line
|
|
``c:\bin\vpython.exe -foo doit.py``
|
|
|
|
The precise details about the names, locations and search order of the
|
|
.ini files is in the launcher documentation [4]_
|
|
|
|
Python Version Qualifiers
|
|
-------------------------
|
|
|
|
Some of the features described allow an optional Python version qualifier
|
|
to be used.
|
|
|
|
A version qualifier starts with a major version number and can optionally
|
|
be followed by a period ('.') and a minor version specifier. If the minor
|
|
qualifier is specified, it may optionally be followed by "-32" to indicate
|
|
the 32bit implementation of that version be used. Note that no "-64"
|
|
qualifier is necessary as this is the default implementation (see below).
|
|
|
|
On 64bit Windows with both 32bit and 64bit implementations of the
|
|
same (major.minor) Python version installed, the 64bit version will
|
|
always be preferred. This will be true for both 32bit and 64bit
|
|
implementations of the launcher - a 32bit launcher will prefer to
|
|
execute a 64bit Python installation of the specified version if
|
|
available. This is so the behavior of the launcher can be predicted
|
|
knowing only what versions are installed on the PC and without
|
|
regard to the order in which they were installed (ie, without knowing
|
|
whether a 32 or 64bit version of Python and corresponding launcher was
|
|
installed last). As noted above, an optional "-32" suffix can be used
|
|
on a version specifier to change this behaviour.
|
|
|
|
If no version qualifiers are found in a command, the environment variable
|
|
``PY_PYTHON`` can be set to specify the default version qualifier - the default
|
|
value is "2". Note this value could specify just a major version (e.g. "2") or
|
|
a major.minor qualifier (e.g. "2.6"), or even major.minor-32.
|
|
|
|
If no minor version qualifiers are found, the environment variable
|
|
``PY_PYTHON{major}`` (where ``{major}`` is the current major version qualifier
|
|
as determined above) can be set to specify the full version. If no such option
|
|
is found, the launcher will enumerate the installed Python versions and use
|
|
the latest minor release found for the major version, which is likely,
|
|
although not guaranteed, to be the most recently installed version in that
|
|
family.
|
|
|
|
In addition to environment variables, the same settings can be configured
|
|
in the .INI file used by the launcher. The section in the INI file is
|
|
called ``[defaults]`` and the key name will be the same as the
|
|
environment variables without the leading ``PY_`` prefix (and note that
|
|
the key names in the INI file are case insensitive.) The contents of
|
|
an environment variable will override things specified in the INI file.
|
|
|
|
Command-line handling
|
|
---------------------
|
|
|
|
Only the first command-line argument will be checked for a shebang line
|
|
and only if that argument does not start with a '-'.
|
|
|
|
If the only command-line argument is "-h" or "--help", the launcher will
|
|
print a small banner and command-line usage, then pass the argument to
|
|
the default Python. This will cause help for the launcher being printed
|
|
followed by help for Python itself. The output from the launcher will
|
|
clearly indicate the extended help information is coming from the
|
|
launcher and not Python.
|
|
|
|
As a concession to interactively launching Python, the launcher will
|
|
support the first command-line argument optionally being a dash ("-")
|
|
followed by a version qualifier, as described above, to nominate a
|
|
specific version be used. For example, while "py.exe" may locate and
|
|
launch the latest Python 2.x implementation installed, a command-line such
|
|
as "py.exe -3" could specify the latest Python 3.x implementation be
|
|
launched, while "py.exe -2.6-32" could specify a 32bit implementation
|
|
Python 2.6 be located and launched. If a Python 2.x implementation is
|
|
desired to be launched with the -3 flag, the command-line would need to be
|
|
similar to "py.exe -2 -3" (or the specific version of Python could
|
|
obviously be launched manually without use of this launcher.) Note that
|
|
this feature can not be used with shebang processing as the file scanned
|
|
for a shebang line and this argument must both be the first argument and
|
|
therefore are mutually exclusive.
|
|
|
|
All other arguments will be passed untouched to the child Python process.
|
|
|
|
Process Launching
|
|
-----------------
|
|
|
|
The launcher offers some conveniences for Python developers working
|
|
interactively - for example, starting the launcher with no command-line
|
|
arguments will launch the default Python with no command-line arguments.
|
|
Further, command-line arguments will be supported to allow a specific
|
|
Python version to be launched interactively - however, these conveniences
|
|
must not detract from the primary purpose of launching scripts and must
|
|
be easy to avoid if desired.
|
|
|
|
The launcher creates a subprocess to start the actual
|
|
interpreter. See **Discussion** below for the rationale.
|
|
|
|
|
|
Discussion
|
|
==========
|
|
|
|
It may be surprising that the launcher is installed into the
|
|
Windows directory, and not the System32 directory. The reason is
|
|
that the System32 directory is not on the Path of a 32-bit process
|
|
running on a 64-bit system. However, the Windows directory is
|
|
always on the path.
|
|
|
|
The launcher that is installed into the Windows directory is a 32-bit
|
|
executable so that the 32-bit CPython installer can provide the same
|
|
binary for both 32-bit and 64-bit Windows installations.
|
|
|
|
Ideally, the launcher process would execute Python directly inside
|
|
the same process, primarily so the parent of the launcher process could
|
|
terminate the launcher and have the Python interpreter terminate. If the
|
|
launcher executes Python as a sub-process and the parent of the launcher
|
|
terminates the launcher, the Python process will be unaffected.
|
|
|
|
However, there are a number of practical problems associated with this
|
|
approach. Windows does not support the ``execv*`` family of Unix functions,
|
|
so this could only be done by the launcher dynamically loading the Python
|
|
DLL, but this would have a number of side-effects. The most serious
|
|
side effect of this is that the value of sys.executable would refer to the
|
|
launcher instead of the Python implementation. Many Python scripts use the
|
|
value of ``sys.executable`` to launch child processes, and these scripts may
|
|
fail to work as expected if the launcher is used. Consider a "parent"
|
|
script with a shebang line of '#! /usr/bin/python3' which attempts to
|
|
launch a child script (with no shebang) via ``sys.executable`` - currently the
|
|
child is launched using the exact same version running the parent script.
|
|
If ``sys.executable`` referred to the launcher the child would be likely
|
|
executed using a Python 2.x version and would be likely to fail with a
|
|
``SyntaxError``.
|
|
|
|
Another hurdle is the support for alternative Python implementations
|
|
using the "customized commands" feature described above, where loading
|
|
the command dynamically into a running executable is not possible.
|
|
|
|
The final hurdle is the rules above regarding 64bit and 32bit programs -
|
|
a 32bit launcher would be unable to load the 64bit version of Python and
|
|
vice-versa.
|
|
|
|
Given these considerations, the launcher will execute its command in a
|
|
child process, remaining alive while the child process is executing, then
|
|
terminate with the same exit code as returned by the child. To address
|
|
concerns regarding the termination of the launcher not killing the child,
|
|
the Win32 Job API will be used to arrange so that the child process is
|
|
automatically killed when the parent is terminated (although children of
|
|
that child process will continue as is the case now.) As this Windows API
|
|
is available in Windows XP and later, this launcher will not work on
|
|
Windows 2000 or earlier.
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] http://linux.die.net/man/2/execve
|
|
|
|
.. [2] http://www.python.org/dev/peps/pep-0394/
|
|
|
|
.. [3] https://bitbucket.org/vinay.sajip/pylauncher
|
|
|
|
.. [4] https://bitbucket.org/vinay.sajip/pylauncher/src/tip/Doc/launcher.rst
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
coding: utf-8
|
|
End:
|