418 lines
20 KiB
Plaintext
418 lines
20 KiB
Plaintext
PEP: 397
|
||
Title: Python launcher for Windows
|
||
Version: $Revision: a57419aee37d $
|
||
Last-Modified: $Date: 2012/06/19 15:13:49 $
|
||
Author: Mark Hammond <mhammond@skippinet.com.au>,
|
||
Martin v. Löwis <martin@v.loewis.de>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/plain
|
||
Created: 15-Mar-2011
|
||
Post-History: 21-July-2011, 17-May-2011, 15-Mar-2011
|
||
|
||
Abstract
|
||
|
||
This PEP describes a Python launcher for the Windows platform. A
|
||
Python launcher is a single executable which uses a number of
|
||
heuristics to locate a Python executable and launch it with a
|
||
specified command line.
|
||
|
||
|
||
Rationale
|
||
|
||
Windows provides "file associations" so an executable can be associated
|
||
with an extension, allowing for scripts to be executed directly in some
|
||
contexts (eg., double-clicking the file in Windows Explorer.) Until now,
|
||
a strategy of "last installed Python wins" has been used and while not
|
||
ideal, has generally been workable due to the conservative changes in
|
||
Python 2.x releases. As Python 3.x scripts are often syntactically
|
||
incompatible with Python 2.x scripts, a different strategy must be used
|
||
to allow files with a '.py' extension to use a different executable based
|
||
on the Python version the script targets. This will be done by borrowing
|
||
the existing practices of another operating system - scripts will be able
|
||
to nominate the version of Python they need by way of a "shebang" line, as
|
||
described below.
|
||
|
||
Unix-like operating systems (referred to simply as "Unix" in this
|
||
PEP) allow scripts to be executed as if they were executable images
|
||
by examining the script for a "shebang" line which specifies the
|
||
actual executable to be used to run the script. This is described in
|
||
detail in the evecve(2) man page [1] and while user documentation will
|
||
be created for this feature, for the purposes of this PEP that man
|
||
page describes a valid shebang line.
|
||
|
||
Additionally, these operating systems provide symbolic-links to
|
||
Python executables in well-known directories. For example, many
|
||
systems will have a link /usr/bin/python which references a
|
||
particular version of Python installed under the operating-system.
|
||
These symbolic links allow Python to be executed without regard for
|
||
where Python it actually installed on the machine (eg., without
|
||
requiring the path where Python is actually installed to be
|
||
referenced in the shebang line or in the PATH.) PEP 394 'The "python"
|
||
command on Unix-Like Systems' [2] describes additional conventions
|
||
for more fine-grained specification of a particular Python version.
|
||
|
||
These 2 facilities combined allow for a portable and somewhat
|
||
predictable way of both starting Python interactively and for allowing
|
||
Python scripts to execute. This PEP describes an implementation of a
|
||
launcher which can offer the same benefits for Python on the Windows
|
||
platform and therefore allows the launcher to be the executable
|
||
associated with '.py' files to support multiple Python versions
|
||
concurrently.
|
||
|
||
While this PEP offers the ability to use a shebang line which should
|
||
work on both Windows and Unix, this is not the primary motivation for
|
||
this PEP - the primary motivation is to allow a specific version to be
|
||
specified without inventing new syntax or conventions to describe
|
||
it.
|
||
|
||
Specification
|
||
|
||
This PEP specifies features of the launcher; a prototype
|
||
implementation is provided in [3] which will be distributed
|
||
together with the Windows installer of Python, but will also be
|
||
available separately (but released along with the Python
|
||
installer). New features may be added to the launcher as
|
||
long as the features prescribed here continue to work.
|
||
|
||
Installation
|
||
|
||
The launcher comes in 2 versions - one which is a console program and
|
||
one which is a "windows" (ie., GUI) program. These 2 launchers correspond
|
||
to the 'python.exe' and 'pythonw.exe' executables which currently ship
|
||
with Python. The console launcher will be named 'py.exe' and the Windows
|
||
one named 'pyw.exe'. The "windows" (ie., GUI) version of the launcher
|
||
will attempt to locate and launch pythonw.exe even if a virtual shebang
|
||
line nominates simply "python" - infact, the trailing 'w' notation is
|
||
not supported in the virtual shebang line at all.
|
||
|
||
The launcher is installed into the Windows directory (see
|
||
discussion below) if installed by a privileged user. The
|
||
stand-alone installer asks for an alternative location of the
|
||
installer, and adds that location to the user's PATH.
|
||
|
||
The installation in the Windows directory is a 32-bit executable
|
||
(see discussion); the standalone installer may also offer to install
|
||
64-bit versions of the launcher.
|
||
|
||
The launcher installation is registered in
|
||
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CurrentVersion\SharedDLLs
|
||
with a reference counter.
|
||
It contains a version resource matching the version number of the
|
||
pythonXY.dll with which it is distributed. Independent
|
||
installations will overwrite older version
|
||
of the launcher with newer versions. Stand-alone releases use
|
||
a release level of 0x10 in FIELD3 of the CPython release on which
|
||
they are based.
|
||
|
||
Once installed, the "console" version of the launcher is
|
||
associated with .py files and the "windows" version associated with .pyw
|
||
files.
|
||
|
||
The launcher is not tied to a specific version of Python - eg., a
|
||
launcher distributed with Python 3.3 should be capable of locating and
|
||
executing any Python 2.x and Python 3.x version. However, the
|
||
launcher binaries have a version resource that is the same as the
|
||
version resource in the Python binaries that they are released with.
|
||
|
||
Python Script Launching
|
||
|
||
The launcher is restricted to launching Python scripts.
|
||
It is not intended as a general-purpose script launcher or
|
||
shebang processor.
|
||
|
||
The launcher supports the syntax of shebang lines as described
|
||
in [1], including all restrictions listed.
|
||
|
||
The launcher supports shebang lines referring to Python
|
||
executables with any of the (regex) prefixes "/usr/bin/", "/usr/local/bin"
|
||
and "/usr/bin/env *", as well as binaries specified without
|
||
|
||
For example, a shebang line of '#! /usr/bin/python' should work even
|
||
though there is unlikely to be an executable in the relative Windows
|
||
directory "\usr\bin". This means that many scripts can use a single
|
||
shebang line and be likely to work on both Unix and Windows without
|
||
modification.
|
||
|
||
The launcher will support fully-qualified paths to executables.
|
||
While this will make the script inherently non-portable, it is a
|
||
feature offered by Unix and would be useful for Windows users in
|
||
some cases.
|
||
|
||
The launcher will be capable of supporting implementations other than
|
||
CPython, such as jython and IronPython, but given both the absence of
|
||
common links on Unix (such as "/usr/bin/jython") and the inability for the
|
||
launcher to automatically locate the installation location of these
|
||
implementations on Windows, the launcher will support this via
|
||
customization options. Scripts taking advantage of this will not be
|
||
portable (as these customization options must be set to reflect the
|
||
configuration of the machine on which the launcher is running) but this
|
||
ability is nonetheless considered worthwhile.
|
||
|
||
On Unix, the user can control which specific version of Python is used
|
||
by adjusting the links in /usr/bin to point to the desired version. As
|
||
the launcher on Windows will not use Windows links, cutomization options
|
||
(exposed via both environment variables and INI files) will be used to
|
||
override the semantics for determining what version of Python will be
|
||
used. For example, while a shebang line of "/usr/bin/python2" will
|
||
automatically locate a Python 2.x implementation, an environment variable
|
||
can override exactly which Python 2.x implementation will be chosen.
|
||
Similarly for "/usr/bin/python" and "/usr/bin/python3". This is
|
||
specified in detail later in this PEP.
|
||
|
||
Shebang line parsing
|
||
|
||
If the first command-line argument does not start with a dash ('-')
|
||
character, an attempt will be made to open that argument as a file
|
||
and parsed for a shebang line according to the rules in [1]::
|
||
|
||
#! interpreter [optional-arg]
|
||
|
||
Once parsed, the command will be categorized according to the following rules:
|
||
|
||
* If the command starts with the definition of a customized command
|
||
followed by a whitespace character (including a newline), the customized
|
||
command will be used. See below for a description of customized
|
||
commands.
|
||
|
||
* The launcher will define a set of prefixes which are considered Unix
|
||
compatible commands to launch Python, namely "/usr/bin/python",
|
||
"/usr/local/bin/python", "/usr/bin/env python", and "python".
|
||
If a command starts with one of these strings will be treated as a
|
||
'virtual command' and the rules described in Python Version Qualifiers
|
||
(below) will be used to locate the executable to use.
|
||
|
||
* Otherwise the command is assumed to be directly ready to execute - ie.
|
||
a fully-qualified path (or a reference to an executable on the PATH)
|
||
optionally followed by arguments. The contents of the string will not
|
||
be parsed - it will be passed directly to the Windows CreateProcess
|
||
function after appending the name of the script and the launcher
|
||
command-line arguments. This means that the rules used by
|
||
CreateProcess will be used, including how relative path names and
|
||
executable references without extensions are treated. Notably, the
|
||
Windows command processor will not be used, so special rules used by the
|
||
command processor (such as automatic appending of extensions other than
|
||
'.exe', support for batch files, etc) will not be used.
|
||
|
||
The use of 'virtual' shebang lines is encouraged as this should
|
||
allow for portable shebang lines to be specified which work on
|
||
multiple operating systems and different installations of the same
|
||
operating system.
|
||
|
||
If the first argument can not be opened as a file or if no valid
|
||
shebang line can be found, the launcher will act as if a shebang line of
|
||
'#!python' was found - ie., a default Python interpreter will be
|
||
located and the arguments passed to that. However, if a valid
|
||
shebang line is found but the process specified by that line can not
|
||
be started, the default interpreter will not be started - the error
|
||
to create the specified child process will cause the launcher to display
|
||
an appropriate message and terminate with a specific exit code.
|
||
|
||
Configuration file
|
||
|
||
Two .ini files will be searched by the launcher - ``py.ini`` in the
|
||
current user's "application data" directory (i.e. the directory returned
|
||
by calling the Windows function SHGetFolderPath with CSIDL_LOCAL_APPDATA,
|
||
%USERPROFILE%\AppData\Local on Vista+,
|
||
%USERPROFILE%\Local Settings\Application Data on XP)
|
||
and ``py.ini`` in the same directory as the launcher. The same .ini
|
||
files are used for both the 'console' version of the launcher (i.e.
|
||
py.exe) and for the 'windows' version (i.e. pyw.exe)
|
||
|
||
|
||
Customization specified in the "application directory" will have
|
||
precedence over the one next to the executable, so a user, who may not
|
||
have write access to the .ini file next to the launcher, can override
|
||
commands in that global .ini file)
|
||
|
||
Virtual commands in shebang lines
|
||
|
||
Virtual Commands are shebang lines which start with strings which would
|
||
be expected to work on Unix platforms - examples include
|
||
'/usr/bin/python', '/usr/bin/env python' and 'python'. Optionally, the
|
||
virtual command may be suffixed with a version qualifier (see below),
|
||
such as '/usr/bin/python2' or '/usr/bin/python3.2'. The command executed
|
||
is based on the rules described in Python Version Qualifiers
|
||
below.
|
||
|
||
Customized Commands
|
||
|
||
The launcher will support the ability to define "Customized Commands" in a
|
||
Windows .ini file (ie, a file which can be parsed by the Windows function
|
||
GetPrivateProfileString). A section called '[commands]' can be created
|
||
with key names defining the virtual command and the value specifying the
|
||
actual command-line to be used for this virtual command.
|
||
|
||
For example, if an INI file has the contents:
|
||
|
||
[commands]
|
||
vpython=c:\bin\vpython.exe -foo
|
||
|
||
Then a shebang line of '#! vpython' in a script named 'doit.py' will
|
||
result in the launcher using the command-line 'c:\bin\vpython.exe -foo
|
||
doit.py'
|
||
|
||
The precise details about the names, locations and search order of the
|
||
.ini files is in the launcher documentation [4]
|
||
|
||
Python Version Qualifiers
|
||
|
||
Some of the features described allow an optional Python version qualifier
|
||
to be used.
|
||
|
||
A version qualifier starts with a major version number and can optionally
|
||
be followed by a period ('.') and a minor version specifier. If the minor
|
||
qualifier is specified, it may optionally be followed by "-32" to indicate
|
||
the 32bit implementation of that version be used. Note that no "-64"
|
||
qualifier is necessary as this is the default implementation (see below).
|
||
|
||
On 64bit Windows with both 32bit and 64bit implementations of the
|
||
same (major.minor) Python version installed, the 64bit version will
|
||
always be preferred. This will be true for both 32bit and 64bit
|
||
implementations of the launcher - a 32bit launcher will prefer to
|
||
execute a 64bit Python installation of the specified version if
|
||
available. This is so the behavior of the launcher can be predicted
|
||
knowing only what versions are installed on the PC and without
|
||
regard to the order in which they were installed (ie, without knowing
|
||
whether a 32 or 64bit version of Python and corresponding launcher was
|
||
installed last). As noted above, an optional "-32" suffix can be used
|
||
on a version specifier to change this behaviour.
|
||
|
||
If no version qualifiers are found in a command, the environment variable
|
||
``PY_PYTHON`` can be set to specify the default version qualifier - the default
|
||
value is "2". Note this value could specify just a major version (e.g. "2") or
|
||
a major.minor qualifier (e.g. "2.6"), or even major.minor-32.
|
||
|
||
If no minor version qualifiers are found, the environment variable
|
||
``PY_PYTHON{major}`` (where ``{major}`` is the current major version qualifier
|
||
as determined above) can be set to specify the full version. If no such option
|
||
is found, the launcher will enumerate the installed Python versions and use
|
||
the latest minor release found for the major version, which is likely,
|
||
although not guaranteed, to be the most recently installed version in that
|
||
family.
|
||
|
||
In addition to environment variables, the same settings can be configured
|
||
in the .INI file used by the launcher. The section in the INI file is
|
||
called ``[defaults]`` and the key name will be the same as the
|
||
environment variables without the leading ``PY\_`` prefix (and note that
|
||
the key names in the INI file are case insensitive.) The contents of
|
||
an environment variable will override things specified in the INI file.
|
||
|
||
Command-line handling
|
||
|
||
Only the first command-line argument will be checked for a shebang line
|
||
and only if that argument does not start with a '-'.
|
||
|
||
If the only command-line argument is "-h" or "--help", the launcher will
|
||
print a small banner and command-line usage, then pass the argument to
|
||
the default Python. This will cause help for the launcher being printed
|
||
followed by help for Python itself. The output from the launcher will
|
||
clearly indicate the extended help information is coming from the
|
||
launcher and not Python.
|
||
|
||
As a concession to interactively launching Python, the launcher will
|
||
support the first command-line argument optionally being a dash ("-")
|
||
followed by a version qualifier, as described above, to nominate a
|
||
specific version be used. For example, while "py.exe" may locate and
|
||
launch the latest Python 2.x implementation installed, a command-line such
|
||
as "py.exe -3" could specify the latest Python 3.x implementation be
|
||
launched, while "py.exe -2.6-32" could specify a 32bit implementation
|
||
Python 2.6 be located and launched. If a Python 2.x implementation is
|
||
desired to be launched with the -3 flag, the command-line would need to be
|
||
similar to "py.exe -2 -3" (or the specific version of Python could
|
||
obviously be launched manually without use of this launcher.) Note that
|
||
this feature can not be used with shebang processing as the file scanned
|
||
for a shebang line and this argument must both be the first argument and
|
||
therefore are mutually exclusive.
|
||
|
||
All other arguments will be passed untouched to the child Python process.
|
||
|
||
Process Launching
|
||
|
||
The launcher offers some conveniences for Python developers working
|
||
interactively - for example, starting the launcher with no command-line
|
||
arguments will launch the default Python with no command-line arguments.
|
||
Further, command-line arguments will be supported to allow a specific
|
||
Python version to be launched interactively - however, these conveniences
|
||
must not detract from the primary purpose of launching scripts and must
|
||
be easy to avoid if desired.
|
||
|
||
The launcher creates a subprocess to start the actual
|
||
interpreter. See `Discussion´ below for the rationale.
|
||
|
||
|
||
Discussion
|
||
|
||
It may be surprising that the launcher is installed into the
|
||
Windows directory, and not the System32 directory. The reason is
|
||
that the System32 directory is not on the Path of a 32-bit process
|
||
running on a 64-bit system. However, the Windows directory is
|
||
always on the path.
|
||
|
||
The launcher that is installed into the Windows directory is a 32-bit
|
||
executable so that the 32-bit CPython installer can provide the same
|
||
binary for both 32-bit and 64-bit Windows installations.
|
||
|
||
Ideally, the launcher process would execute Python directly inside
|
||
the same process, primarily so the parent of the launcher process could
|
||
terminate the launcher and have the Python interpreter terminate. If the
|
||
launcher executes Python as a sub-process and the parent of the launcher
|
||
terminates the launcher, the Python process will be unaffected.
|
||
|
||
However, there are a number of practical problems associated with this
|
||
approach. Windows does not support the execv* family of Unix functions,
|
||
so this could only be done by the launcher dynamically loading the Python
|
||
DLL, but this would have a number of side-effects. The most serious
|
||
side effect of this is that the value of sys.executable would refer to the
|
||
launcher instead of the Python implementation. Many Python scripts use the
|
||
value of sys.executable to launch child processes, and these scripts may
|
||
fail to work as expected if the launcher is used. Consider a "parent"
|
||
script with a shebang line of '#! /usr/bin/python3' which attempts to
|
||
launch a child script (with no shebang) via sys.executable - currently the
|
||
child is launched using the exact same version running the parent script.
|
||
If sys.executable referred to the launcher the child would be likely
|
||
executed using a Python 2.x version and would be likely to fail with a
|
||
SyntaxError.
|
||
|
||
Another hurdle is the support for alternative Python implementations
|
||
using the "customized commands" feature described above, where loading
|
||
the command dynamically into a running executable is not possible.
|
||
|
||
The final hurdle is the rules above regarding 64bit and 32bit programs -
|
||
a 32bit launcher would be unable to load the 64bit version of Python and
|
||
vice-versa.
|
||
|
||
Given these considerations, the launcher will execute its command in a
|
||
child process, remaining alive while the child process is executing, then
|
||
terminate with the same exit code as returned by the child. To address
|
||
concerns regarding the termination of the launcher not killing the child,
|
||
the Win32 Job API will be used to arrange so that the child process is
|
||
automatically killed when the parent is terminated (although children of
|
||
that child process will continue as is the case now.) As this Windows API
|
||
is available in Windows XP and later, this launcher will not work on
|
||
Windows 2000 or earlier.
|
||
|
||
References
|
||
|
||
[1] http://linux.die.net/man/2/execve
|
||
|
||
[2] http://www.python.org/dev/peps/pep-0394/
|
||
|
||
[3] https://bitbucket.org/vinay.sajip/pylauncher
|
||
|
||
[4] https://bitbucket.org/vinay.sajip/pylauncher/src/tip/Doc/launcher.rst
|
||
|
||
Copyright
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|