352 lines
18 KiB
Plaintext
352 lines
18 KiB
Plaintext
PEP: 397
|
||
Title: Python launcher for Windows
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Mark Hammond <mhammond@skippinet.com.au>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/plain
|
||
Created: 15-Mar-2011
|
||
Post-History: 21-July-2011, 17-May-2011, 15-Mar-2011
|
||
|
||
Abstract
|
||
|
||
This PEP describes a Python launcher for the Windows platform. A
|
||
Python launcher is a single executable which uses a number of
|
||
heuristics to locate a Python executable and launch it with a
|
||
specified command line.
|
||
|
||
|
||
Rationale
|
||
|
||
Windows provides "file associations" so an executable can be associated
|
||
with an extension, allowing for scripts to be executed directly in some
|
||
contexts (eg., double-clicking the file in Windows Explorer.) Until now,
|
||
a strategy of "last installed Python wins" has been used and while not
|
||
ideal, has generally been workable due to the conservative changes in
|
||
Python 2.x releases. As Python 3.x scripts are often syntactically
|
||
incompatible with Python 2.x scripts, a different strategy must be used
|
||
to allow files with a '.py' extension to use a different executable based
|
||
on the Python version the script targets. This will be done by borrowing
|
||
the existing practices of another operating system - scripts will be able
|
||
to nominate the version of Python they need by way of a "shebang" line, as
|
||
described below.
|
||
|
||
Unix-like operating systems (referred to simply as "Unix" in this
|
||
PEP) allow scripts to be executed as if they were executable images
|
||
by examining the script for a "shebang" line which specifies the
|
||
actual executable to be used to run the script. This is described in
|
||
detail in the evecve(2) man page [1] and while user documentation will
|
||
be created for this feature, for the purposes of this PEP that man
|
||
page describes a valid shebang line.
|
||
|
||
Additionally, these operating systems provide symbolic-links to
|
||
Python executables in well-known directories. For example, many
|
||
systems will have a link /usr/bin/python which references a
|
||
particular version of Python installed under the operating-system.
|
||
These symbolic links allow Python to be executed without regard for
|
||
where Python it actually installed on the machine (eg., without
|
||
requiring the path where Python is actually installed to be
|
||
referenced in the shebang line or in the PATH.) PEP 394 'The "python"
|
||
command on Unix-Like Systems' [2] describes additional conventions
|
||
for more fine-grained specification of a particular Python version.
|
||
|
||
These 2 facilities combined allow for a portable and somewhat
|
||
predictable way of both starting Python interactively and for allowing
|
||
Python scripts to execute. This PEP describes an implementation of a
|
||
launcher which can offer the same benefits for Python on the Windows
|
||
platform and therefore allows the launcher to be the executable
|
||
associated with '.py' files to support multiple Python versions
|
||
concurrently.
|
||
|
||
While this PEP offers the ability to use a shebang line which should
|
||
work on both Windows and Unix, this is not the primary motivation for
|
||
this PEP - the primary motivation is to allow a specific version to be
|
||
specified without inventing new syntax or conventions to describe it.
|
||
|
||
An overview of the launcher.
|
||
|
||
This PEP outlines the general functionality and key guidelines of a
|
||
launcher for Windows. It is accompanied by an implementation [3],
|
||
written in C, which defines the detailed implementation. Over
|
||
time, changes to the implementation may be desired - if the changes
|
||
adhere to the guidelines in this PEP and have been made following
|
||
the standard Python development model this PEP need not change.
|
||
In other words, this PEP makes no attempt to describe in detail every
|
||
feature offered by the launcher but instead to offer guidelines the
|
||
launcher should adhere to.
|
||
|
||
The launcher will come in 2 versions - one which is a console program and
|
||
one which is a "windows" (ie., GUI) program. These 2 launchers correspond
|
||
to the 'python.exe' and 'pythonw.exe' executables which currently ship
|
||
with Python. The console launcher will be named 'py.exe' and the Windows
|
||
one named 'pyw.exe'. The "windows" (ie., GUI) version of the launcher
|
||
will attempt to locate and launch pythonw.exe even if a virtual shebang
|
||
line nominates simply "python" - infact, the trailing 'w' notation will
|
||
not be supported in the virtual shebang line at all.
|
||
|
||
The launcher will be distributed with all future versions of Python
|
||
and if possible, should be installed directly into the Windows directory
|
||
(note that the System32 directory is not a good option as this directory
|
||
is not on the default PATH for 32bit processes on a 64bit Windows.) If
|
||
the launcher can't be installed in the Windows directory, the installer
|
||
can suggest or choose an alternative, but it will be the responsibility
|
||
of the user to ensure this directory is on the PATH.
|
||
|
||
Once installed, the "console" version of the launcher should be
|
||
associated with .py files and the "windows" version associated with .pyw
|
||
files.
|
||
|
||
The launcher will not be tied to a specific version of Python - eg., a
|
||
launcher distributed with Python 3.3 should be capable of locating and
|
||
executing any Python 2.x and Python 3.x version. Future versions of the
|
||
launcher should remain backwards compatible with older versions, so later
|
||
versions of Python can install an updated version of the launcher without
|
||
impacting how the previously installed version of the launcher is used.
|
||
|
||
The launcher may offer some conveniences for Python developers working
|
||
interactively - for example, starting the launcher with no command-line
|
||
arguments will launch the default Python with no command-line arguments.
|
||
Further, command-line arguments will be supported to allow a specific
|
||
Python version to be launched interactively - however, these conveniences
|
||
must not detract from the primary purpose of launching scripts and must
|
||
be easy to avoid if desired.
|
||
|
||
Guidelines for a Python launcher.
|
||
|
||
The Python launcher described in this PEP will intentionally be
|
||
constrained to the use-cases described in the Rationale section
|
||
above. It will not attempt to be a general purpose script launcher
|
||
or shebang processor.
|
||
|
||
The launcher should support for format of shebang lines as described
|
||
in [1], including all restrictions listed.
|
||
|
||
The launcher should support shebang lines commonly found on Unix.
|
||
For example, a shebang line of '#! /usr/bin/python' should work even
|
||
though there is unlikely to be an executable in the relative Windows
|
||
directory "\usr\bin". This means that many scripts can use a single
|
||
shebang line and be likely to work on both Unix and Windows without
|
||
modification.
|
||
|
||
The launcher will support fully-qualified paths to executables.
|
||
While this will make the script inherently non-portable, it is a
|
||
feature offered by Unix and would be useful for Windows users in
|
||
some cases.
|
||
|
||
The launcher will be capable of supporting implementations other than
|
||
CPython, such as jython and IronPython, but given both the absence of
|
||
common links on Unix (such as "/usr/bin/jython") and the inability for the
|
||
launcher to automatically locate the installation location of these
|
||
implementations on Windows, the launcher will support this via
|
||
customization options. Scripts taking advantage of this will not be
|
||
portable (as these customization options must be set to reflect the
|
||
configuration of the machine on which the launcher is running) but this
|
||
ability is nonetheless considered worthwhile.
|
||
|
||
On Unix, the user can control which specific version of Python is used
|
||
by adjusting the links in /usr/bin to point to the desired version. As
|
||
the launcher on Windows will not use Windows links, cutomization options
|
||
(exposed via both environment variables and INI files) will be used to
|
||
override the semantics for determining what version of Python will be
|
||
used. For example, while a shebang line of "/usr/bin/python2" will
|
||
automatically locate a Python 2.x implementation, an environment variable
|
||
can override exactly which Python 2.x implementation will be chosen.
|
||
Similarly for "/usr/bin/python" and "/usr/bin/python3". This is
|
||
specified in detail later in this PEP.
|
||
|
||
Shebang line parsing
|
||
|
||
If the first command-line argument does not start with a dash ('-')
|
||
character, an attempt will be made to open that argument as a file
|
||
and parsed for a shebang line according to the rules in [1]. Once
|
||
parsed, the command will be categorized according to the following rules:
|
||
|
||
* If the command starts with the definition of a customized command
|
||
followed by a whitespace character (including a newline), the customized
|
||
command will be used. See below for a description of customized
|
||
commands.
|
||
|
||
* The launcher will define a set of strings which are considered Unix
|
||
compatible commands to launch Python, such as '/usr/bin/python' etc.
|
||
If a command matching one of these strings will be treated as a
|
||
'virtual command' and the rules described in Python Version Qualifiers
|
||
(below) will be used to locate the executable to use.
|
||
|
||
* Otherwise the command is assumed to be directly ready to execute - ie.
|
||
a fully-qualified path (or a reference to an executable on the PATH)
|
||
optionally followed by arguments. The contents of the string will not
|
||
be parsed - it will be passed directly to the Windows CreateProcess
|
||
function after appending the name of the script and the launcher
|
||
command-line arguments. This means that the rules used by
|
||
CreateProcess will be used, including how relative path names and
|
||
executable references without extensions are treated. Notably, the
|
||
Windows command processor will not be used, so special rules used by the
|
||
command processor (such as automatic appending of extensions other than
|
||
'.exe', support for batch files, etc) will not be used.
|
||
|
||
The use of 'virtual' shebang lines will be encouraged as this should
|
||
allow for portable shebang lines to be specified which work on
|
||
multiple operating systems and different installations of the same
|
||
operating system.
|
||
|
||
If the first argument can not be opened as a file or if no valid
|
||
shebang line can be found, the launcher will act as if a shebang line of
|
||
'#!python' was found - ie., a default Python interpreter will be
|
||
located and the arguments passed to that. However, if a valid
|
||
shebang line is found but the process specified by that line can not
|
||
be started, the default interpreter will not be started - the error
|
||
to create the specified child process will cause the launcher to display
|
||
an appropriate message and terminate with a specific exit code.
|
||
|
||
Virtual commands in shebang lines:
|
||
|
||
Virtual Commands are shebang lines which start with strings which would
|
||
be expected to work on Unix platforms - examples include
|
||
'/usr/bin/python', '/usr/bin/env python' and 'python'. Optionally, the
|
||
virtual command may be suffixed with a version qualifier (see below),
|
||
such as '/usr/bin/python2' or '/usr/bin/python3.2'. The command executed
|
||
is based on the rules described in Python Version Qualifiers below.
|
||
|
||
Customized Commands:
|
||
|
||
The launcher will support the ability to define "Customized Commands" in a
|
||
Windows .ini file (ie, a file which can be parsed by the Windows function
|
||
GetPrivateProfileString). A section called '[commands]' can be created
|
||
with key names defining the virtual command and the value specifying the
|
||
actual command-line to be used for this virtual command.
|
||
|
||
For example, if an INI file has the contents:
|
||
|
||
[commands]
|
||
vpython=c:\bin\vpython.exe -foo
|
||
|
||
Then a shebang line of '#! vpython' in a script named 'doit.py' will
|
||
result in the launcher using the command-line 'c:\bin\vpython.exe -foo
|
||
doit.py'
|
||
|
||
The precise details about the names, locations and search order of the
|
||
.ini files is in the launcher documentation [4]
|
||
|
||
Python Version Qualifiers
|
||
|
||
Some of the features described allow an optional Python version qualifier
|
||
to be used.
|
||
|
||
A version qualifier starts with a major version number and can optionally
|
||
be followed by a period ('.') and a minor version specifier. If the minor
|
||
qualifier is specified, it may optionally be followed by "-32" to indicate
|
||
the 32bit implementation of that version be used. Note that no "-64"
|
||
qualifier is necessary as this is the default implementation (see below).
|
||
|
||
On 64bit Windows with both 32bit and 64bit implementations of the
|
||
same (major.minor) Python version installed, the 64bit version will
|
||
always be preferred. This will be true for both 32bit and 64bit
|
||
implementations of the launcher - a 32bit launcher will prefer to
|
||
execute a 64bit Python installation of the specified version if
|
||
available. This is so the behavior of the launcher can be predicted
|
||
knowing only what versions are installed on the PC and without
|
||
regard to the order in which they were installed (ie, without knowing
|
||
whether a 32 or 64bit version of Python and corresponding launcher was
|
||
installed last). As noted above, an optional "-32" suffix can be used
|
||
on a version specifier to change this behaviour.
|
||
|
||
The launcher will support various customization options to allow
|
||
fine-grained control over which specific Python version is chosen given
|
||
a partial or empty version qualifier - see the launcher documentation [4]
|
||
for details.
|
||
|
||
Command-line handling
|
||
|
||
Only the first command-line argument will be checked for a shebang line
|
||
and only if that argument does not start with a '-'.
|
||
|
||
If the only command-line argument is "-h" or "--help", the launcher will
|
||
print a small banner and command-line usage, then pass the argument to
|
||
the default Python. This will cause help for the launcher being printed
|
||
followed by help for Python itself. The output from the launcher will
|
||
clearly indicate the extended help information is coming from the
|
||
launcher and not Python.
|
||
|
||
As a concession to interactively launching Python, the launcher will
|
||
support the first command-line argument optionally being a dash ("-")
|
||
followed by a version qualifier, as described above, to nominate a
|
||
specific version be used. For example, while "py.exe" may locate and
|
||
launch the latest Python 2.x implementation installed, a command-line such
|
||
as "py.exe -3" could specify the latest Python 3.x implementation be
|
||
launched, while "py.exe -2.6-32" could specify a 32bit implementation
|
||
Python 2.6 be located and launched. If a Python 2.x implementation is
|
||
desired to be launched with the -3 flag, the command-line would need to be
|
||
similar to "py.exe -2 -3" (or the specific version of Python could
|
||
obviously be launched manually without use of this launcher.) Note that
|
||
this feature can not be used with shebang processing as the file scanned
|
||
for a shebang line and this argument must both be the first argument and
|
||
therefore are mutually exclusive.
|
||
|
||
All other arguments will be passed untouched to the child Python process.
|
||
|
||
Process Launching
|
||
|
||
Ideally, the launcher process would execute Python directly inside
|
||
the same process, primarily so the parent of the launcher process could
|
||
terminate the launcher and have the Python interpreter terminate. If the
|
||
launcher executes Python as a sub-process and the parent of the launcher
|
||
terminates the launcher, the Python process will be unaffected.
|
||
|
||
However, there are a number of practical problems associated with this
|
||
approach. Windows does not support the execv* family of Unix functions,
|
||
so this could only be done by the launcher dynamically loading the Python
|
||
DLL, but this would have a number of side-effects. The most serious
|
||
side effect of this is that the value of sys.executable would refer to the
|
||
launcher instead of the Python implementation. Many Python scripts use the
|
||
value of sys.executable to launch child processes, and these scripts may
|
||
fail to work as expected if the launcher is used. Consider a "parent"
|
||
script with a shebang line of '#! /usr/bin/python3' which attempts to
|
||
launch a child script (with no shebang) via sys.executable - currently the
|
||
child is launched using the exact same version running the parent script.
|
||
If sys.executable referred to the launcher the child would be likely
|
||
executed using a Python 2.x version and would be likely to fail with a
|
||
SyntaxError.
|
||
|
||
Another hurdle is the support for alternative Python implementations
|
||
using the "customized commands" feature described above, where loading
|
||
the command dynamically into a running executable is not possible.
|
||
|
||
The final hurdle is the rules above regarding 64bit and 32bit programs -
|
||
a 32bit launcher would be unable to load the 64bit version of Python and
|
||
vice-versa.
|
||
|
||
Given these considerations, the launcher will execute its command in a
|
||
child process, remaining alive while the child process is executing, then
|
||
terminate with the same exit code as returned by the child. To address
|
||
concerns regarding the termination of the launcher not killing the child,
|
||
the Win32 Job API will be used to arrange so that the child process is
|
||
automatically killed when the parent is terminated (although children of
|
||
that child process will continue as is the case now.) As this Windows API
|
||
is available in Windows XP and later, this launcher will not work on
|
||
Windows 2000 or earlier.
|
||
|
||
References
|
||
|
||
[1] http://linux.die.net/man/2/execve
|
||
|
||
[2] http://www.python.org/dev/peps/pep-0394/
|
||
|
||
[3] https://bitbucket.org/vinay.sajip/pylauncher
|
||
|
||
[4] https://bitbucket.org/vinay.sajip/pylauncher/src/tip/Doc/launcher.rst
|
||
|
||
Copyright
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|