404 lines
20 KiB
Plaintext
404 lines
20 KiB
Plaintext
PEP: 397
|
||
Title: Python launcher for Windows
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Mark Hammond <mhammond@skippinet.com.au>
|
||
Status: Draft
|
||
Type: Standards Track
|
||
Content-Type: text/plain
|
||
Created: 15-Mar-2011
|
||
Post-History: 17-May-2011, 15-Mar-2011
|
||
|
||
Abstract
|
||
|
||
This PEP describes a Python launcher for the Windows platform. A
|
||
Python launcher is a single executable which uses a number of
|
||
heuristics to locate a Python executable and launch it with a
|
||
specified command line.
|
||
|
||
|
||
Rationale
|
||
|
||
Windows provides "file associations" so an executable can be associated
|
||
with an extension, allowing for scripts to be executed directly in some
|
||
contexts (eg., double-clicking the file in Windows Explorer.) Until now,
|
||
a strategy of "last installed Python wins" has been used and while not
|
||
ideal, has generally been workable due to the conservative changes in
|
||
Python 2.x releases. As Python 3.x scripts are often syntactically
|
||
incompatible with Python 2.x scripts, a different strategy must be used
|
||
to allow files with a '.py' extension to use a different executable based
|
||
on the Python version the script targets. This will be done by borrowing
|
||
the existing practices of another operating system - scripts will be able
|
||
to nominate the version of Python they need by way of a "shebang" line, as
|
||
described below.
|
||
|
||
Unix-like operating systems (referred to simply as "Unix" in this
|
||
PEP) allow scripts to be executed as if they were executable images
|
||
by examining the script for a "shebang" line which specifies the
|
||
actual executable to be used to run the script. This is described in
|
||
detail in the evecve(2) man page [1] and while user documentation will
|
||
be created for this feature, for the purposes of this PEP that man
|
||
page describes a valid shebang line.
|
||
|
||
Additionally, these operating systems provide symbolic-links to
|
||
Python executables in well-known directories. For example, many
|
||
systems will have a link /usr/bin/python which references a
|
||
particular version of Python installed under the operating-system.
|
||
These symbolic links allow Python to be executed without regard for
|
||
where Python it actually installed on the machine (eg., without
|
||
requiring the path where Python is actually installed to be
|
||
referenced in the shebang line or in the PATH.) PEP 394 'The "python"
|
||
command on Unix-Like Systems' [2] describes additional conventions
|
||
for more fine-grained specification of a particular Python version.
|
||
|
||
These 2 facilities combined allow for a portable and somewhat
|
||
predictable way of both starting Python interactively and for allowing
|
||
Python scripts to execute. This PEP describes an implementation of a
|
||
launcher which can offer the same benefits for Python on the Windows
|
||
platform and therefore allows the launcher to be the executable
|
||
associated with '.py' files to support multiple Python versions
|
||
concurrently.
|
||
|
||
While this PEP offers the ability to use a shebang line which should
|
||
work on both Windows and Unix, this is not the primary motivation for
|
||
this PEP - the primary motivation is to allow a specific version to be
|
||
specified without inventing new syntax or conventions to describe it.
|
||
|
||
An overview of the launcher.
|
||
|
||
This PEP outlines the general functionality and key guidelines of a
|
||
launcher for Windows. It is accompanied by a reference implementation
|
||
[3], written in Python, which defines the detailed implementation. Over
|
||
time, changes to the implementation may be desired - if the changes
|
||
adhere to the guidelines in this PEP and have been made following
|
||
the standard Python development model this PEP need not change.
|
||
In other words, this PEP makes no attempt to describe in detail every
|
||
feature offered by the launcher but instead to offer guidelines the
|
||
launcher should adhere to.
|
||
|
||
The actual implementation of the launcher will be written in C and
|
||
will not link directly or indirectly with Python, but the Python
|
||
based reference implementation should always remain the canonical
|
||
description of the functionality. Any changes proposed to the
|
||
launcher functionality should first be made and agreed to in the
|
||
Python implementation, after which time the C implementation can be
|
||
changed to reflect the new reference. Any deviations between the
|
||
functionality of the reference implementation versus the C
|
||
implementation due should imply the C implementation is in error
|
||
(although some exceptions will need to be made due to the nature of the
|
||
reference implementation being in Python)
|
||
|
||
It is expected that 2 versions of the launcher will be needed - one
|
||
which is a console program and one which is a "windows" (ie., GUI)
|
||
program. These 2 launchers correspond to the 'python.exe' and
|
||
'pythonw.exe' executables which currently ship with Python. The
|
||
console launcher will be named 'py.exe' and the Windows one named
|
||
'pyw.exe'. The "windows" (ie., GUI) version of the launcher will attempt
|
||
to locate and launch pythonw.exe even if a virtual shebang line nominates
|
||
simply "python" - infact, the trailing 'w' notation will not be supported
|
||
in the virtual shebang line at all.
|
||
|
||
The launcher will be distributed with all future versions of Python
|
||
and if possible, should be installed somewhere likely to already be
|
||
on the system PATH (eg., the Windows System32) directory. If installed,
|
||
the "console" version of the launcher should be associated with .py files
|
||
and the "windows" version associated with .pyw files.
|
||
|
||
The launcher will not be tied to a specific version of Python - eg., a
|
||
launcher distributed with Python 3.3 should be capable of locating and
|
||
executing any Python 2.x and Python 3.x version. Future versions of the
|
||
launcher should remain backwards compatible with older versions, so later
|
||
versions of Python can install an updated version of the launcher without
|
||
impacting how the previously installed version of the launcher is used.
|
||
|
||
The launcher will offer some conveniences for Python developers working
|
||
interactively - for example, starting the launcher with no command-line
|
||
arguments will launch the default Python with no command-line arguments.
|
||
Further, command-line arguments will be supported to allow a specific
|
||
Python version to be launched interactively - however, these conveniences
|
||
will not detract from the primary purpose of launching scripts and will
|
||
be easy to avoid if desired.
|
||
|
||
Guidelines for a Python launcher.
|
||
|
||
The Python launcher described in this PEP will intentionally be
|
||
constrained to the use-cases described in the Rationale section
|
||
above. It will not attempt to be a general purpose script launcher
|
||
or shebang processor.
|
||
|
||
The launcher should support for format of shebang lines as described
|
||
in [1], including all restrictions listed.
|
||
|
||
The launcher should support shebang lines commonly found on Unix.
|
||
For example, a shebang line of '!# /usr/bin/python' should work even
|
||
though there is unlikely to be an executable in the relative Windows
|
||
directory "\usr\bin". This means that many scripts can use a single
|
||
shebang line and be likely to work on both Unix and Windows without
|
||
modification.
|
||
|
||
The launcher will support fully-qualified paths to executables.
|
||
While this will make the script inherently non-portable, it is a
|
||
feature offered by Unix and would be useful for Windows users in
|
||
some cases.
|
||
|
||
The launcher will be capable of supporting implementations other than
|
||
CPython, such as jython and IronPython, but given both the absence of
|
||
common links on Unix (such as "/usr/bin/jython") and the inability for the
|
||
launcher to automatically locate the installation location of these
|
||
implementations on Windows, the launcher will support this via
|
||
customization options. Scripts taking advantage of this will not be
|
||
portable (as these customization options must be set to reflect the
|
||
configuration of the machine on which the launcher is running) but this
|
||
ability is nonetheless considered worthwhile.
|
||
|
||
On Unix, the user can control which specific version of Python is used
|
||
by adjusting the links in /usr/bin to point to the desired version. As
|
||
the launcher on Windows will not use Windows links, environment variables
|
||
will be used to override the semantics for determining exactly what
|
||
version of Python will be used. For example, while a shebang line of
|
||
"/usr/bin/python2" will automatically locate a Python 2.x implementation,
|
||
an environment variable can override exactly which Python 2.x
|
||
implementation will be chosen. Similarly for "/usr/bin/python" and
|
||
"/usr/bin/python3". This is specified in detail later in this PEP.
|
||
|
||
Shebang line parsing
|
||
|
||
If the first command-line argument does not start with a dash ('-')
|
||
character, an attempt will be made to open that argument as a file
|
||
and parsed for a shebang line according to the rules in [1]. Once
|
||
parsed, the command will be categorized according to the following rules:
|
||
|
||
* If the command starts with the definition of a customized command
|
||
followed by a space character, the customized command will be used.
|
||
See below for a description of customized commands.
|
||
|
||
* If the command starts with the strings '/usr/bin/python',
|
||
'/usr/bin/env python' or 'python' the command will be treated as a
|
||
'virtual command' and the rules described in Python Version Qualifiers
|
||
(below) will be used to locate the executable to use.
|
||
|
||
* Otherwise the command is assumed to be directly ready to execute - ie.
|
||
a fully-qualified path optionally followed by arguments. The contents
|
||
of the string will not be parsed - it will be passed directly to the
|
||
Windows CreateProcess function after appending the name of the script
|
||
and the launcher command-line arguments. This means that the rules used
|
||
by CreateProcess will be used, including how relative path names and
|
||
executable references without extensions are treated. Notably, the
|
||
Windows command processor will not be used, so special rules used by the
|
||
command processor (such as automatic appending of extensions other than
|
||
'.exe', support for batch files, etc) will not be used.
|
||
|
||
The use of 'virtual' shebang lines will be encouraged as this should
|
||
allow for portable shebang lines to be specified which work on
|
||
multiple operating systems and different installations of the same
|
||
operating system.
|
||
|
||
If the first argument can not be opened as a file or if no valid
|
||
shebang line can be found, the launcher will act as if a shebang line of
|
||
'!#python' was found - ie., a default Python interpreter will be
|
||
located and the arguments passed to that. However, if a valid
|
||
shebang line is found but the process specified by that line can not
|
||
be started, the default interpreter will not be started - the error
|
||
to create the specified child process will cause the launcher to display
|
||
an appropriate message and terminate with a specific exit code.
|
||
|
||
Virtual commands in shebang lines:
|
||
|
||
Virtual Commands are shebang lines which start with the strings
|
||
'/usr/bin/python', '/usr/bin/env python' or 'python'. Optionally, the
|
||
virtual command may be suffixed with a version qualifier (see below),
|
||
such as '/usr/bin/python2' or '/usr/bin/python3.2'. The command executed
|
||
is based on the rules described in Python Version Qualifiers below.
|
||
|
||
Customized Commands:
|
||
|
||
The launcher will support the ability to define "Customized Commands" in a
|
||
Windows .ini file (ie, a file which can be parsed by the Windows function
|
||
GetPrivateProfileString). A section called '[commands]' can be created
|
||
with key names defining the virtual command and the value specifying the
|
||
actual command-line to be used for this virtual command.
|
||
|
||
For example, if an INI file has the contents:
|
||
|
||
[commands]
|
||
vpython=c:\bin\vpython.exe -foo
|
||
|
||
Then a shebang line of '#! vpython' in a script named 'doit.py' will
|
||
result in the launcher using the command-line 'c:\bin\vpython.exe -foo
|
||
doit.py'
|
||
|
||
Two .ini files will be searched by the launcher - 'py.ini' in the
|
||
current user's "application data" directory (ie, the directory returned
|
||
by calling the Windows function SHGetFolderPath with CSIDL_APPDATA) and
|
||
'py.ini' in the same directory as the launcher. Commands specified
|
||
in the "application directory" will have precendence over the one next to
|
||
the executable, and the same .ini files will be used by both the Windows
|
||
and Console versions of the launcher.
|
||
|
||
Python Version Qualifiers
|
||
|
||
Some of the features described allow an optional Python version qualifier
|
||
to be used.
|
||
|
||
A version qualifier starts with a major version number and can optionally
|
||
be followed by a period ('.') and a minor version specifier. If the minor
|
||
qualifier is specified, it may optionally be followed by "-32" to indicate
|
||
the 32bit implementation of that version be used. Note that no "-64"
|
||
qualifier is necessary as this is the default implementation (see below).
|
||
|
||
If no version qualifiers are found, the environment variable PY_PYTHON can
|
||
be set to specify the default version qualifier - the default value is
|
||
"2". Note this value could specify just a major version (eg., "2") or a
|
||
major.minor qualifier (eg., "2.6"), or even major.minor-32.
|
||
|
||
If no minor version qualfiers are found, the environment variable
|
||
PY_PYTHON{major} (where {major} is the current major version qualifier
|
||
as determined above) can be set to specify the full version. If no such
|
||
option is found, the launcher will enumerate the installed Python
|
||
versions and use the latest minor release found for the major version,
|
||
which is likely, although not guaranteed, to be the most recently
|
||
installed version in that family.
|
||
|
||
On 64bit Windows with both 32bit and 64bit implementations of the
|
||
same (major.minor) Python version installed, the 64bit version will
|
||
always be preferred. This will be true for both 32bit and 64bit
|
||
implementations of the launcher - a 32bit launcher will prefer to
|
||
execute a 64bit Python installation of the specified version if
|
||
available. This is so the behavior of the launcher can be predicted
|
||
knowing only what versions are installed on the PC and without
|
||
regard to the order in which they were installed. As noted above, an
|
||
optional "-32" suffix can be used on a version specifier to change this
|
||
behaviour.
|
||
|
||
Examples:
|
||
|
||
* If no relevant options are set, the commands 'python' and
|
||
'python2' will use the latest Python 2.x version installed and
|
||
the command 'python3' will use the lastest Python 3.x installed.
|
||
|
||
* The commands 'python3.1' and 'python2.7' will not consult any
|
||
options at all as the versions are fully specified.
|
||
|
||
* if 'PY_PYTHON=3', the commands 'python' and 'python3' will both use
|
||
the latest installed Python 3 version.
|
||
|
||
* if 'PY_PYTHON=3.1-32', the command 'python' will use the 32bit
|
||
implementation of 3.1 whereas the command 'python3' will use the latest
|
||
installed Python (PY_PYTHON was not considered at all as a major
|
||
version was specified.)
|
||
|
||
* if 'PY_PYTHON=3' and 'PY_PYTHON3=3.1', the commands
|
||
'python' and 'python3' will both use specifically 3.1
|
||
|
||
In addition to environment variables, the same settings can be configured
|
||
in the .INI file used by the launcher. The section in the INI file will
|
||
be called '[defaults]' and the key name will be the same as the
|
||
environment variables without the leading 'PY_' prefix (and note that
|
||
the key names in the INI file are case insensitive. The contents of
|
||
an environment variable will override things specified in the INI file.
|
||
|
||
For example:
|
||
* Setting 'PY_PYTHON=3.1' is equivilent to the INI file
|
||
containing:
|
||
[defaults]
|
||
python=3.1
|
||
|
||
* Setting 'PY_PYTHON=3' and 'PY_PYTHON3=3.1' is equivilent to the INI file
|
||
containing:
|
||
[defaults]
|
||
python=3
|
||
python3=3.1
|
||
|
||
Command-line handling
|
||
|
||
Only the first command-line argument will be checked for a shebang line
|
||
and only if that argument does not start with a '-'.
|
||
|
||
If the only command-line argument is "-h" or "--help", the launcher will
|
||
print a small banner and command-line usage, then pass the argument to
|
||
the default Python. This will cause help for the launcher being printed
|
||
followed by help for Python itself. The output from the launcher will
|
||
attempt to clearly indicate the extended help information is coming from
|
||
the launcher and not Python.
|
||
|
||
As a concession to interactively launching Python, the launcher will
|
||
support the first command-line argument optionally being a dash ("-")
|
||
followed by a version qualifier, as described above, to nominate a
|
||
specific version be used. For example, while "py.exe" may locate and
|
||
launch the latest Python 2.x implementation installed, a command-line such
|
||
as "py.exe -3" could specify the latest Python 3.x implementation be
|
||
launched, while "py.exe -2.6-32" could specify a 32bit implementation
|
||
Python 2.6 be located and launched. If a Python 2.x implementation is
|
||
desired to be launched with the -3 flag, the command-line would need to be
|
||
similar to "py.exe -2 -3" (or the specific version of Python could
|
||
obviously be launched manually without use of this launcher.) Note that
|
||
this feature can not be used with shebang processing as the file scanned
|
||
for a shebang line and this argument must both be the first argument and
|
||
therefore are mutually exclusive.
|
||
|
||
All other arguments will be passed untouched to the child Python process.
|
||
|
||
Process Launching
|
||
|
||
Ideally, the launcher process would execute Python directly inside
|
||
the same process, primarily so the parent of the launcher process could
|
||
terminate the launcher and have the Python interpreter terminate. If the
|
||
launcher executes Python as a sub-process and the parent of the launcher
|
||
terminates the launcher, the Python process will be unaffected.
|
||
|
||
However, there are a number of practical problems associated with this
|
||
approach. Windows does not support the execv* family of Unix functions,
|
||
so this could only be done by the launcher dynamically loading the Python
|
||
DLL, but this would have a number of side-effects. The most serious
|
||
side effect of this is that the value of sys.executable would refer to the
|
||
launcher instead of the Python implementation. Many Python scripts use the
|
||
value of sys.executable to launch child processes, and these scripts may
|
||
fail to work as expected if the launcher is used. Consider a "parent"
|
||
script with a shebang line of '#! /usr/bin/python3' which attempts to
|
||
launch a child script (with no shebang) via sys.executable - currently the
|
||
child is launched using the exact same version running the parent script.
|
||
If sys.executable referred to the launcher the child would be likely
|
||
executed using a Python 2.x version and would be likely to fail with a
|
||
SyntaxError. A solution for this would need to be found before this could
|
||
be considered.
|
||
|
||
Another hurdle is the support for alternative Python implementations
|
||
using the "customized commands" feature described above, where loading
|
||
the command dynamically into a running executable is not possible.
|
||
|
||
The final hurdle is the rules above regarding 64bit and 32bit programs -
|
||
a 32bit launcher would be unable to load the 64bit version of Python and
|
||
vice-versa.
|
||
|
||
Given these considerations, the launcher will execute it's command in a
|
||
child process, remaining alive while the child process is executing, then
|
||
terminate with the same exit code as returned by the child. To address the
|
||
concerns regarding the termination of the launcher not killing the child,
|
||
the Win32 Job API will be used to arrange so that the child process is
|
||
automatically killed when the parent is terminated (although children of
|
||
that child process will continue as is the case now.) As this Windows API
|
||
is available in Windows XP and later, this launcher will not work on
|
||
Windows 2000 or earlier.
|
||
|
||
References
|
||
|
||
[1] http://linux.die.net/man/2/execve
|
||
|
||
[2] http://www.python.org/dev/peps/pep-0394/
|
||
|
||
[3] http://bugs.python.org/issue11629
|
||
|
||
Copyright
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|