362 lines
14 KiB
Plaintext
362 lines
14 KiB
Plaintext
PEP: 441
|
||
Title: Improving Python ZIP Application Support
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Daniel Holth <dholth@gmail.com>,
|
||
Paul Moore <p.f.moore@gmail.com>
|
||
Discussions-To: https://mail.python.org/pipermail/python-dev/2015-February/138277.html
|
||
Status: Accepted
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 30 March 2013
|
||
Post-History: 30 March 2013, 1 April 2013, 16 February 2015
|
||
Resolution: https://mail.python.org/pipermail/python-dev/2015-February/138578.html
|
||
|
||
Improving Python ZIP Application Support
|
||
========================================
|
||
|
||
Python has had the ability to execute directories or ZIP-format
|
||
archives as scripts since version 2.6 [1]_. When invoked with a zip
|
||
file or directory as its first argument the interpreter adds that
|
||
directory to sys.path and executes the ``__main__`` module. These
|
||
archives provide a great way to publish software that needs to be
|
||
distributed as a single file script but is complex enough to need to
|
||
be written as a collection of modules.
|
||
|
||
This feature is not as popular as it should be mainly because it was
|
||
not promoted as part of Python 2.6 [2]_, so that it is relatively
|
||
unknown, but also because the Windows installer does not register a
|
||
file extension (other than ``.py``) for this format of file, to associate
|
||
with the launcher.
|
||
|
||
This PEP proposes to fix these problems by re-publicising the feature,
|
||
defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications"
|
||
and "Windowed Python ZIP Applications", and providing some simple
|
||
tooling to manage the format.
|
||
|
||
A New Python ZIP Application Extension
|
||
======================================
|
||
|
||
The terminology "Python Zip Application" will be the formal term used
|
||
for a zip-format archive that contains Python code in a form that can
|
||
be directly executed by Python (specifically, it must have a
|
||
``__main__.py`` file in the root directory of the archive). The
|
||
extension ``.pyz`` will be formally associated with such files.
|
||
|
||
The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python
|
||
Zip Applications" with the platform launcher so they can be executed.
|
||
A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a
|
||
windowed application, indicating whether the console should appear
|
||
when running the app.
|
||
|
||
On Unix, it would be ideal if the ``.pyz`` extension and the name
|
||
"Python Zip Application" were registered (in the mime types database?).
|
||
However, such an association is out of scope for this PEP.
|
||
|
||
Python Zip applications can be prefixed with a ``#!`` line
|
||
pointing to the correct Python interpreter and an optional
|
||
explanation::
|
||
|
||
#!/usr/bin/env python3
|
||
# Python application packed with zipapp module
|
||
(binary contents of archive)
|
||
|
||
On Unix, this allows the OS to run the file with the correct
|
||
interpreter, via the standard "shebang" support. On Windows, the
|
||
Python launcher implements shebang support.
|
||
|
||
However, it is always possible to execute a ``.pyz`` application by
|
||
supplying the filename to the Python interpreter directly.
|
||
|
||
As background, ZIP archives are defined with a footer containing
|
||
relative offsets from the end of the file. They remain valid when
|
||
concatenated to the end of any other file. This feature is completely
|
||
standard and is how self-extracting ZIP archives and the bdist_wininst
|
||
installer format work.
|
||
|
||
|
||
Minimal Tooling: The zipapp Module
|
||
==================================
|
||
|
||
This PEP also proposes including a module for working with these
|
||
archives. The module will contain functions for working with Python
|
||
zip application archives, and a command line interface (via ``python
|
||
-m zipapp``) for their creation and manipulation.
|
||
|
||
More complete tools for managing Python Zip Applications are
|
||
encouraged as 3rd party applications on PyPI. Currently, pyzzer [5]_
|
||
and pex [6]_ are two such tools.
|
||
|
||
Module Interface
|
||
----------------
|
||
|
||
The zipapp module will provide the following functions:
|
||
|
||
``create_archive(source, target=None, interpreter=None, main=None)``
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
||
Create an application archive from *source*. The source can be any
|
||
of the following:
|
||
|
||
* The name of a directory, in which case a new application archive
|
||
will be created from the content of that directory.
|
||
* The name of an existing application archive file, in which case the
|
||
file is copied to the target. The file name should include the
|
||
``.pyz`` or ``.pyzw`` extension, if required.
|
||
* A file object open for reading in bytes mode. The content of the
|
||
file should be an application archive, and the file object is
|
||
assumed to be positioned at the start of the archive.
|
||
|
||
The *target* argument determines where the resulting archive will be
|
||
written:
|
||
|
||
* If it is the name of a file, the archive will be written to that
|
||
file.
|
||
* If it is an open file object, the archive will be written to that
|
||
file object, which must be open for writing in bytes mode.
|
||
* If the target is omitted (or None), the source must be a directory
|
||
and the target will be a file with the same name as the source, with
|
||
a ``.pyz`` extension added.
|
||
|
||
The *interpreter* argument specifies the name of the Python
|
||
interpreter with which the archive will be executed. It is written as
|
||
a "shebang" line at the start of the archive. On Unix, this will be
|
||
interpreted by the OS, and on Windows it will be handled by the Python
|
||
launcher. Omitting the *interpreter* results in no shebang line being
|
||
written. If an interpreter is specified, and the target is a
|
||
filename, the executable bit of the target file will be set.
|
||
|
||
The *main* argument specifies the name of a callable which will be
|
||
used as the main program for the archive. It can only be specified if
|
||
the source is a directory, and the source does not already contain a
|
||
``__main__.py`` file. The *main* argument should take the form
|
||
"pkg.module:callable" and the archive will be run by importing
|
||
"pkg.module" and executing the given callable with no arguments. It
|
||
is an error to omit *main* if the source is a directory and does not
|
||
contain a ``__main__.py`` file, as otherwise the resulting archive
|
||
would not be executable.
|
||
|
||
If a file object is specified for *source* or *target*, it is the
|
||
caller's responsibility to close it after calling create_archive.
|
||
|
||
When copying an existing archive, file objects supplied only need
|
||
``read`` and ``readline``, or ``write`` methods. When creating an
|
||
archive from a directory, if the target is a file object it will be
|
||
passed to the ``zipfile.ZipFile`` class, and must supply the methods
|
||
needed by that class.
|
||
|
||
``get_interpreter(archive)``
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
||
Returns the interpreter specified in the shebang line of the
|
||
*archive*. If there is no shebang, the function returns ``None``.
|
||
The *archive* argument can be a filename or a file-like object open
|
||
for reading in bytes mode.
|
||
|
||
|
||
Command Line Usage
|
||
------------------
|
||
|
||
The zipapp module can be run with the python ``-m`` flag. The command
|
||
line interface is as follows::
|
||
|
||
python -m zipapp directory [options]
|
||
|
||
Create an archive from the given directory. An archive will
|
||
be created from the contents of that directory. The archive
|
||
will have the same name as the source directory with a .pyz
|
||
extension.
|
||
|
||
The following options can be specified:
|
||
|
||
-o archive / --output archive
|
||
|
||
The destination archive will have the specified name. The
|
||
given name will be used as written, so should include the
|
||
".pyz" or ".pyzw" extension.
|
||
|
||
-p interpreter / --python interpreter
|
||
|
||
The given interpreter will be written to the shebang line
|
||
of the archive. If this option is not given, the archive
|
||
will have no shebang line.
|
||
|
||
-m pkg.mod:fn / --main pkg.mod:fn
|
||
|
||
The source directory must not have a __main__.py file. The
|
||
archiver will write a __main__.py file into the target
|
||
which calls fn from the module pkg.mod.
|
||
|
||
The behaviour of the command line interface matches that of
|
||
``zipapp.create_archive()``.
|
||
|
||
In addition, it is possible to use the command line interface to work
|
||
with an existing archive::
|
||
|
||
python -m zipapp app.pyz --show
|
||
|
||
Displays the shebang line of an archive. Output is of the
|
||
form
|
||
|
||
Interpreter: /usr/bin/env
|
||
or
|
||
Interpreter: <none>
|
||
|
||
and is intended for diagnostic use, not for scripts.
|
||
|
||
python -m zipapp app.pyz -o newapp.pyz [-p interpreter]
|
||
|
||
Copy app.pyz to newapp.pyz, modifying the shebang line based
|
||
on the -p option (as for creating an archive, no -p option
|
||
means remove the shebang line). Specifying a destination is
|
||
mandatory.
|
||
|
||
In-place modification of an archive is *not* supported, as the
|
||
risk of damaging archives is too great for a simple tool.
|
||
|
||
As noted, the archives are standard zip files, and so can be unpacked
|
||
using any standard ZIP utility or Python's zipfile module. For this
|
||
reason, no interfaces to list the contents of an archive, or unpack
|
||
them, are provided or needed.
|
||
|
||
FAQ
|
||
---
|
||
|
||
Are you sure a standard ZIP utility can handle ``#!`` at the beginning?
|
||
Absolutely. The zipfile specification allows for arbitrary data to
|
||
be prepended to a zipfile. This feature is commonly used by
|
||
"self-extracting zip" programs. If your archive program can't
|
||
handle this, it is a bug in your archive program.
|
||
|
||
Isn't zipapp just a very thin wrapper over the zipfile module?
|
||
Yes. If you prefer to build your own Python zip application
|
||
archives using other tools, they will work just as well. The
|
||
zipapp module is a convenience, nothing more.
|
||
|
||
Why not use just use a ``.zip`` or ``.py`` extension?
|
||
Users expect a ``.zip`` file to be opened with an archive tool, and
|
||
expect a ``.py`` file to contain readable text. Both would be
|
||
confusing for this use case.
|
||
|
||
How does this compete with existing package formats?
|
||
The sdist, bdist and wheel formats are designed for packaging of
|
||
modules to be installed into an existing Python installation.
|
||
They are not intended to be used without installing. The
|
||
executable zip format is specifically designed for standalone use,
|
||
without needing to be installed. They are in effect a multi-file
|
||
version of a standalone Python script.
|
||
|
||
Rejected Proposals
|
||
==================
|
||
|
||
Convenience Values for Shebang Lines
|
||
------------------------------------
|
||
|
||
Is it worth having "convenience" forms for any of the common
|
||
interpreter values? For example, ``-p 3`` meaning the same as ``-p
|
||
"/usr/bin/env python3"``. It would save a lot of typing for the
|
||
common cases, as well as giving cross-platform options for people who
|
||
don't want or need to understand the intricacies of shebang handling
|
||
on "other" platforms.
|
||
|
||
Downsides are that it's not obvious how to translate the
|
||
abbreviations. For example, should "3" mean "/usr/bin/env python3",
|
||
"/usr/bin/python3", "python3", or something else? Also, there is no
|
||
obvious short form for the key case of "/usr/bin/env python" (any
|
||
available version of Python), which could easily result in scripts
|
||
being written with overly-restrictive shebang lines.
|
||
|
||
Overall, this seems like there are more problems than benefits, and as
|
||
a result has been dropped from consideration.
|
||
|
||
Registering ``.pyz`` as a Media Type
|
||
------------------------------------
|
||
|
||
It was suggested [3]_ that the ``.pyz`` extension should be registered
|
||
in the Unix database of extensions. While it makes sense to do this
|
||
as an equivalent of the Windows installer registering the extension,
|
||
the ``.py`` extension is not listed in the media types database [4]_.
|
||
It doesn't seem reasonable to register ``.pyz`` without ``.py``, so
|
||
this idea has been omitted from this PEP. An interested party could
|
||
arrange for *both* ``.py`` and ``.pyz`` to be registered at a future
|
||
date.
|
||
|
||
Default Interpreter
|
||
-------------------
|
||
|
||
The initial draft of this PEP proposed using ``/usr/bin/env python``
|
||
as the default interpreter. Unix users have problems with this
|
||
behaviour, as the default for the python command on many distributions
|
||
is Python 2, and it is felt that this PEP should prefer Python 3 by
|
||
default. However, using a command of ``python3`` can result in
|
||
unexpected behaviour for Windows users, where the default behaviour of
|
||
the launcher for the command ``python`` is commonly customised by users,
|
||
but the behaviour of ``python3`` may not be modified to match.
|
||
|
||
As a result, the principle "in the face of ambiguity, refuse to guess"
|
||
has been invoked, and archives have no shebang line unless explicitly
|
||
requested. On Windows, the archives will still be run (with the
|
||
default Python) by the launcher, and on Unix, the archives can be run
|
||
by explicitly invoking the desired Python interpreter.
|
||
|
||
Command Line Tool to Manage Shebang Lines
|
||
-----------------------------------------
|
||
|
||
It is conceivable that users would want to modify the shebang line for
|
||
an existing archive, or even just display the current shebang line.
|
||
This is tricky to do so with existing tools (zip programs typically
|
||
ignore prepended data totally, and text editors can have trouble
|
||
editing files containing binary data).
|
||
|
||
The zipapp module provides functions to handle the shebang line, but
|
||
does not include a command line interface to that functionality. This
|
||
is because it is not clear how to provide one without the resulting
|
||
interface being over-complex and potentially confusing. Changing the
|
||
shebang line is expected to be an uncommon requirement.
|
||
|
||
Reference Implementation
|
||
========================
|
||
|
||
A reference implementation is at http://bugs.python.org/issue23491.
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] Allow interpreter to execute a zip file
|
||
(http://bugs.python.org/issue1739468)
|
||
|
||
.. [2] Feature is not documented
|
||
(http://bugs.python.org/issue17359)
|
||
|
||
.. [3] Discussion of adding a .pyz mime type on python-dev
|
||
(https://mail.python.org/pipermail/python-dev/2015-February/138338.html)
|
||
|
||
.. [4] Register of media types
|
||
(http://www.iana.org/assignments/media-types/media-types.xhtml)
|
||
|
||
.. [5] pyzzer - A tool for creating Python-executable archives
|
||
(https://pypi.python.org/pypi/pyzzer)
|
||
|
||
.. [6] pex - The PEX packaging toolchain
|
||
(https://pypi.python.org/pypi/pex)
|
||
|
||
The discussion of this PEP took place on the python-dev mailing list,
|
||
in the thread starting at
|
||
https://mail.python.org/pipermail/python-dev/2015-February/138277.html
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed into the public domain.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|