python-peps/pep-0534.txt

148 lines
6.5 KiB
Plaintext
Raw Normal View History

PEP: 534
Title: Distributing a Subset of the Standard Library
Version: $Revision$
Last-Modified: $Date$
Author: Tomáš Orsava <tomas.n@orsava.cz>,
Petr Viktorin <encukou@gmail.com>,
Nick Coghlan <ncoghlan@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 5-Sep-2016
Python-Version: 3.7
Post-History:
Abstract
========
Python is often being built or distributed without its full standard library.
However, there is as of yet no standard, user friendly way of properly informing the user about the failure to import such missing `stdlib` modules. This PEP proposes a mechanism for identifying standard library modules and informing the user appropriately.
Motivation
==========
There are several use cases for including only a subset of Python's standard
library. However, there is so far no user-friendly mechanism for informing the user *why* a stdlib module is missing and how to remedy the situation appropriately.
CPython
-------
When one of Python standard library modules (such as ``_sqlite3``) cannot be
compiled during a Python build because of missing dependencies (e.g. SQLite
header files), the module is simply skipped. If you then install this compiled Python and use it to try to import one of the
missing modules, Python will fail with a ModuleNotFoundError_.
.. _ModuleNotFoundError:
https://docs.python.org/3.7/library/exceptions.html#ModuleNotFoundError
This can confuse users who may not understand why a cleanly built Python is
missing standard library modules.
Linux and other distributions
-----------------------------
Many Linux and other distributions are already separating out parts of the
standard library to standalone packages. Among the most commonly excluded
modules are the ``tkinter`` module, since it draws in a dependency on the
graphical environment, and the ``test`` package, as it only serves to test
Python internally and is about as big as the rest of the standard library put
together.
The methods of omission of these modules differ. For example, Debian patches
the file ``Lib/tkinter/__init__.py`` to envelop the line ``import _tkinter`` in
a *try-except* block and upon encountering an ``ImportError`` it simply adds
the following to the error message: ``please install the python3-tk package``
[#debian-patch]_. Fedora and other distributions simply don't include the
omitted modules, potentially leaving users baffled as to where to find them.
Specification
=============
The `sysconfig`_ module will be extended by two functions: `sysconfig.get_stdlib_modules()`, which will provide a list of the names of all Python standard library modules, and a function `sysconfig.get_optional_modules()`, that will list optional `stdlib` module names. The results of the latter function—`sysconfig.get_optional_modules()`—as well as of the existing `sys.builtin_module_names` will both be subsets of the full list provided by `sysconfig.get_stdlib_modules()`. These added lists will be generated during the Python build process and saved in `_sysconfigdata-*.py` file along with other `sysconfig`_ values.
.. _`sysconfig`: https://docs.python.org/3.7/library/sysconfig.html
The default implementation of the `sys.excepthook`_ function will then be modified to dispense an appropriate message when it detects a failure to import a module identified by one of the two new `sysconfig`_ functions as belonging to the Python standard library.
.. _`sys.excepthook`: https://docs.python.org/3.7/library/sys.html#sys.excepthook
Rationale
=========
The `sys.excepthook`_ function gets called when a raised exception is uncaught and the program is about to exit or—in an interactive session—the control is being returned to the prompt. This makes it a perfect place for customized error messages, as it will not influence caught errors and thus not slow down normal execution of Python scripts.
The inclusion of the functions `sysconfig.get_stdlib_modules()` and `sysconfig.get_optional_modules()` will also provide a long sought-after way of easily listing the names of Python standard library modules [#stackoverflow-stdlib]_, which will—among other benefits—make it easier for code analysis, profiling, and error reporting tools to offer runtime `--ignore-stdlib` flags.
Ideas leading up to this PEP were discussed on the `python-dev mailing list`_ and subsequently on `python-ideas`_.
.. _`python-dev mailing list`:
https://mail.python.org/pipermail/python-dev/2016-July/145534.html
.. _`python-ideas`:
https://mail.python.org/pipermail/python-ideas/2016-December/043907.html
Recommendation for Downstream Distributors
==========================================
By patching `site.py`_ [*]_ to provide their own implementation of the `sys.excepthook`_ function, Python distributors can display tailor-made error messages for any uncaught exceptions, including informing the user of a proper, distro-specific way to install missing standard library modules upon encountering a `ModuleNotFoundError`_. Some downstream distributors are already using this method of patching `sys.excepthook` to integrate with platform crash reporting mechanisms.
.. _`site.py`: https://docs.python.org/3.7/library/site.html
.. _`sitecustomize.py`: `site.py`_
An `example implementation`_ is attached to this PEP.
.. _`example implementation`: `Reference and Example Implementation`_
Backwards Compatibility
=======================
No problems with backwards compatibility are expected. Distributions that are
already patching Python modules to provide custom handling of missing
dependencies can continue to do so unhindered.
Reference and Example Implementation
====================================
TBD. The finer details will depend on what's practical given the capabilities
of the build system.
.. Reference implementation can be found on `GitHub`_ and is also accessible in the form of a `patch`_.
.. _`GitHub`: https://github.com/torsava/cpython/pull/1
.. _`patch`: https://github.com/torsava/cpython/pull/1.patch
Notes and References
====================
.. [*] Or `sitecustomize.py`_ for organizations with their own custom
Python variant.
.. [#debian-patch]
http://bazaar.launchpad.net/~doko/python/pkg3.5-debian/view/head:/patches/tkinter-import.diff
.. [#stackoverflow-stdlib]
http://stackoverflow.com/questions/6463918/how-can-i-get-a-list-of-all-the-python-standard-library-modules
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: