273 lines
10 KiB
ReStructuredText
273 lines
10 KiB
ReStructuredText
PEP: 632
|
||
Title: Deprecate distutils module
|
||
Author: Steve Dower <steve.dower@python.org>
|
||
Discussions-To: https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134
|
||
Status: Final
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 03-Sep-2020
|
||
Python-Version: 3.10
|
||
Post-History: 03-Sep-2020, 22-Jan-2021
|
||
Resolution: https://mail.python.org/archives/list/python-dev@python.org/thread/TXU6TVOMBLQU3SV57DMMOA5Y2E67AW7P/
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
The distutils module [1]_ has for a long time recommended using the
|
||
setuptools package [2]_ instead. Setuptools has recently integrated a
|
||
complete copy of distutils and is no longer dependent on the standard
|
||
library [3]_. Pip has been silently replacing distutils with
|
||
setuptools when installing packages for a long time already, and the
|
||
distutils documentation has stated that it is being phased out since
|
||
2014 (or earlier). It is time to remove it from the standard library.
|
||
|
||
|
||
Motivation
|
||
==========
|
||
|
||
distutils [1]_ is a largely undocumented and unmaintained collection
|
||
of utilities for packaging and distributing Python packages, including
|
||
compilation of native extension modules. It defines a configuration
|
||
format that describes a Python distribution and provides the tools to
|
||
convert a directory of source code into a source distribution, and
|
||
some forms of binary distribution. Because of its place in the
|
||
standard library, many updates can only be released with a major
|
||
release, and users cannot rely on particular fixes being available.
|
||
|
||
setuptools [2]_ is a better documented and well maintained enhancement
|
||
based on distutils. While it provides very similar functionality, it
|
||
is much better able to support users on earlier Python releases, and
|
||
can respond to bug reports more quickly. A number of platform-specific
|
||
enhancements already exist in setuptools that have not been added to
|
||
distutils, and there is been a long-standing recommendation in the
|
||
distutils documentation to prefer setuptools.
|
||
|
||
Historically, setuptools has extended distutils using subclassing and
|
||
monkeypatching, but has now taken a copy of the underlying code. [3]_
|
||
As a result, the second last major dependency on distutils is gone and
|
||
there is no need to keep it in the standard library.
|
||
|
||
The final dependency on distutils is CPython itself, which uses it to
|
||
build native extension modules in the standard library (except on
|
||
Windows). Because this is a CPython build-time dependency, it is
|
||
possible to continue to use distutils for this specific case without
|
||
it being part of the standard library.
|
||
|
||
Deprecation and removal will make it obvious that issues should be
|
||
fixed in the setuptools project, and will reduce a source of bug
|
||
reports and unnecessary test maintenance. It will also help promote
|
||
the development of alternative build backends, which can now be
|
||
supported more easily thanks to :pep:`517`.
|
||
|
||
|
||
Specification
|
||
=============
|
||
|
||
In Python 3.10 and 3.11, distutils will be formally marked as
|
||
deprecated. All known issues will be closed at this time.
|
||
``import distutils`` will raise a deprecation warning. New issues that
|
||
would be considered release blocking may still be fixed, but support
|
||
for new tools or platforms will not be added.
|
||
|
||
During Python 3.10 and 3.11, uses of distutils within the standard
|
||
library may change to use alternative APIs.
|
||
|
||
In Python 3.12, distutils will no longer be installed by ``make
|
||
install`` or any of the first-party distribution. Third-party
|
||
redistributors should no longer include distutils in their bundles or
|
||
repositories.
|
||
|
||
This PEP makes no specification on migrating the parts of the CPython
|
||
build process that currently use distutils. Depending on
|
||
contributions, this migration may occur at any time.
|
||
|
||
After Python 3.12 is started and when the CPython build process no
|
||
longer depends on distutils being in the standard library, the entire
|
||
``Lib/distutils`` directory and ``Lib/test/test_distutils.py`` file
|
||
will be removed from the repository.
|
||
|
||
Other references to distutils will be cleaned up. As of Python 3.9's
|
||
initial release, the following modules have references in code or
|
||
comments:
|
||
|
||
* Lib/ctypes/util.py
|
||
* Lib/site.py
|
||
* Lib/sysconfig.py
|
||
* Lib/_aix_support.py
|
||
* Lib/_bootsubprocess.py
|
||
* Lib/_osx_support.py
|
||
* Modules/_decimal/tests/formathelper.py
|
||
|
||
The following Tools in CPython also refer to distutils. Note that none
|
||
of these are installed with CPython:
|
||
|
||
* PC/layout (references will be removed)
|
||
* Tools/msi (references will be removed)
|
||
* Tools/peg_generator (will be adapted to a different build tool)
|
||
* Tools/test2to3 (example project will be removed)
|
||
|
||
As the distutils code is already included in setuptools, there is no
|
||
need to republish it in any other form. Those who require access to
|
||
the functionality should use setuptools or an alternative build
|
||
backend.
|
||
|
||
Backwards Compatibility
|
||
=======================
|
||
|
||
Code that imports distutils will no longer work from Python 3.12.
|
||
|
||
The suggested migration path is to use the equivalent (though not
|
||
identical) imports from setuptools (see [5]_), or to migrate to an
|
||
alternative build backend (see :pep:`517`).
|
||
|
||
Code already exists in setuptools to transparently switch ``setup.py``
|
||
files using distutils onto their equivalents, and so most working
|
||
build scripts are already known to work with setuptools. Such scripts
|
||
may need to update their import statements. Consult the setuptools
|
||
documentation for specific migration advice. [5]_
|
||
|
||
Some projects use alternate sets of patches over distutils, notably,
|
||
numpy.distutils. [6]_ Projects that we know are doing this have been
|
||
informed.
|
||
|
||
Many build scripts use custom commands or narrowly scoped patches. As
|
||
these packages are already subject to setuptools overriding distutils,
|
||
we expect minimal disruption as a result of distutils being removed.
|
||
Scripts may still need to be updated to avoid importing distutils.
|
||
|
||
|
||
Reference Implementation
|
||
========================
|
||
|
||
setuptools version 48 includes the complete copy of distutils, and as
|
||
such is no longer dependent on the standard library's copy. Most
|
||
implementation issues they have faced are due to the continuing
|
||
existence of distutils in the standard library, and so removal will
|
||
improve the stability of their implementation.
|
||
|
||
There is not yet a reference implementation for the removal of
|
||
distutils from the standard library, nor is there an implementation
|
||
for CPython's native module builds without relying on the standard
|
||
library copy of distutils.
|
||
|
||
|
||
Migration Advice
|
||
================
|
||
|
||
.. note::
|
||
This section suggests some alternative replacements for popular
|
||
functionality that is being formally deprecated with this PEP. It
|
||
is current at time of writing, but is not kept up to date.
|
||
|
||
For these modules or types, ``setuptools`` is the best substitute:
|
||
|
||
* ``distutils.ccompiler``
|
||
* ``distutils.cmd.Command``
|
||
* ``distutils.command``
|
||
* ``distutils.config``
|
||
* ``distutils.core.Distribution``
|
||
* ``distutils.errors``
|
||
|
||
For these modules or types, use the standards-defined Python Packaging
|
||
Authority packages specified:
|
||
|
||
* ``distutils.version`` — use the ``packaging`` package
|
||
|
||
For these modules or functions, use the standard library module shown:
|
||
|
||
* ``distutils.fancy_getopt`` — use the ``argparse`` module
|
||
* ``distutils.spawn.find_executable`` — use the ``shutil.which`` function
|
||
* ``distutils.spawn.spawn`` — use the ``subprocess.run`` function
|
||
* ``distutils.sysconfig`` — use the ``sysconfig`` module
|
||
* ``distutils.util.get_platform`` — use the ``platform`` module
|
||
|
||
For these functions, and any others not mentioned here, you will need
|
||
to reimplement the functionality yourself. The legacy documentation
|
||
can be found at https://docs.python.org/3.9/distutils/apiref.html
|
||
|
||
* ``distutils.dir_util.create_tree``
|
||
* ``distutils.util.change_root``
|
||
* ``distutils.util.strtobool``
|
||
|
||
|
||
Rejected Ideas
|
||
==============
|
||
|
||
Deprecate but do not delete
|
||
---------------------------
|
||
|
||
The primary concern with this approach is that distutils most
|
||
frequently breaks because of platform differences, which means that
|
||
without maintenance, it will stop working out-of-sync with any
|
||
Python release. This makes it impossible for libraries to reliably
|
||
detect when they will stop working.
|
||
|
||
In contrast, this PEP proposes a concrete date, known well in advance,
|
||
when distutils will stop working, and commits to not breaking the API
|
||
before that time. This gives maintainers a predictable schedule,
|
||
ensures any breakage occurs at a point where users will already be
|
||
expecting changed behavior, and provides a reliable detection
|
||
mechanism (specifically, that ``import distutils`` raises).
|
||
|
||
Finally, as long as distutils remains in the standard library in any
|
||
form, it will interfere with third-party packages that provide shims
|
||
or replacements, including setuptools. Completely removing the
|
||
package at a known version makes it possible for third-parties to
|
||
safely use a substitute.
|
||
|
||
|
||
Only deprecate the setuptools-like functionality
|
||
------------------------------------------------
|
||
|
||
This suggestion assumes that there exists a volunteer to maintain
|
||
whatever is left, which is not true. It also implies that anybody
|
||
knows which functionality should remain, which as seen in the
|
||
discussions is not at all clear.
|
||
|
||
Most helper functions in distutils already have supported (and
|
||
improved) alternatives, often in the standard library, and there is
|
||
little that can be done to the legacy versions without breaking
|
||
compatibility. (And any break requiring maintainers to update their
|
||
code is essentially equivalent to requiring them to import a different
|
||
function.)
|
||
|
||
The last point from the previous section also applies here.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] distutils - Building and installing Python modules
|
||
(https://docs.python.org/3.9/library/distutils.html)
|
||
|
||
.. [2] setuptools - PyPI
|
||
(https://pypi.org/project/setuptools/)
|
||
|
||
.. [3] setuptools Issue #417 - Adopt distutils
|
||
(https://github.com/pypa/setuptools/issues/417)
|
||
|
||
.. [5] Porting from Distutils
|
||
(https://setuptools.readthedocs.io/en/latest/deprecated/distutils-legacy.html)
|
||
|
||
.. [6] Packaging (numpy.distutils)
|
||
(https://numpy.org/doc/stable/reference/distutils.html)
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document is placed in the public domain or under the
|
||
CC0-1.0-Universal license, whichever is more permissive.
|
||
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|