python-peps/peps/pep-0229.rst

PEP: 229
Title: Using Distutils to Build Python
Version: $Revision$
Last-Modified: $Date$
Author: A.M. Kuchling <amk@amk.ca>
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 16-Nov-2000
Python-Version: 2.1
Post-History:


Introduction
============

The ``Modules/Setup`` mechanism has some flaws:

* People have to remember to uncomment bits of ``Modules/Setup`` in
  order to get all the possible modules.

* Moving ``Setup`` to a new version of Python is tedious; new modules
  have been added, so you can't just copy the older version, but
  have to reconcile the two versions.

* Users have to figure out where the needed libraries, such as
  ``zlib``, are installed.


Proposal
========

Use the Distutils to build the modules that come with Python.

The changes can be broken up into several pieces:

1. The Distutils needs some Python modules to be able to build
   modules.  Currently I believe the minimal list is ``posix``, ``_sre``,
   and ``string``.

   These modules will have to be built before the Distutils can be
   used, so they'll simply be hardwired into ``Modules/Makefile`` and
   be automatically built.

2. A top-level setup.py script will be written that checks the
   libraries installed on the system and compiles as many modules
   as possible.

3. ``Modules/Setup`` will be kept and settings in it will override
   ``setup.py``'s usual behavior, so you can disable a module known
   to be buggy, or specify particular compilation or linker flags.
   However, in the common case where ``setup.py`` works correctly,
   everything in ``Setup`` will remain commented out.  The other
   ``Setup.*`` become unnecessary, since nothing will be generating
   ``Setup`` automatically.

The patch was checked in for Python 2.1, and has been subsequently
modified.


Implementation
==============

Patch #102588 on SourceForge contains the proposed patch.
Currently the patch tries to be conservative and to change as few
files as possible, in order to simplify backing out the patch.
For example, no attempt is made to rip out the existing build
mechanisms.  Such simplifications can wait for later in the beta
cycle, when we're certain the patch will be left in, or they can
wait for Python 2.2.

The patch makes the following changes:

* Makes some required changes to distutils/sysconfig (these will
  be checked in separately)

* In the top-level ``Makefile.in``, the "sharedmods" target simply
  runs ``"./python setup.py build"``, and "sharedinstall" runs
  ``"./python setup.py install"``.  The "clobber" target also deletes
  the ``build/`` subdirectory where Distutils puts its output.

* ``Modules/Setup.config.in`` only contains entries for the ``gc`` and ``thread``
  modules; the ``readline``, ``curses``, and ``db`` modules are removed because
  it's now ``setup.py``'s job to handle them.

* ``Modules/Setup.dist`` now contains entries for only 3 modules --
  ``_sre``, ``posix``, and ``strop``.

* The ``configure`` script builds ``setup.cfg`` from ``setup.cfg.in``.  This
  is needed for two reasons: to make building in subdirectories
  work, and to get the configured installation prefix.

* Adds ``setup.py`` to the top directory of the source tree.  ``setup.py``
  is the largest piece of the puzzle, though not the most
  complicated.  ``setup.py`` contains a subclass of the ``BuildExt``
  class, and extends it with a ``detect_modules()`` method that does
  the work of figuring out when modules can be compiled, and adding
  them to the 'exts' list.


Unresolved Issues
=================

Do we need to make it possible to disable the 3 hard-wired modules
without manually hacking the Makefiles?  [Answer: No.]

The Distutils always compile modules as shared libraries.  How do
we support compiling them statically into the resulting Python
binary?

[Answer: building a Python binary with the Distutils should be
feasible, though no one has implemented it yet.  This should be
done someday, but isn't a pressing priority as messing around with
the top-level ``Makefile.pre.in`` is good enough.]


Copyright
=========

This document has been placed in the public domain.



..
  Local Variables:
  mode: indented-text
  indent-tabs-mode: nil
  End: