python-peps/peps/pep-0390.rst

PEP: 390
Title: Static metadata for Distutils
Version: $Revision$
Last-Modified: $Date$
Author: Tarek Ziadé <tarek@ziade.org>
BDFL-Delegate: Alyssa Coghlan
Discussions-To: distutils-sig@python.org
Status: Rejected
Type: Standards Track
Topic: Packaging
Content-Type: text/x-rst
Created: 10-Oct-2009
Python-Version: 2.7, 3.2
Post-History:
Resolution: https://mail.python.org/pipermail/distutils-sig/2013-April/020597.html

Abstract
========

This PEP describes a new section and a new format for the ``setup.cfg`` file,
that allows describing the Metadata of a package without using ``setup.py``.


Rejection Notice
================

As distutils2 is no longer going to be incorporated into the standard
library, this PEP was rejected by Alyssa Coghlan in late April, 2013.

A replacement PEP based on :pep:`426` (metadata 2.0) will be created that
defines the minimum amount of information needed to generate an sdist
archive given a source tarball or VCS checkout.


Rationale
=========

Today, if you want to list all the Metadata of a distribution (see :pep:`314`)
that is not installed, you need to use the ``setup.py`` command line interface.

So, basically, you download it, and run::

   $ python setup.py --name
   Distribute

   $ python setup.py --version
   0.6.4

Where ``name`` and ``version`` are metadata fields. This is working fine but
as soon as the developers add more code in ``setup.py``, this feature might
break or in worst cases might do unwanted things on the target system.

Moreover, when an OS packager wants to get the metadata of a distribution
he is re-packaging, he might encounter some problems to understand
the ``setup.py`` file he's working with.

So the rationale of this PEP is to provide a way to declare the metadata
in a static configuration file alongside ``setup.py`` that doesn't require
any third party code to run.


Adding a ``metadata`` section in ``setup.cfg``
==============================================

The first thing we want to introduce is a ``[metadata]`` section, in the
``setup.cfg`` file, that may contain any field from the Metadata::

   [metadata]
   name = Distribute
   version = 0.6.4

The ``setup.cfg`` file is used to avoid adding yet another configuration
file to work with in Distutils.

This file is already read by Distutils when a command is executed, and
if the ``metadata`` section is found, it will be used to fill the metadata
fields. If an option that corresponds to a Metadata field is given to
``setup()``, it will override the value that was possibly present in
``setup.cfg``.

Notice that ``setup.py`` is still used and can be required to define some
options that are not part of the Metadata fields. For instance, the
``sdist`` command can use options like ``packages`` or ``scripts``.


Multi-lines values
==================

Some Metadata fields can have multiple values. To keep ``setup.cfg`` compatible
with ``ConfigParser`` and the :rfc:`822` ``LONG HEADER FIELDS`` (see section 3.1.1),
these are expressed with ``,``-separated values::

    requires = pywin32, bar > 1.0, foo

When this variable is read, the values are parsed and transformed into a list:
``['pywin32', 'bar > 1.0', 'foo']``.


Context-dependant sections
==========================

The ``metadata`` section will also be able to use context-dependant sections.

A context-dependant section is a section with a condition about the execution
environment. Here's some examples::

   [metadata]
   name = Distribute
   version = 0.6.4

   [metadata:sys_platform == 'win32']
   requires = pywin32, bar > 1.0
   obsoletes = pywin31

   [metadata:os_machine == 'i386']
   requires = foo

   [metadata:python_version == '2.4' or python_version == '2.5']
   requires = bar

   [metadata:'linux' in sys_platform]
   requires = baz

Every ``[metadata:condition]`` section will be used only if the condition
is met when the file is read. The background motivation for these
context-dependant sections is to be able to define requirements that varies
depending on the platform the distribution might be installed on.
(see :pep:`314`).

The micro-language behind this is the simplest possible: it compares only
strings, with the ``==`` and ``in`` operators (and their opposites), and
with the ability to combine expressions. It makes it also easy to understand
to non-pythoneers.

The pseudo-grammar is ::

    EXPR [in|==|!=|not in] EXPR [or|and] ...

where ``EXPR`` belongs to any of those:

- python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
- os_name = os.name
- sys_platform = sys.platform
- platform_version = platform.version()
- platform_machine = platform.machine()
- a free string, like ``2.4``, or ``win32``

Notice that ``in`` is restricted to strings, meaning that it is not possible
to use other sequences like tuples or lists on the right side.

Distutils will provide a function that is able to generate the metadata
of a distribution, given a ``setup.cfg`` file, for the execution environment::

   >>> from distutils.util import local_metadata
   >>> local_metadata('setup.cfg')
   <DistributionMetadata instance>

This means that a vanilla Python will be able to read the metadata of a
package without running any third party code.

Notice that this feature is not restricted to the ``metadata`` namespace.
Consequently, any other section can be extended with such context-dependant
sections.

Impact on PKG-INFO generation and PEP 314
=========================================

When ``PKG-INFO`` is generated by Distutils, every field that relies on a
condition will have that condition written at the end of the line, after a
``;`` separator::

    Metadata-Version: 1.2
    Name: distribute
    Version: 0.6.4
    ...
    Requires: pywin32, bar > 1.0; sys_platform == 'win32'
    Requires: foo; os_machine == 'i386'
    Requires: bar; python_version == '2.4' or python_version == '2.5'
    Requires: baz; 'linux' in sys_platform
    Obsoletes = pywin31; sys_platform == 'win32'
    ...
    Classifier: Development Status :: 5 - Production/Stable
    Classifier: Intended Audience :: Developers
    Classifier: License :: OSI Approved :: Python Software Foundation License

Notice that this file can be opened with the ``DistributionMetadata`` class.
This class will be able to use the micro-language using the execution
environment.

Let's run in on a ``Python 2.5 i386 Linux``::

    >>> from distutils.dist import DistributionMetadata
    >>> metadata = DistributionMetadata('PKG_INFO')
    >>> metadata.get_requires()
    ['foo', 'bar', 'baz']

The execution environment can be overridden in case we want to get the metadata
for another environment::

    >>> env = {'python_version': '2.4',
    ...        'os_name': 'nt',
    ...        'sys_platform': 'win32',
    ...        'platform_version': 'MVCC++ 6.0'
    ...        'platform_machine': 'i386'}
    ...
    >>> metadata = DistributionMetadata('PKG_INFO', environment=env)
    >>> metadata.get_requires()
    ['bar > 1.0', 'foo', 'bar']

:pep:`314` is changed accordingly, meaning that each field will be able to
have that extra condition marker.

Compatibility
=============

This change is based on a new metadata ``1.2`` format meaning that
Distutils will be able to distinguish old PKG-INFO files from new ones.

The ``setup.cfg`` file change will stay ``ConfigParser``-compatible and
will not break existing ``setup.cfg`` files.

Limitations
===========

We are not providing ``<`` and ``>`` operators at this time, and
``python_version`` is a regular string. This implies using ``or`` operators
when a section needs to be restricted to a couple of Python versions.
Although, if :pep:`386` is accepted, ``python_version`` could be changed
internally into something comparable with strings, and
``<`` and ``>`` operators introduced.

Last, if a distribution is unable to set all metadata fields in ``setup.cfg``,
that's fine, the fields will be set to ``UNKNOWN`` when ``local_metadata`` is
called. Getting ``UNKNOWN`` values will mean that it might be necessary to
run the ``setup.py`` command line interface to get the whole set of metadata.

Acknowledgments
===============

The Distutils-SIG.


Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: