From 03e4ec6740dfa7459a347857fe6436a40f2e114d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tarek=20Ziad=C3=A9?= Date: Sun, 22 Feb 2009 02:33:52 +0000 Subject: [PATCH] started the work on PEP 376 for Distutils --- pep-0376.txt | 248 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 248 insertions(+) create mode 100644 pep-0376.txt diff --git a/pep-0376.txt b/pep-0376.txt new file mode 100644 index 000000000..33b1d27ac --- /dev/null +++ b/pep-0376.txt @@ -0,0 +1,248 @@ +PEP: 376 +Title: Changing the .egg-info structure +Version: $Revision:$ +Last-Modified: $Date:$ +Author: Tarek Ziadé +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 22-Feb-2009 +Python-Version: 2.7, 3.1 +Post-History: + +.. contents:: + +Abstract +======== + +This PEP proposes various enhancements for Distutils: + +- a new format for the .egg-info structure. +- an install script to install package in Python. +- an uninstall script to uninstall a package in Python. + +Rationale +========= + +There are three problems right now in the way packages +are installed in Python: + +- There are too many different ways to install a package in + Python nowadays + +- There is no way to uninstall a package. + +- There is no API to get the metadata of installed + packages. + +How packages are installed +-------------------------- + +Right now, when a package is installed in Python, using the +Distutils `install` command, the `install_egg_info` subcommand is +called in order to create an `.egg-info` file in the site-packages directory, +right beside the package itself. + +For example, if the `zlib` package is installed, two elements +will be installed in `site-packages`:: + + - zlib + - zlib-2.5.2-py2.6.egg-info + +Where `zlib` is the package itself, and `zlib-2.5.2-py2.6.egg-info` is +a file containing the package metadata as described in PEP 314. + +This file corresponds to the file called `PKG-INFO`, built by +the `sdist` command. + +The problem is that many people use `easy_install` (setuptools) or `pip` +to install their packages. And these third-party tool does not install +the packages the same way Distutils does. + +- `easy_install` creates an `EGG-INFO` directory inside an `.egg` directory, + and adds a `PKG-INFO` file inside this directory, amongst other files. + +- `pip` creates an `.egg-info` directory inside the site-packages directory + besides the package, and adds an `PKG-INFO` file inside it. + +They both add other files in the `EGG-INFO` or `.egg-info` directory, and +create or modify `.pth` files. `pip` also creates one `.pth` file +per installed package, which may lead to slow initialisation of Python. + +The uninstall command +--------------------- + +Python doesn't provide any `uninstall` command. If you want to uninstall +a package, you have to be a power user and remove the package directory +from the right site-packages directory, then look over the right pth +files. And this method differs, depending on the tools you are using. + +The worst is that you depend on the way the packager created his package. +When you call `python setup.py install`, it will not be installed the same way +depending on the tool used by the packager (distutils or setuptools). + +But there's a common behavior : files are copied in your installation. +And there's a way to keep track of theses file, so to remove them. + +Installing a package +-------------------- + +There are too many different ways to install a package in Python: + +- by hand, by getting a distribution and running the install command +- using `easy_install`, the script provided by setuptools +- using `pip` + +The problem is: they do no install the package the same way, +and Python should provide one and only one way to do it. + +What this PEP proposes +---------------------- + +To address those issues, this PEP proposes a few changes: + +- a new `.egg-info` structure using a directory; +- a list of elements this directory holds; +- some new functions in `pkgutil` +- adding an install and an uninstall script + +.egg-info becomes a directory +============================= + +The first change would be to make `.egg-info` become a directory and +hold the `PKG-INFO` file built by the `write_pkg_file` method. + +This change will not impact Python itself, because this file is not +used anywhere yet in the standard library. So there's no need of +deprecation. + +Although, it will impact the `setuptools` and `pip` project, but given +the fact that they already works with a directory that contains a +`PKG-INFO` file, the change will be small. + +For example, if the `zlib` package is installed, two elements +will be installed in `site-packages`:: + + - zlib + - zlib-2.5.2-py2.6.egg-info/ + PKG-INFO + +To be able to implement this change, the impacted code in Distutils +is the `install_egg_info` command. + +Adding MANIFEST and RECORD in the .egg-info directory +===================================================== + +Some files can be added inside the `.egg-info` directory at installation +time. They will all be UPPERCASE files. + +- the `MANIFEST` file built by the `sdist` command. Notice that + some fixes were made lately on the default files added in `MANIFEST` + when `MANIFEST.in` is not provided (see #2279 for instance). + +- the `RECORD` file will hold the list of installed files. These + correspond to the files listed by the `record` option of the `install` + command, and will always be generated. This will allow uninstallation, like + explained later in this PEP. + +The `install` command will record by default installed files in the +RECORD file. + +The `sdist` module will introduce an `EGG_INFO_FILES` constant to list +all files located in the `.egg-info` directory:: + + from collections import namedtuple + + EggInfos = namedtuple('EggInfo', 'manifest record pkg_info') + + # files added in egg-info + EGG_INFO_FILES = EggInfos('MANIFEST', 'RECORD', 'PKG-INFO') + +Back to our `zlib` example, we will have:: + + - zlib + - zlib-2.5.2-py2.6.egg-info/ + PKG-INFO + MANIFEST + RECORD + +New functions in pkgutil +======================== + +To use the `.egg-info` directory content, we need to add in the standard library a set of +APIs. The best place to put these APIs seems to be `pkgutil`. + +The new functions added in the package are : + +- get_egg_info(pkg_name) -> directory or None + + Scans all site-packages directories and look for all `pkg_name*.egg-info` + directory. If founded, returns the path. If the directory is not found, + returns None. + +- get_metadata(pkg_name) -> DistributionMetadata or None + + Uses `get_egg_info` to get the `PKG-INFO` file, and returns a + `DistributionMetadata` instance that contains the metadata. + This will require a small change in `DistributionMetadata` (see #4908). + +- get_egg_info_file(pkg_name, filename) -> file object or None + + Uses `get_egg_info` and gets any file inside the directory, + pointed by filename. + + filename is any value founded in `distutils.sdist.EGG_INFO_FILES` + +Let's use it over our `zlib` example:: + + >>> from pkgutil import get_egg_info, get_metadata, get_egg_info_file + >>> get_egg_info('zlib') + '/opt/local/lib/python2.6/site-packages/zlib-2.5.2-py2.6.egg-info' + >>> metadata = get_metadata('zlib') + >>> metadata.version + '2.5.2' + >>> from distutils.dist import EGG_INFO_FILES + >>> get_metadata('zlib', EGG_INFO_FILES.manifest).read() + some + ... + files + +Adding an install and an uninstall script +========================================= + +`easy_install` and `pip` does basically the same work, besides other features +that are not discussed here. + +- they look for the package at PyPI +- they download it and build it +- they install it inside the site-packages directory +- they add an entry in a .pth file + +A new script called `install.py` is added in a new directory called `scripts` +in Distutils, and let people run an installation using:: + + $ python -m 'distutils.scripts.install' zlib + +An uninstall command is added as well, that removes the files recorded +in the RECORD file. This removal will warn on file that no longer exists +and will not take care of side effects, like the removal of a file +used by another element of the system. + +XXX work to be done here : specification of the two commands +(probably a mix of easy_install and pip) + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: +