From 36212596ba24a233a0e2cfef51001dfe5315c921 Mon Sep 17 00:00:00 2001 From: Thomas Kluyver Date: Fri, 14 Jul 2017 09:25:30 +0200 Subject: [PATCH] PEP 517: Update hooks in line with discussion (#307) - renamed hooks - replaced input preparation hook with build_directory parameter - split out hook entries into separate sections --- pep-0517.txt | 216 +++++++++++++++++++++++++++++---------------------- 1 file changed, 123 insertions(+), 93 deletions(-) diff --git a/pep-0517.txt b/pep-0517.txt index 785aaf282..5fa1ff64e 100644 --- a/pep-0517.txt +++ b/pep-0517.txt @@ -164,76 +164,22 @@ with tools that do not use this spec. The build backend object is expected to have attributes which provide some or all of the following hooks. The common ``config_settings`` -argument is described after the individual hooks:: +argument is described after the individual hooks. - def get_build_wheel_requires(config_settings): - ... +Mandatory hooks +=============== -This hook MUST return an additional list of strings containing PEP 508 -dependency specifications, above and beyond those specified in the -``pyproject.toml`` file, to be installed when building a wheel. Example:: - - def get_build_wheel_requires(config_settings): - return ["wheel >= 0.25", "setuptools"] - -Optional. If not defined, the default implementation is equivalent to -``return []``. +build_wheel +----------- :: - def prepare_wheel_metadata(metadata_directory, config_settings): - ... + build_wheel(wheel_directory, config_settings=None, build_directory=None, metadata_directory=None): + ... -Must create a ``.dist-info`` directory containing wheel metadata -inside the specified ``metadata_directory`` (i.e., creates a directory -like ``{metadata_directory}/{package}-{version}.dist-info/``. This -directory MUST be a valid ``.dist-info`` directory as defined in the -wheel specification, except that it need not contain ``RECORD`` or -signatures. The hook MAY also create other files inside this -directory, and a build frontend MUST ignore such files; the intention -here is that in cases where the metadata depends on build-time -decisions, the build backend may need to record these decisions in -some convenient format for re-use by the actual wheel-building step. - -This must return the basename (not the full path) of the ``.dist-info`` -directory it creates, as a unicode string. - -Optional. If a build frontend needs this information and the method is -not defined, it should call ``build_wheel`` and look at the resulting -metadata directly. - -:: - - def prepare_wheel_build_files(build_directory, config_settings): - ... - -Must copy or create any files needed to build a wheel of this package into -``build_directory``. For instance, ``pyproject.toml`` should -be copied unmodified into the root of this directory. For tools such -as `setuptools_scm `_, this may include -extracting some information from a version control system. -The ``build_wheel`` hook will subsequently be run from the ``build_directory`` -populated by this hook. The contents of the resulting wheel should be the same -whether ``build_wheel`` is invoked in an original source directory, the build -directory populated by this hook, or an unpacked sdist directory. - -Because the wheel will be built from a temporary build directory, ``build_wheel`` -may create intermediate files in the working directory, and does not need to -take care to clean them up. - -The return value will be ignored. - -Optional. If this hook is not defined, frontends may call ``build_sdist`` -and unpack the archive to use as a build directory. Backends in which -building an sdist has additional requirements should define -``prepare_wheel_build_files``. - -:: - - def build_wheel(wheel_directory, config_settings, metadata_directory=None): - ... - -Must build a .whl file, and place it in the specified ``wheel_directory``. +Must build a .whl file, and place it in the specified ``wheel_directory``. It +must return the basename (not the full path) of the ``.whl`` file it creates, +as a unicode string. If the build frontend has previously called ``prepare_wheel_metadata`` and depends on the wheel resulting from this call to have metadata @@ -244,31 +190,37 @@ metadata. The directory passed in by the build frontend MUST be identical to the directory created by ``prepare_wheel_metadata``, including any unrecognized files it created. -This must return the basename (not the full path) of the ``.whl`` file it -creates, as a unicode string. +If build_directory is not None, it is a unicode string containing the +path to a directory where intermediate build artifacts may be stored. +This may be empty, or it may contain artifacts from a previous build to +be used as a cache. The backend is responsible for determining whether +any cached artifacts are outdated. When a build_directory is provided, +the backend should not create or modify any files in the source +directory (the working directory where the hook is called). If the +backend cannot reliably avoid modifying the directory it builds from, it +should copy any files it needs to build_directory and perform the build +there. -Mandatory. +If build_directory is None, the backend may do an 'in place' build which +modifies the source directory. The semantics of this are not specified +here. + +Whatever the value of build_directory, the backend may also store intermediates +in other cache locations or temporary directories, which it is responsible for +managing. The presence or absence of any caches should not make a +material difference to the final result of the build. + +build_sdist +----------- :: - def get_build_sdist_requires(config_settings): - ... - -This hook MUST return an additional list of strings containing PEP 508 -dependency specifications, above and beyond those specified in the -``pyproject.toml`` file. These dependencies will be installed for building an -sdist. - -Optional. If not defined, the default implementation is equivalent to -``return []``. - -:: - - def build_sdist(sdist_directory, config_settings): - ... + def build_sdist(sdist_directory, config_settings=None): + ... Must build a .tar.gz source distribution and place it in the specified -``sdist_directory``. +``sdist_directory``. It must return the basename (not the full path) of the +``.tar.gz`` file it creates, as a unicode string. A .tar.gz source distribution (sdist) contains a single top-level directory called ``{name}-{version}`` (e.g. ``foo-1.0``), containing the source files of the @@ -285,11 +237,75 @@ specifies UTF-8 based file names. This is not yet the default for the tarfile module shipped with Python 3.6, so backends using the tarfile module need to explicitly pass ``format=tarfile.PAX_FORMAT``. -This must return the basename (not the full path) of the ``.tar.gz`` file it -creates, as a unicode string. +Frontends performing a local installation may want to produce an sdist as an +intermediate, but are advised to be prepared to use a fallback if it fails, as +e.g. some backends may require version control tools to build an sdist. + +Optional hooks +============== + +get_requires_for_build_wheel +---------------------------- + +:: + + def get_requires_for_build_wheel(config_settings=None): + ... + +This hook MUST return an additional list of strings containing PEP 508 +dependency specifications, above and beyond those specified in the +``pyproject.toml`` file, to be installed when calling the ``build_wheel`` or +``prepare_metadata_for_build_wheel`` hooks. + +Example:: + + def get_requires_for_build_wheel(config_settings): + return ["wheel >= 0.25", "setuptools"] + +If not defined, the default implementation is equivalent to ``return []``. + +prepare_metadata_for_build_wheel +-------------------------------- + +:: + + def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None): + ... + +Must create a ``.dist-info`` directory containing wheel metadata +inside the specified ``metadata_directory`` (i.e., creates a directory +like ``{metadata_directory}/{package}-{version}.dist-info/``). This +directory MUST be a valid ``.dist-info`` directory as defined in the +wheel specification, except that it need not contain ``RECORD`` or +signatures. The hook MAY also create other files inside this +directory, and a build frontend MUST preserve, but otherwise ignore, such files; +the intention +here is that in cases where the metadata depends on build-time +decisions, the build backend may need to record these decisions in +some convenient format for re-use by the actual wheel-building step. + +This must return the basename (not the full path) of the ``.dist-info`` +directory it creates, as a unicode string. + +If a build frontend needs this information and the method is +not defined, it should call ``build_wheel`` and look at the resulting +metadata directly. + +get_requires_for_build_sdist +---------------------------- + +:: + + def get_requires_for_build_sdist(config_settings=None): + ... + +This hook MUST return an additional list of strings containing PEP 508 +dependency specifications, above and beyond those specified in the +``pyproject.toml`` file. These dependencies will be installed when calling the +``build_sdist`` hook. + +If not defined, the default implementation is equivalent to ``return []``. -Mandatory, but it may not succeed in all situations: for instance, some tools -can only build an sdist from a VCS checkout. .. note:: Editable installs @@ -372,18 +388,18 @@ following criteria: - All requirements specified by the project's build-requirements must be available for import from Python. In particular: - - The ``get_build_wheel_requires`` and ``get_build_sdist_requires`` hooks are + - The ``get_requires_for_build_wheel`` and ``get_requires_for_build_sdist`` hooks are executed in an environment which contains the bootstrap requirements specified in the ``pyproject.toml`` file. - - The ``prepare_wheel_metadata``, ``prepare_wheel_build_files`` and - ``build_wheel`` hooks are executed in an environment which contains the + - The ``prepare_metadata_for_build_wheel`` and ``build_wheel`` hooks are + executed in an environment which contains the bootstrap requirements from ``pyproject.toml`` and those specified by the - ``get_build_wheel_requires`` hook. + ``get_requires_for_build_wheel`` hook. - The ``build_sdist`` hook is executed in an environment which contains the bootstrap requirements from ``pyproject.toml`` and those specified by the - ``get_build_sdist_requires`` hook. + ``get_requires_for_build_sdist`` hook. - This must remain true even for new Python subprocesses spawned by the build environment, e.g. code like:: @@ -469,7 +485,8 @@ hood and apply duct tape when necessary. Source distributions ====================== -For now, we continue with the legacy sdist format which is mostly +We continue with the legacy sdist format, adding some new restrictions. +This format is mostly undefined, but basically comes down to: a file named ``{NAME}-{VERSION}.{EXT}``, which unpacks into a buildable source tree called ``{NAME}-{VERSION}/``. Traditionally these have always @@ -480,6 +497,15 @@ Integration frontends require that an sdist named ``{NAME}-{VERSION}.{EXT}`` will generate a wheel named ``{NAME}-{VERSION}-{COMPAT-INFO}.whl``. +The new restrictions for sdists built by PEP 517 backends are: + +- They will be gzipped tar archives, with the ``.tar.gz`` extension. Zip + archives, or other compression formats for tarballs, are not allowed at + present. +- Tar archives must be created in the modern POSIX.1-2001 pax tar format, which + uses UTF-8 for file names. +- The source tree contained in an sdist is expected to include the + ``pyproject.toml`` file. =================================== Comparison to competing proposals @@ -808,6 +834,10 @@ automatically upgrade packages to the new format: format is already standardised). Close control of archive creation is important for reproducible builds. And it's not clear that tasks requiring an unpacked distribution will be more common than those requiring an archive. +* We considered an extra hook to copy files to a build directory before invoking + ``build_wheel``. Looking at existing build systems, we found that passing + a build directory into ``build_wheel`` makes more sense for many tools than + pre-emptively copying files into a build directory. =========== Copyright