Minor changes made by Carl Meyer.
This commit is contained in:
Vinay Sajip
parent
0ba175b321
commit
75c5dc7c4b
269
pep-0405.txt
269
pep-0405.txt
|
|
@ -42,9 +42,9 @@ which does copy the Python binary, is forced to duplicate much of
|
|||
Python's ``site`` module and manually symlink/copy an ever-changing
|
||||
set of standard-library modules into the virtual environment in order
|
||||
to perform a delicate boot-strapping dance at every startup.
|
||||
(Virtualenv copies the binary because symlinking it does not provide
|
||||
isolation, as Python dereferences a symlinked executable before
|
||||
searching for ``sys.prefix``.)
|
||||
(Virtualenv must copy the binary in order to provide isolation, as
|
||||
Python dereferences a symlinked executable before searching for
|
||||
``sys.prefix``.)
|
||||
|
||||
The ``PYTHONHOME`` environment variable, Python's only existing
|
||||
built-in solution for virtual environments, requires
|
||||
|
|
@ -85,22 +85,22 @@ Python binary belongs to a virtual environment, and the value of the
|
|||
create this virtual environment.
|
||||
|
||||
In this case, prefix-finding continues as normal using the value of
|
||||
the ``home`` key as the effective Python binary location, which
|
||||
results in ``sys.prefix`` being set to the system installation prefix,
|
||||
while ``sys.site_prefix`` is set to the directory containing
|
||||
the ``home`` key as the effective Python binary location, which finds
|
||||
the prefix of the base installation. ``sys.base_prefix`` is set to
|
||||
this value, while ``sys.prefix`` is set to the directory containing
|
||||
``pyvenv.cfg``.
|
||||
|
||||
(If ``pyvenv.cfg`` is not found or does not contain the ``home`` key,
|
||||
prefix-finding continues normally, and ``sys.site_prefix`` will be
|
||||
equal to ``sys.prefix``.)
|
||||
prefix-finding continues normally, and ``sys.prefix`` will be equal to
|
||||
``sys.base_prefix``.)
|
||||
|
||||
The ``site`` and ``sysconfig`` standard-library modules are modified
|
||||
such that site-package directories ("purelib" and "platlib", in
|
||||
``sysconfig`` terms) are found relative to ``sys.site_prefix``, while
|
||||
other directories (the standard library, include files) are still
|
||||
found relative to ``sys.prefix``.
|
||||
such that the standard library and header files are are found relative
|
||||
to ``sys.base_prefix``, while site-package directories ("purelib" and
|
||||
"platlib", in ``sysconfig`` terms) are still found relative to
|
||||
``sys.prefix``.
|
||||
|
||||
(Also, ``sys.site_exec_prefix`` is added, and handled similarly with
|
||||
(Also, ``sys.base_exec_prefix`` is added, and handled similarly with
|
||||
regard to ``sys.exec_prefix``.)
|
||||
|
||||
Thus, a Python virtual environment in its simplest form would consist
|
||||
|
|
@ -166,7 +166,13 @@ Multiple paths can be given to ``pyvenv``, in which case an identical
|
|||
virtualenv will be created, according to the given options, at each
|
||||
provided path.
|
||||
|
||||
The ``venv`` module also adds a ``pysetup3`` script into each venv.
|
||||
The ``venv`` module also provides "shell activation scripts" for POSIX
|
||||
and Windows systems which simply add the virtual environment's ``bin``
|
||||
(or ``Scripts``) directory to the front of the user's shell PATH.
|
||||
This is not strictly necessary for use of a virtual environment (as an
|
||||
explicit path to the venv's python binary or scripts can just as well
|
||||
be used), but it is convenient.
|
||||
|
||||
In order to allow ``pysetup`` and other Python package managers to
|
||||
install packages into the virtual environment the same way they would
|
||||
install into a normal Python installation, and avoid special-casing
|
||||
|
|
@ -179,6 +185,7 @@ So a typical virtual environment layout on a POSIX system would be::
|
|||
bin/python3
|
||||
bin/python
|
||||
bin/pysetup3
|
||||
include/
|
||||
lib/python3.3/site-packages/
|
||||
|
||||
While on a Windows system::
|
||||
|
|
@ -188,12 +195,13 @@ While on a Windows system::
|
|||
Scripts/python3.dll
|
||||
Scripts/pysetup3.exe
|
||||
Scripts/pysetup3-script.py
|
||||
... other DLLs and pyds...
|
||||
... other DLLs and pyds...
|
||||
Include/
|
||||
Lib/site-packages/
|
||||
|
||||
Third-party packages installed into the virtual environment will have
|
||||
their Python modules placed in the ``site-packages`` directory, and
|
||||
their executables placed in ``bin/`` or ``Scripts\``.
|
||||
their executables placed in ``bin/`` or ``Scripts``.
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
@ -266,22 +274,18 @@ following keyword arguments on instantiation:
|
|||
target directory instead of raising an exception. Defaults to
|
||||
``False``.
|
||||
|
||||
* ``use_symlinks`` - A Boolean value indicating whether to attempt to
|
||||
* ``symlinks`` - A Boolean value indicating whether to attempt to
|
||||
symlink the Python binary (and any necessary DLLs or other binaries,
|
||||
e.g. ``pythonw.exe``), rather than copying. Defaults to ``False``.
|
||||
|
||||
The returned env-builder is an object with a ``create`` method, which
|
||||
takes as required argument the path (absolute or relative to the
|
||||
current directory) of the target directory which is to contain the
|
||||
virtual environment. The ``create`` method either creates the
|
||||
environment in the specified directory, or raises an appropriate
|
||||
exception.
|
||||
The instantiated env-builder has a ``create`` method, which takes as
|
||||
required argument the path (absolute or relative to the current
|
||||
directory) of the target directory which is to contain the virtual
|
||||
environment. The ``create`` method either creates the environment in
|
||||
the specified directory, or raises an appropriate exception.
|
||||
|
||||
Creators of third-party virtual environment tools are free to use the
|
||||
provided ``EnvBuilder`` class as a base class.
|
||||
|
||||
The ``venv`` module also provides a module-level function as a
|
||||
convenience::
|
||||
The ``venv`` module also provides a module-level ``create`` function
|
||||
as a convenience::
|
||||
|
||||
def create(env_dir,
|
||||
system_site_packages=False, clear=False, use_symlinks=False):
|
||||
|
|
@ -291,6 +295,9 @@ convenience::
|
|||
use_symlinks=use_symlinks)
|
||||
builder.create(env_dir)
|
||||
|
||||
Creators of third-party virtual environment tools are free to use the
|
||||
provided ``EnvBuilder`` class as a base class.
|
||||
|
||||
The ``create`` method of the ``EnvBuilder`` class illustrates the
|
||||
hooks available for customization::
|
||||
|
||||
|
|
@ -322,18 +329,18 @@ of these methods are:
|
|||
under Windows, DLLs) in the environment.
|
||||
|
||||
* ``post_setup`` - A (no-op by default) hook method which can be
|
||||
overridden in third party implementations to pre-install packages or
|
||||
overridden in third party subclasses to pre-install packages or
|
||||
install scripts in the virtual environment.
|
||||
|
||||
In addition, ``EnvBuilder`` provides a utility method that can be
|
||||
called from ``post_setup`` in subclasses to assist in installing
|
||||
scripts into the virtual environment. The method ``install_scripts``
|
||||
accepts as arguments the ``context`` object (see above) and a
|
||||
bytestring. The bytestring should be a base64-encoded zip file
|
||||
containing directories "common", "posix", "nt", each containing
|
||||
scripts destined for the bin directory in the environment. The
|
||||
contents of "common" and the directory corresponding to ``os.name``
|
||||
are copied after doing some text replacement of placeholders:
|
||||
custom scripts into the virtual environment. The method
|
||||
``install_scripts`` accepts as arguments the ``context`` object (see
|
||||
above) and a path to a directory. The directory should contain
|
||||
subdirectories "common", "posix", "nt", each containing scripts
|
||||
destined for the bin directory in the environment. The contents of
|
||||
"common" and the directory corresponding to ``os.name`` are copied
|
||||
after doing some text replacement of placeholders:
|
||||
|
||||
* ``__VENV_DIR__`` is replaced with absolute path of the environment
|
||||
directory.
|
||||
|
|
@ -349,27 +356,10 @@ are copied after doing some text replacement of placeholders:
|
|||
|
||||
The ``DistributeEnvBuilder`` subclass in the reference implementation
|
||||
illustrates how the customization hook can be used in practice to
|
||||
pre-install Distribute and shell activation scripts into the virtual
|
||||
environment. It's not envisaged that ``DistributeEnvBuilder`` will be
|
||||
actually added to Python core, but it makes the reference
|
||||
implementation more immediately useful for testing and exploratory
|
||||
purposes.
|
||||
|
||||
The "shell activation scripts" provided by ``DistributeEnvBuilder``
|
||||
simply add the virtual environment's ``bin/`` (or ``Scripts\``)
|
||||
directory to the front of the user's shell PATH. This is not strictly
|
||||
necessary for use of a virtual environment (as an explicit path to the
|
||||
venv's python binary or scripts can just as well be used), but it is
|
||||
convenient.
|
||||
|
||||
This PEP does not propose that the ``venv`` module in core Python will
|
||||
add such activation scripts by default, as they are shell-specific.
|
||||
Adding activation scripts for the wide variety of possible shells is
|
||||
an added maintenance burden, and is left to third-party extension
|
||||
tools.
|
||||
|
||||
No doubt the process of PEP review will show up any customization
|
||||
requirements which have not yet been considered.
|
||||
pre-install Distribute into the virtual environment. It's not
|
||||
envisaged that ``DistributeEnvBuilder`` will be actually added to
|
||||
Python core, but it makes the reference implementation more
|
||||
immediately useful for testing and exploratory purposes.
|
||||
|
||||
|
||||
Backwards Compatibility
|
||||
|
|
@ -405,33 +395,40 @@ does not mention ``site-packages``.
|
|||
|
||||
__ http://docs.python.org/dev/library/sys.html#sys.prefix
|
||||
|
||||
This PEP currently proposes to leave ``sys.prefix`` pointing to the
|
||||
base system installation (which is where the standard library and
|
||||
header files are found), and introduce a new value in ``sys``
|
||||
(``sys.site_prefix``) to point to the prefix for ``site-packages``.
|
||||
This maintains the documented semantics of ``sys.prefix``, but risks
|
||||
breaking isolation if third-party code uses ``sys.prefix`` rather than
|
||||
``sys.site_prefix`` or the appropriate ``site`` API to find
|
||||
Maintaining this documented definition would mean leaving
|
||||
``sys.prefix`` pointing to the base system installation (which is
|
||||
where the standard library and header files are found), and
|
||||
introducing a new value in ``sys`` (something like
|
||||
``sys.site_prefix``) to point to the prefix for ``site-packages``.
|
||||
This would maintain the documented semantics of ``sys.prefix``, but
|
||||
risk breaking isolation if third-party code uses ``sys.prefix`` rather
|
||||
than ``sys.site_prefix`` or the appropriate ``site`` API to find
|
||||
site-packages directories.
|
||||
|
||||
The most notable case is probably `setuptools`_ and its fork
|
||||
`distribute`_, which mostly use ``distutils``/``sysconfig`` APIs, but
|
||||
do use ``sys.prefix`` directly to build up a list of site directories
|
||||
for pre-flight checking where ``pth`` files can usefully be placed.
|
||||
It would be trivial to modify these tools (currently only
|
||||
`distribute`_ is Python 3 compatible) to check ``sys.site_prefix`` and
|
||||
fall back to ``sys.prefix`` if it doesn't exist (for earlier versions
|
||||
of Python). If Distribute is modified in this way and released before
|
||||
Python 3.3 is released with the ``venv`` module, there would be no
|
||||
likely reason for an older version of Distribute to ever be installed
|
||||
in a virtual environment.
|
||||
`distribute`_, which mostly use ``distutils``and ``sysconfig`` APIs,
|
||||
but do use ``sys.prefix`` directly to build up a list of site
|
||||
directories for pre-flight checking where ``pth`` files can usefully
|
||||
be placed.
|
||||
|
||||
In terms of other third-party usage, a `Google Code Search`_ turns up
|
||||
what appears to be a roughly even mix of usage between packages using
|
||||
``sys.prefix`` to build up a site-packages path and packages using it
|
||||
to e.g. eliminate the standard-library from code-execution tracing.
|
||||
Either choice that's made here will require one or the other of these
|
||||
uses to be updated.
|
||||
Otherwise, a `Google Code Search`_ turns up what appears to be a
|
||||
roughly even mix of usage between packages using ``sys.prefix`` to
|
||||
build up a site-packages path and packages using it to e.g. eliminate
|
||||
the standard-library from code-execution tracing.
|
||||
|
||||
Although it requires modifying the documented definition of
|
||||
``sys.prefix``, this PEP prefers to have ``sys.prefix`` point to the
|
||||
virtual environment (where ``site-packages`` is found), and introduce
|
||||
``sys.base_prefix`` to point to the standard library and Python header
|
||||
files. Rationale for this choice:
|
||||
|
||||
* It is preferable to err on the side of greater isolation of the
|
||||
virtual environment.
|
||||
|
||||
* Virtualenv already modifies ``sys.prefix`` to point at the virtual
|
||||
environment, and in practice this has not been a problem.
|
||||
|
||||
* No modification is required to setuptools/distribute.
|
||||
|
||||
.. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools
|
||||
.. _distribute: http://packages.python.org/distribute/
|
||||
|
|
@ -441,72 +438,6 @@ uses to be updated.
|
|||
Open Questions
|
||||
==============
|
||||
|
||||
Naming of the new ``sys`` prefix attributes
|
||||
-------------------------------------------
|
||||
|
||||
The name ``sys.site_prefix`` was chosen with the following
|
||||
considerations in mind:
|
||||
|
||||
* Inasmuch as "site" has a meaning in Python, it means a combination
|
||||
of Python version, standard library, and specific set of
|
||||
site-packages. This is, fundamentally, what a venv is (although it
|
||||
shares the standard library with its "base" site).
|
||||
|
||||
* It is the Python ``site`` module which implements adding
|
||||
site-packages directories to ``sys.path``, so ``sys.site_prefix`` is
|
||||
a prefix used (and set) primarily by the ``site`` module.
|
||||
|
||||
A concern has been raised that the term ``site`` in Python is already
|
||||
overloaded and of unclear meaning, and this usage will increase the
|
||||
overload.
|
||||
|
||||
One proposed alternative is ``sys.venv_prefix``, which has the
|
||||
advantage of being clearly related to the venv implementation. The
|
||||
downside of this proposal is that it implies the attribute is only
|
||||
useful/relevant when in a venv and should be absent or ``None`` when
|
||||
not in a venv. This imposes an unnecessary extra burden on code using
|
||||
the attribute: ``sys.venv_prefix if sys.venv_prefix else sys.prefix``.
|
||||
The prefix attributes are more usable and general if they are always
|
||||
present and set, and split by meaning (stdlib vs site-packages,
|
||||
roughly), rather than specifically tied to venv. Also, third-party
|
||||
code should be encouraged to not know or care whether it is running in
|
||||
a virtual environment or not; this option seems to work against that
|
||||
goal.
|
||||
|
||||
Another option would be ``sys.local_prefix``, which has both the
|
||||
advantage and disadvantage, depending on perspective, that it
|
||||
introduces the new term "local" rather than drawing on existing
|
||||
associations with the term "site".
|
||||
|
||||
|
||||
Why not modify sys.prefix?
|
||||
--------------------------
|
||||
|
||||
As discussed above under `Backwards Compatibility`_, this PEP proposes
|
||||
to add ``sys.site_prefix`` as "the prefix relative to which
|
||||
site-package directories are found". This maintains compatibility
|
||||
with the documented meaning of ``sys.prefix`` (as the location
|
||||
relative to which the standard library can be found), but means that
|
||||
code assuming that site-packages directories are found relative to
|
||||
``sys.prefix`` will not respect the virtual environment correctly.
|
||||
|
||||
Since it is unable to modify ``distutils``/``sysconfig``,
|
||||
`virtualenv`_ is forced to instead re-point ``sys.prefix`` at the
|
||||
virtual environment.
|
||||
|
||||
An argument could be made that this PEP should follow virtualenv's
|
||||
lead here (and introduce something like ``sys.base_prefix`` to point
|
||||
to the standard library and header files), since virtualenv already
|
||||
does this and it doesn't appear to have caused major problems with
|
||||
existing code.
|
||||
|
||||
Another argument in favor of this is that it would be preferable to
|
||||
err on the side of greater, rather than lesser, isolation. Changing
|
||||
``sys.prefix`` to point to the virtual environment and introducing a
|
||||
new ``sys.base_prefix`` attribute would err on the side of greater
|
||||
isolation in the face of existing code's use of ``sys.prefix``.
|
||||
|
||||
|
||||
What about include files?
|
||||
-------------------------
|
||||
|
||||
|
|
@ -537,21 +468,6 @@ course this is not hard to do, but it does seem inelegant. OTOH it's
|
|||
really because there's no supporting concept in ``Python/sysconfig``.
|
||||
|
||||
|
||||
Interface with packaging tools
|
||||
------------------------------
|
||||
|
||||
Some work will be needed in packaging tools (Python 3.3 packaging,
|
||||
Distribute) to support implementation of this PEP. For example:
|
||||
|
||||
* How Distribute and packaging use ``sys.prefix`` and/or
|
||||
``sys.site_prefix``. Clearly, in practice we'll need to use
|
||||
Distribute for a while, until packages have migrated over to usage
|
||||
of setup.cfg.
|
||||
|
||||
* How packaging and Distribute set up shebang lines in scripts which they
|
||||
install in virtual environments.
|
||||
|
||||
|
||||
Testability and Source Build Issues
|
||||
-----------------------------------
|
||||
|
||||
|
|
@ -587,37 +503,6 @@ to not require that Xcode is installed.
|
|||
.. _Virtualenv uses: https://github.com/pypa/virtualenv/issues/168
|
||||
|
||||
|
||||
Activation and Utility Scripts
|
||||
------------------------------
|
||||
|
||||
Virtualenv provides shell "activation" scripts as a user convenience,
|
||||
to put the virtual environment's Python binary first on the shell
|
||||
PATH. This is a maintenance burden, as separate activation scripts
|
||||
need to be provided and maintained for every supported shell. For
|
||||
this reason, this PEP proposes to leave such scripts to be provided by
|
||||
third-party extensions; virtual environments created by the core
|
||||
functionality would be used by directly invoking the environment's
|
||||
Python binary or scripts.
|
||||
|
||||
If we are going to rely on external code to provide these
|
||||
conveniences, we need to check with existing third-party projects in
|
||||
this space (virtualenv, zc.buildout) and ensure that the proposed API
|
||||
meets their needs.
|
||||
|
||||
(Virtualenv would be fine with the proposed API; it would become a
|
||||
relatively thin wrapper with a subclass of the env builder that adds
|
||||
shell activation and automatic installation of ``pip`` inside the
|
||||
virtual environment).
|
||||
|
||||
|
||||
Provide a mode that is isolated only from user site packages?
|
||||
-------------------------------------------------------------
|
||||
|
||||
Is there sufficient rationale for providing a mode that isolates the
|
||||
venv from :pep:`370` user site packages, but not from the system-level
|
||||
site-packages?
|
||||
|
||||
|
||||
Other Python implementations?
|
||||
-----------------------------
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue