Updates to PEP 405.

pep-0405.txt

@@ -222,15 +222,32 @@ their executables placed in ``bin/`` or ``Scripts``.
     those files when Python is run from the venv.
 
 
+Sysconfig install schemes and user-site
+---------------------------------------
+
+This approach explicitly chooses not to introduce a new sysconfig
+install scheme for venvs. Rather, by modifying ``sys.prefix`` we
+ensure that existing install schemes which base locations on
+``sys.prefix`` will simply work in a venv. Installation to other
+install schemes (for instance, the user-site schemes) whose paths are
+not relative to ``sys.prefix``, will not be affected by a venv at all.
+
+It may be feasible to create an alternative implementation of Python
+virtual environments based on a virtual-specific sysconfig scheme, but
+it would be less robust, as it would require more code to be aware of
+whether it is operating within a virtual environment or not.
+
+
 Copies versus symlinks
 ----------------------
 
 The technique in this PEP works equally well in general with a copied
-or symlinked Python binary (and other needed DLLs on Windows).
-Symlinking is preferable where possible, because in the case of an
-upgrade to the underlying Python installation, a Python executable
-copied in a venv might become out-of-sync with the installed standard
-library and require manual upgrade.
+or symlinked Python binary (and other needed DLLs and the ``Include``
+directory on Windows).  Symlinking is preferable where possible,
+because in the case of an upgrade to the underlying Python
+installation, a Python executable copied in a venv might become
+out-of-sync with the installed standard library and require manual
+upgrade.
 
 There are some cross-platform difficulties with symlinks:
 
@@ -251,11 +268,50 @@ option has no effect on OS X framework builds, since symlinking can
 never work there, and has no advantages).
 
 On Windows, if ``--symlink`` is not used, this means that if the
-underlying Python installation is upgraded, the Python binary and DLLs
-in the venv should be updated, or there could be issues of mismatch
-with the upgraded standard library. The pyvenv script accepts a
-``--upgrade`` option for easily performing this upgrade on an existing
-venv.
+underlying Python installation is upgraded, the Python binary, DLLs,
+and include files in the venv should be updated, or there could be
+issues of mismatch with the upgraded standard library. The pyvenv
+script accepts a ``--upgrade`` option for easily performing this
+upgrade on an existing venv.
+
+
+Include files
+-------------
+
+Current virtualenv handles include files in this way:
+
+On POSIX systems where the installed Python's include files are found
+in ``${base_prefix}/include/pythonX.X``, virtualenv creates
+``${venv}/include/`` and symlink ``${base_prefix}/include/pythonX.X``
+to ``${venv}/include/pythonX.X``. On Windows, where Python's include
+files are found in ``{{ sys.prefix }}/Include`` and symlinks are not
+reliably available, virtualenv copies ``{{ sys.prefix }}/Include`` to
+``${venv}/Include``. This ensures that extension modules built and
+installed within the virtualenv will always find the Python header
+files they need in the expected location relative to ``sys.prefix``.
+
+This solution is not ideal when an extension module installs its own
+header files, as the default installation location for those header
+files may be a symlink to a system directory that may not be
+writable. One installer, pip, explicitly works around this by
+installing header files to a nonstandard location
+``${venv}/include/site/pythonX.X/``, as in Python there's currently no
+standard abstraction for a site-specific include directory.
+
+This PEP proposes a slightly different approach, though one with
+essentially the same effect and the same set of advantages and
+disadvantages. Rather than symlinking or copying include files into
+the venv, we simply modify the sysconfig schemes so that header files
+are always looked for relative to ``base_prefix`` rather than
+``prefix``. (We also create an ``include/`` directory within the venv
+
+Better handling of include files in distutils/packaging and, by
+extension, pyvenv, is an area that may deserve its own future PEP. For
+now, we propose that the behavior of virtualenv has thus far proved
+itself to be at least "good enough" in practice.
+
+[Open question: should pyvenv instead simply copy the behavior of
+virtualenv entirely, to avoid introducing unexpected new issues here?]
 
 
 API
@@ -449,39 +505,6 @@ Other Python implementations will need to replicate the new
 locating and parsing the ``pyvenv.cfg`` file, if it is present.
 
 
-Open Questions
-==============
-
-What about include files?
--------------------------
-
-For example, ZeroMQ installs ``zmq.h`` and ``zmq_utils.h`` in
-``$VE/include``, whereas SIP (part of PyQt4) installs sip.h by default
-in ``$VE/include/pythonX.Y``.  With virtualenv, everything works
-because the PythonX.Y include is symlinked, so everything that's
-needed is in ``$VE/include``.  At the moment the reference
-implementation doesn't do anything with include files, besides
-creating the include directory; this might need to change, to
-copy/symlink ``$VE/include/pythonX.Y``.
-
-As in Python there's no abstraction for a site-specific include
-directory, other than for platform-specific stuff, then the user
-expectation would seem to be that all include files anyone could ever
-want should be found in one of just two locations, with sysconfig
-labels "include" & "platinclude".
-
-There's another issue: what if includes are Python-version-specific?
-For example, SIP installs by default into ``$VE/include/pythonX.Y``
-rather than ``$VE/include``, presumably because there's
-version-specific stuff in there - but even if that's not the case with
-SIP, it could be the case with some other package.  And the problem
-that gives is that you can't just symlink the ``include/pythonX.Y``
-directory, but actually have to provide a writable directory and
-symlink/copy the contents from the system ``include/pythonX.Y``.  Of
-course this is not hard to do, but it does seem inelegant.  OTOH it's
-really because there's no supporting concept in ``Python/sysconfig``.
-
-
 Reference Implementation
 ========================