PEP 406 (ImportEngine API): defer this until 3.4 at the earliest, don't propose altering the PEP 302 APIs
This commit is contained in:
parent
7b079b3a7f
commit
b2fc8ab349
127
pep-0406.txt
127
pep-0406.txt
|
@ -3,11 +3,12 @@ Title: Improved Encapsulation of Import State
|
||||||
Version: $Revision$
|
Version: $Revision$
|
||||||
Last-Modified: $Date$
|
Last-Modified: $Date$
|
||||||
Author: Nick Coghlan <ncoghlan@gmail.com>, Greg Slodkowicz <jergosh@gmail.com>
|
Author: Nick Coghlan <ncoghlan@gmail.com>, Greg Slodkowicz <jergosh@gmail.com>
|
||||||
Status: Draft
|
Status: Deferred
|
||||||
Type: Standards Track
|
Type: Standards Track
|
||||||
Content-Type: text/x-rst
|
Content-Type: text/x-rst
|
||||||
Created: 4-Jul-2011
|
Created: 4-Jul-2011
|
||||||
Post-History: 31-Jul-2011, 13-Nov-2011
|
Python-Version: 3.4
|
||||||
|
Post-History: 31-Jul-2011, 13-Nov-2011, 4-Dec-2011
|
||||||
|
|
||||||
Abstract
|
Abstract
|
||||||
========
|
========
|
||||||
|
@ -18,17 +19,27 @@ into a single object. Creating new instances of this object would then provide
|
||||||
an alternative to completely replacing the built-in implementation of the
|
an alternative to completely replacing the built-in implementation of the
|
||||||
import statement, by overriding the ``__import__()`` function. To work with
|
import statement, by overriding the ``__import__()`` function. To work with
|
||||||
the builtin import functionality and importing via import engine objects,
|
the builtin import functionality and importing via import engine objects,
|
||||||
module importers and loaders must accept an optional ``engine`` parameter. In
|
this PEP proposes a context management based approach to temporarily replacing
|
||||||
that sense, this PEP constitutes a revision of finder and loader interfaces
|
the global import state.
|
||||||
described in PEP 302 [1]_. However, the standard import process will not
|
|
||||||
supply the additional argument, so this proposal remains fully backwards
|
|
||||||
compatible.
|
|
||||||
|
|
||||||
The PEP also proposes inclusion of a ``GlobalImportEngine`` subclass and a
|
The PEP also proposes inclusion of a ``GlobalImportEngine`` subclass and a
|
||||||
globally accessible instance of that class, which "writes through" to the
|
globally accessible instance of that class, which "writes through" to the
|
||||||
process global state and invokes importers and loaders without the additional
|
process global state. This provides a backwards compatible bridge between the
|
||||||
``engine`` argument. This provides a backwards compatible bridge between the
|
proposed encapsulated API and the legacy process global state, and allows
|
||||||
proposed encapsulated API and the legacy process global state.
|
straightforward support for related state updates (e.g. selectively
|
||||||
|
invalidating path cache entries when ``sys.path`` is modified).
|
||||||
|
|
||||||
|
|
||||||
|
PEP Deferral
|
||||||
|
============
|
||||||
|
|
||||||
|
The import system is already seeing substantial changes in Python 3.3, to
|
||||||
|
natively handle packages split across multiple directories (PEP 382) and
|
||||||
|
(potentially) to make the import semantics in the main module better match
|
||||||
|
those in other modules (PEP 395).
|
||||||
|
|
||||||
|
Accordingly, the proposal in this PEP will not be seriously considered until
|
||||||
|
Python 3.4 at the earliest.
|
||||||
|
|
||||||
|
|
||||||
Rationale
|
Rationale
|
||||||
|
@ -58,6 +69,10 @@ The namespace PEPs (especially PEP 402) raise a potential need for
|
||||||
*additional* process global state, in order to correctly update package paths
|
*additional* process global state, in order to correctly update package paths
|
||||||
as ``sys.path`` is modified.
|
as ``sys.path`` is modified.
|
||||||
|
|
||||||
|
Finally, providing a coherent object for all this state makes it feasible to
|
||||||
|
also provide context management features that allow the import state to be
|
||||||
|
temporarily substituted.
|
||||||
|
|
||||||
|
|
||||||
Proposal
|
Proposal
|
||||||
========
|
========
|
||||||
|
@ -68,11 +83,10 @@ be used as an alternative to the built-in ``__import__()`` when
|
||||||
desired and also an ``import_module()`` method, equivalent to
|
desired and also an ``import_module()`` method, equivalent to
|
||||||
``importlib.import_module()`` [3]_.
|
``importlib.import_module()`` [3]_.
|
||||||
|
|
||||||
Since the new style finders and loaders should also have the option to
|
Since there are global import state invariants that are assumed and should be
|
||||||
modify the global import state, we introduce a ``GlobalImportState``
|
maintained, we introduce a ``GlobalImportState`` class with an interface
|
||||||
class with an interface identical to ``ImportEngine`` but taking
|
identical to ``ImportEngine`` but directly accessing the current global import
|
||||||
advantage of the global state. This can be easily implemented using
|
state. This can be easily implemented using class properties.
|
||||||
class properties.
|
|
||||||
|
|
||||||
|
|
||||||
Specification
|
Specification
|
||||||
|
@ -121,6 +135,14 @@ The proposed extension consists of the following objects:
|
||||||
methods like ``ImportEngine`` but writes through to the global state
|
methods like ``ImportEngine`` but writes through to the global state
|
||||||
in ``sys``.
|
in ``sys``.
|
||||||
|
|
||||||
|
To support various namespace package mechanisms, when ``sys.path`` is altered,
|
||||||
|
tools like ``pkgutil.extend_path`` should be used to also modify other parts
|
||||||
|
of the import state (in this case, package ``__path__`` attributes). The path
|
||||||
|
importer cache should also be invalidated when a variety of changes are made.
|
||||||
|
|
||||||
|
The ``ImportEngine`` API will provide convenience methods that automatically
|
||||||
|
make related import state updates as part of a single operation.
|
||||||
|
|
||||||
|
|
||||||
Global variables
|
Global variables
|
||||||
~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~
|
||||||
|
@ -133,24 +155,26 @@ Global variables
|
||||||
a copy of the process global import state.
|
a copy of the process global import state.
|
||||||
|
|
||||||
|
|
||||||
Necessary changes to finder/loader interfaces:
|
No changes to finder/loader interfaces
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
``find_module (cls, fullname, path=None, engine=None)``
|
Rather than attempting to update the PEP 302 APIs to accept additional state,
|
||||||
|
this PEP proposes that ``ImportEngine`` support the content management
|
||||||
|
protocol (similar to the context substitution mechanisms in the ``decimal``
|
||||||
|
module).
|
||||||
|
|
||||||
``load_module (cls, fullname, path=None, engine=None)``
|
The context management mechanism for ``ImportEngine`` would:
|
||||||
|
|
||||||
The only difference between engine compatible and PEP 302 compatible
|
* On entry:
|
||||||
finders/loaders is the presence of an additional ``engine`` parameter.
|
* Acquire the import lock
|
||||||
This is intended to specify an ImportEngine instance or subclass thereof.
|
* Substitute the global import state with the import engine's own state
|
||||||
This parameter is optional so that engine compatible finders and
|
* On exit:
|
||||||
loaders can be made backwards compatible with PEP 302 calling conventions by
|
* Restore the previous global import state
|
||||||
falling back on ``engine.sysengine`` with the following simple pattern::
|
* Release the import lock
|
||||||
|
|
||||||
def find_module(cls, fullname, path=None, engine=None):
|
The precise API for this is TBD (but will probably use a distinct context
|
||||||
if not engine:
|
management object, along the lines of that created by
|
||||||
engine = importlib.engine.sysengine
|
``decimal.localcontext``).
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
Open Issues
|
Open Issues
|
||||||
|
@ -185,35 +209,38 @@ normally.
|
||||||
cache - it's only loading them directly which causes problems.
|
cache - it's only loading them directly which causes problems.
|
||||||
|
|
||||||
|
|
||||||
Nested imports
|
Scope of substitution
|
||||||
~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The reference implementation currently applies only to the outermost import.
|
Related to the previous open issue is the question of what state to substitute
|
||||||
Any imports by the module being imported will be handled using the standard
|
when using the context management API. It is currently the case that replacing
|
||||||
import machinery.
|
``sys.modules`` can be unreliable due to cached references and there's the
|
||||||
|
underlying fact that having independent copies of some modules is simply
|
||||||
|
impossible due to platform limitations.
|
||||||
|
|
||||||
One way to handle this is to place the burden on the implementation of module
|
As part of this PEP, it will be necessary to document explicitly:
|
||||||
loaders to set ``module.__dict__["__import__"] = engine.__import__`` before
|
|
||||||
running the module's code. The ``importlib`` design facilities this by
|
|
||||||
allowing the change to be made in one place (``_LoaderBasics._load_module``).
|
|
||||||
|
|
||||||
|
* Which parts of the global import state can be substituted (and declare code
|
||||||
Scope of API updates
|
which caches references to that state without dealing with the substitution
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
case buggy)
|
||||||
|
* Which parts must be modified in-place (and hence are not substituted by the
|
||||||
The reference implementation focuses on finding and loading modules. There
|
``ImportEngine`` context management API, or otherwise scoped to
|
||||||
may be other PEP 302 APIs that should also be updated to accept an optional
|
``ImportEngine`` instances)
|
||||||
``engine`` parameter.
|
|
||||||
|
|
||||||
|
|
||||||
Reference Implementation
|
Reference Implementation
|
||||||
========================
|
========================
|
||||||
|
|
||||||
A reference implementation [4]_ based on Brett Cannon's importlib has been
|
A reference implementation [4]_ for an earlier draft of this PEP, based on
|
||||||
developed by Greg Slodkowicz as part of the 2011 Google Summer of Code. Note
|
Brett Cannon's importlib has been developed by Greg Slodkowicz as part of the
|
||||||
that the current implementation avoids modifying existing code, and hence
|
2011 Google Summer of Code. Note that the current implementation avoids
|
||||||
duplicates a lot of things unnecessarily. An actual implementation would just
|
modifying existing code, and hence duplicates a lot of things unnecessarily.
|
||||||
modify any such affected code in place.
|
An actual implementation would just modify any such affected code in place.
|
||||||
|
|
||||||
|
That earlier draft of the PEP proposed change the PEP 302 APIs to support passing
|
||||||
|
in an optional engine instance. This had the (serious) downside of not correctly
|
||||||
|
affecting further imports from the imported module, hence the change to the
|
||||||
|
context management based proposal for substituting the global state.
|
||||||
|
|
||||||
|
|
||||||
References
|
References
|
||||||
|
|
Loading…
Reference in New Issue