iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[PEP 451] Updates in response to comments. 

This includes the addition of a list of terms and concepts.
This commit is contained in:
Eric Snow 2013-09-24 23:31:29 -06:00
parent 98242cd315
commit c05cf5453c
1 changed files with 322 additions and 129 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

451
pep-0451.txt
View File

@ -16,21 +16,143 @@ Resolution:
Abstract Abstract
======== ========
This PEP proposes to add a new class to ``importlib.machinery`` called This PEP proposes to add a new class to importlib.machinery called
``ModuleSpec``. It will be authoritative for all the import-related "ModuleSpec". It will provide all the import-related information used
information about a module, and will be available without needing to to load a module and will be available without needing to load the
load the module first. Finders will directly provide a module's spec module first. Finders will directly provide a module's spec instead of
instead of a loader (which they will continue to provide indirectly). a loader (which they will continue to provide indirectly). The import
The import machinery will be adjusted to take advantage of module specs, machinery will be adjusted to take advantage of module specs, including
including using them to load modules. using them to load modules.
Terms and Concepts
==================
The changes in this proposal are an opportunity to make several
existing terms and concepts more clear, whereas currently they are
(unfortunately) ambiguous. New concepts are also introduced in this
proposal. Finally, it's worth explaining a few other existing terms
with which people may not be so familiar. For the sake of context, here
is a brief summary of all three groups of terms and concepts. A more
detailed explanation of the import system is found at
[import_system_docs]_.
finder
------
A "finder" is an object that identifies the loader that the import
system should use to load a module. Currently this is accomplished by
calling the finder's find_module() method, which returns the loader.
Finders are strictly responsible for providing the loader, which they do
through their find_module() method. The import system then uses that
loader to load the module.
loader
------
A "loader" is an object that is used to load a module during import.
Currently this is done by calling the loader's load_module() method. A
loader may also provide APIs for getting information about the modules
it can load, as well as about data from sources associated with such a
module.
Right now loaders (via load_module()) are responsible for certain
boilerplate import-related operations. These are:
1. perform some (module-related) validation;
2. create the module object;
3. set import-related attributes on the module;
4. "register" the module to sys.modules;
5. exec the module;
6. clean up in the event of failure while loading the module.
This all takes place during the import system's call to
Loader.load_module().
origin
------
This is a new term and concept. The idea of it exists subtly in the
import system already, but this proposal makes the concept explicit.
"origin" is the import context means the system (or resource within a
system) from which a module originates. For the purposes of this
proposal, "origin" is also a string which identifies such a resource or
system. "origin" is applicable to all modules.
For example, the origin for built-in and frozen modules is the
interpreter itself. The import system already identifies this origin as
"built-in" and "frozen", respectively. This is demonstrated in the
following module repr: "<module 'sys' (built-in)>".
In fact, the module repr is already a relatively reliable, though
implicit, indicator of a module's origin. Other modules also indicate
their origin through other means, as described in the entry for
"location".
It is up to the loader to decide on how to interpret and use a module's
origin, if at all.
location
--------
This is a new term. However the concept already exists clearly in the
import system, as associated with the ``__file__`` and ``__path__``
attributes of modules, as well as the name/term "path" elsewhere.
A "location" is a resource or "place", rather than a system at large,
from which a module is loaded. It qualifies as an "origin". Examples
of locations include filesystem paths and URLs. A location is
identified by the name of the resource, but may not necessarily identify
the system to which the resource pertains. In such cases the loader
would have to identify the system itself.
In contrast to other kinds of module origin, a location cannot be
inferred by the loader just by the module name. Instead, the loader
must be provided with a string to identify the location, usually by the
finder that generates the loader. The loader then uses this information
to locate the resource from which it will load the module. In theory
you could load the module at a given location under various names.
The most common example of locations in the import system are the
files from which source and extension modules are loaded. For these
modules the location is identified by the string in the ``__file__``
attribute. Although ``__file__`` isn't particularly accurate for some
modules (e.g. zipped), it is currently the only way that the import
system indicates that a module has a location.
A module that has a location may be called "locatable".
cache
-----
The import system stores compiled modules in the __pycache__ directory
as an optimization. This module cache that we use today was provided by
PEP 3147. For this proposal, the relevant API for module caching is the
``__cache__`` attribute of modules and the cache_from_source() function
in importlib.util. Loaders are responsible for putting modules into the
cache (and loading out of the cache). Currently the cache is only used
for compiled source modules. However, this proposal explicitly allows
package
-------
The concept does not change, nor does the term. However, the
distinction between modules and packages is mostly superficial.
Packages *are* modules. They simply have a ``__path__`` attribute and
import may add attributes bound to submodules. The typical perceived
difference is a source of confusion. This proposal explicitly
de-emphasizes the distinction between packages and modules where it
makes sense to do so.
Motivation Motivation
========== ==========
The import system has evolved over the lifetime of Python. In late 2002 The import system has evolved over the lifetime of Python. In late 2002
PEP 302 introduced standardized import hooks via ``finders`` and PEP 302 introduced standardized import hooks via finders and
``loaders`` and ``sys.meta_path``. The ``importlib`` module, introduced loaders and sys.meta_path. The importlib module, introduced
with Python 3.1, now exposes a pure Python implementation of the APIs with Python 3.1, now exposes a pure Python implementation of the APIs
described by PEP 302, as well as of the full import system. It is now described by PEP 302, as well as of the full import system. It is now
much easier to understand and extend the import system. While a benefit much easier to understand and extend the import system. While a benefit
@ -48,32 +170,30 @@ generally only meaningful to the import system. It would be nice to
have a per-module namespace in which to put future import-related have a per-module namespace in which to put future import-related
information and to pass around within the import system. Secondly, information and to pass around within the import system. Secondly,
there's an API void between finders and loaders that causes undue there's an API void between finders and loaders that causes undue
complexity when encountered. complexity when encountered. The PEP 420 (namespace packages)
implementation had to work around this. The complexity surfaced again
during recent efforts on a separate proposal. [ref_files_pep]_
Currently finders are strictly responsible for providing the loader, The `finder`_ and `loader`_ sections above detail current responsibility
through their find_module() method, which the import system will use to of both. Notably, loaders are not required to provide any of the
load the module. The loader is then responsible for doing some checks, functionality of their load_module() through other methods. Thus,
creating the module object, setting import-related attributes, though the import-related information about a module is likely available
"installing" the module to ``sys.modules``, and loading the module, without loading the module, it is not otherwise exposed.
along with some cleanup. This all takes place during the import
system's call to ``Loader.load_module()``. Loaders also provide some
APIs for accessing data associated with a module.
Loaders are not required to provide any of the functionality of Furthermore, the requirements assocated with load_module() are
``load_module()`` through other methods. Thus, though the import-
related information about a module is likely available without loading
the module, it is not otherwise exposed.
Furthermore, the requirements assocated with ``load_module()`` are
common to all loaders and mostly are implemented in exactly the same common to all loaders and mostly are implemented in exactly the same
way. This means every loader has to duplicate the same boilerplate way. This means every loader has to duplicate the same boilerplate
code. ``importlib.util`` provides some tools that help with this, but code. importlib.util provides some tools that help with this, but
it would be more helpful if the import system simply took charge of it would be more helpful if the import system simply took charge of
these responsibilities. The trouble is that this would limit the degree these responsibilities. The trouble is that this would limit the degree
of customization that ``load_module()`` facilitates. This is a gap of customization that load_module() could easily continue to facilitate.
between finders and loaders which this proposal aims to fill.
Finally, when the import system calls a finder's ``find_module()``, the More importantly, While a finder *could* provide the information that
the loader's load_module() would need, it currently has no consistent
way to get it to the loader. This is a gap between finders and loaders
which this proposal aims to fill.
Finally, when the import system calls a finder's find_module(), the
finder makes use of a variety of information about the module that is finder makes use of a variety of information about the module that is
useful outside the context of the method. Currently the options are useful outside the context of the method. Currently the options are
limited for persisting that per-module information past the method call, limited for persisting that per-module information past the method call,
@ -89,32 +209,31 @@ loaders.
As an example of complexity attributable to this flaw, the As an example of complexity attributable to this flaw, the
implementation of namespace packages in Python 3.3 (see PEP 420) added implementation of namespace packages in Python 3.3 (see PEP 420) added
``FileFinder.find_loader()`` because there was no good way for FileFinder.find_loader() because there was no good way for
``find_module()`` to provide the namespace search locations. find_module() to provide the namespace search locations.
The answer to this gap is a ``ModuleSpec`` object that contains the The answer to this gap is a ModuleSpec object that contains the
per-module information and takes care of the boilerplate functionality per-module information and takes care of the boilerplate functionality
involved with loading the module. involved with loading the module.
(The idea gained momentum during discussions related to another PEP.[1]_)
Specification Specification
============= =============
The goal is to address the gap between finders and loaders while The goal is to address the gap between finders and loaders while
changing as little of their semantics as possible. Though some changing as little of their semantics as possible. Though some
functionality and information is moved to the new ``ModuleSpec`` type, functionality and information is moved to the new ModuleSpec type,
their behavior should remain the same. However, for the sake of clarity their behavior should remain the same. However, for the sake of clarity
the finder and loader semantics will be explicitly identified. the finder and loader semantics will be explicitly identified.
This is a high-level summary of the changes described by this PEP. More Here is a high-level summary of the changes described by this PEP. More
detail is available in later sections. detail is available in later sections.
importlib.machinery.ModuleSpec (new) importlib.machinery.ModuleSpec (new)
------------------------------------ ------------------------------------
A specification for a module's import-system-related state. A specification for a module's import-system-related state. See the
`ModuleSpec`_ section below for a more detailed description.
* ModuleSpec(name, loader, \*, origin=None, loader_state=None, is_package=None) * ModuleSpec(name, loader, \*, origin=None, loader_state=None, is_package=None)
@ -122,49 +241,67 @@ Attributes:
* name - a string for the name of the module. * name - a string for the name of the module.
* loader - the loader to use for loading. * loader - the loader to use for loading.
* origin - a string for the location from which the module is loaded, * origin - the name of the place from which the module is loaded,
e.g. "builtin" for built-in modules and the filename for modules e.g. "builtin" for built-in modules and the filename for modules
loaded from source. loaded from source.
* submodule_search_locations - list of strings for where to find * submodule_search_locations - list of strings for where to find
submodules, if a package (None otherwise). submodules, if a package (None otherwise).
* loader_state - a container of extra data for use during loading. * loader_state - a container of extra module-specific data for use
* cached (property) - a string for where the compiled module will be during loading.
stored (see PEP 3147). * cached (property) - a string for where the compiled module should be
* package (RO-property) - the name of the module's parent (or None). stored.
* has_location (RO-property) - the module's origin refers to a location. * parent (RO-property) - the name of the package to which the module
belongs as a submodule (or None).
* has_location (RO-property) - a flag indicating whether or not the
module's "origin" attribute refers to a location.
Instance Methods: Instance Methods:
* module_repr() - provide a repr string for the spec'ed module. * module_repr() - provide a repr string for the spec'ed module;
non-locatable modules will use their origin (e.g. "built-in").
* init_module_attrs(module) - set any of a module's import-related * init_module_attrs(module) - set any of a module's import-related
attributes that aren't already set. attributes that aren't already set.
importlib.util Additions importlib.util Additions
------------------------ ------------------------
These are ModuleSpec factory functions, meant as a convenience for
finders. See the `Factory Functions`_ section below for more detail.
* spec_from_file_location(name, location, \*, loader=None, submodule_search_locations=None) * spec_from_file_location(name, location, \*, loader=None, submodule_search_locations=None)
- factory for file-based module specs. - build a spec from file-oriented information and loader APIs.
* from_loader(name, loader, \*, origin=None, is_package=None) - factory * from_loader(name, loader, \*, origin=None, is_package=None) - build
based on information provided by loaders. a spec with missing information filled in by using loader APIs.
* spec_from_module(module, loader=None) - factory based on existing
import-related module attributes. This function is expected to be This factory function is useful for some backward-compatibility
used only in some backward-compatibility situations. situations:
* spec_from_module(module, loader=None) - build a spec based on the
import-related attributes of an existing module.
Other API Additions Other API Additions
------------------- -------------------
* importlib.find_spec(name, path=None) will work exactly the same as
importlib.find_loader() (which it replaces), but return a spec instead
of a loader.
For loaders:
* importlib.abc.Loader.exec_module(module) will execute a module in its * importlib.abc.Loader.exec_module(module) will execute a module in its
own namespace. It replaces ``importlib.abc.Loader.load_module()``. own namespace. It replaces importlib.abc.Loader.load_module(), taking
* importlib.abc.Loader.create_module(spec) (optional) will return a new over its module execution functionality.
* importlib.abc.Loader.create_module(spec) (optional) will return the
module to use for loading. module to use for loading.
For modules:
* Module objects will have a new attribute: ``__spec__``. * Module objects will have a new attribute: ``__spec__``.
* importlib.find_spec(name, path=None) will return the spec for a
module.
API Changes API Changes
----------- -----------
* ``InspectLoader.is_package()`` will become optional. * InspectLoader.is_package() will become optional.
Deprecations Deprecations
------------ ------------
@ -183,7 +320,8 @@ Deprecations
Removals Removals
-------- --------
These were introduced prior to Python 3.4's release. These were introduced prior to Python 3.4's release, so they can simply
be removed.
* importlib.abc.Loader.init_module_attrs() * importlib.abc.Loader.init_module_attrs()
* importlib.util.module_to_load() * importlib.util.module_to_load()
@ -193,6 +331,7 @@ Other Changes
* The import system implementation in importlib will be changed to make * The import system implementation in importlib will be changed to make
use of ModuleSpec. use of ModuleSpec.
* importlib.reload() will make use of ModuleSpec.
* Import-related module attributes (other than ``__spec__``) will no * Import-related module attributes (other than ``__spec__``) will no
longer be used directly by the import system. longer be used directly by the import system.
* Import-related attributes should no longer be added to modules * Import-related attributes should no longer be added to modules
@ -224,6 +363,31 @@ What Will not Change?
* Loaders will still be responsible for module data APIs. * Loaders will still be responsible for module data APIs.
* importlib.reload() will still overwrite the import-related attributes. * importlib.reload() will still overwrite the import-related attributes.
Responsibilities
----------------
Here's a quick breakdown of where responsibilities lie after this PEP.
finders:
* create loader
* create spec
loaders:
* create module (optional)
* execute module
ModuleSpec:
* orchestrate module loading
* boilerplate for module loading, including managing sys.modules and
setting import-related attributes
* create module if loader doesn't
* call loader.exec_module(), passing in the module in which to exec
* contain all the information the loader needs to exec the module
* provide the repr for modules
What Will Existing Finders and Loaders Have to Do Differently? What Will Existing Finders and Loaders Have to Do Differently?
============================================================== ==============================================================
@ -232,14 +396,14 @@ Immediately? Nothing. The status quo will be deprecated, but will
continue working. However, here are the things that the authors of continue working. However, here are the things that the authors of
finders and loaders should change relative to this PEP: finders and loaders should change relative to this PEP:
* Implement ``find_spec()`` on finders. * Implement find_spec() on finders.
* Implement ``exec_module()`` on loaders, if possible. * Implement exec_module() on loaders, if possible.
The ModuleSpec factory functions in importlib.util are intended to be The ModuleSpec factory functions in importlib.util are intended to be
helpful for converting existing finders. ``from_loader()`` and helpful for converting existing finders. from_loader() and
``from_file_location()`` are both straight-forward utilities in this from_file_location() are both straight-forward utilities in this
regard. In the case where loaders already expose methods for creating regard. In the case where loaders already expose methods for creating
and preparing modules, ``ModuleSpec.from_module()`` may be useful to and preparing modules, ModuleSpec.from_module() may be useful to
the corresponding finder. the corresponding finder.
For existing loaders, exec_module() should be a relatively direct For existing loaders, exec_module() should be a relatively direct
@ -250,13 +414,13 @@ uncommon cases the loader should also implement create_module().
ModuleSpec Users ModuleSpec Users
================ ================
``ModuleSpec`` objects have 3 distinct target audiences: Python itself, ModuleSpec objects have 3 distinct target audiences: Python itself,
import hooks, and normal Python users. import hooks, and normal Python users.
Python will use specs in the import machinery, in interpreter startup, Python will use specs in the import machinery, in interpreter startup,
and in various standard library modules. Some modules are and in various standard library modules. Some modules are
import-oriented, like pkgutil, and others are not, like pickle and import-oriented, like pkgutil, and others are not, like pickle and
pydoc. In all cases, the full ``ModuleSpec`` API will get used. pydoc. In all cases, the full ModuleSpec API will get used.
Import hooks (finders and loaders) will make use of the spec in specific Import hooks (finders and loaders) will make use of the spec in specific
ways. First of all, finders may use the spec factory functions in ways. First of all, finders may use the spec factory functions in
@ -300,7 +464,7 @@ functionality::
raise raise
return sys.modules[spec.name] return sys.modules[spec.name]
These steps are exactly what ``Loader.load_module()`` is already These steps are exactly what Loader.load_module() is already
expected to do. Loaders will thus be simplified since they will only expected to do. Loaders will thus be simplified since they will only
need to implement exec_module(). need to implement exec_module().
@ -319,7 +483,7 @@ Attributes
---------- ----------
Each of the following names is an attribute on ModuleSpec objects. A Each of the following names is an attribute on ModuleSpec objects. A
value of ``None`` indicates "not set". This contrasts with module value of None indicates "not set". This contrasts with module
objects where the attribute simply doesn't exist. Most of the objects where the attribute simply doesn't exist. Most of the
attributes correspond to the import-related attributes of modules. Here attributes correspond to the import-related attributes of modules. Here
is the mapping. The reverse of this mapping is used by is the mapping. The reverse of this mapping is used by
@ -338,8 +502,8 @@ loader_state \-
has_location \- has_location \-
========================== ============== ========================== ==============
\* Set on the module only if spec.has_location is true. | \* Set on the module only if spec.has_location is true.
\*\* Set on the module only if the spec attribute is not None. | \*\* Set on the module only if the spec attribute is not None.
While package and has_location are read-only properties, the remaining While package and has_location are read-only properties, the remaining
attributes can be replaced after the module spec is created and even attributes can be replaced after the module spec is created and even
@ -349,46 +513,41 @@ involve changing the state of a module's spec.
**origin** **origin**
origin is a string for the place from which the module originates. "origin" is a string for the name of the place from which the module
Aside from the informational value, it is also used in module_repr(). originates. See `origin`_ above. Aside from the informational value,
it is also used in module_repr(). In the case of a spec where
The module attribute ``__file__`` has a similar but more restricted "has_location" is true, ``__file__`` is set to the value of "origin".
meaning. Not all modules have it set (e.g. built-in modules). However, For built-in modules "origin" would be set to "built-in".
``origin`` is applicable to all modules. For built-in modules it would
be set to "built-in".
**has_location** **has_location**
Some modules can be loaded by reference to a location, e.g. a filesystem As explained in the `location`_ section above, many modules are
path or a URL or something of the sort. Having the location lets you "locatable", meaning there is a corresponding resource from which the
load the module, but in theory you could load that module under various module will be loaded and that resource can be described by a string.
names. In contrast, non-locatable modules can't be loaded in this fashion, e.g.
In contrast, non-located modules can't be loaded in this fashion, e.g.
builtin modules and modules dynamically created in code. For these, the builtin modules and modules dynamically created in code. For these, the
name is the only way to access them, so they have an "origin" but not a name is the only way to access them, so they have an "origin" but not a
"location". "location".
This attribute reflects whether or not the module is locatable. If it "has_location" is true if the module is locatable. In that case the
is, origin must be set to the module's location and ``__file__`` will be spec's origin is used as the location and ``__file__`` is set to
set on the module. Not all locatable modules will be cachable, but most spec.origin. If additional location information is required (e.g.
will. zipimport), that information may be stored in spec.loader_state.
The corresponding module attribute name, ``__file__``, is somewhat "has_location" may be implied from the existence of a load_data() method
inaccurate and potentially confusing, so we will use a more explicit on the loader.
combination of origin and has_location to represent the same
information. Having a separate "filename" is unncessary since we have Incidently, not all locatable modules will be cachable, but most will.
"origin".
**submodule_search_locations** **submodule_search_locations**
The list of location strings, typically directory paths, in which to The list of location strings, typically directory paths, in which to
search for submodules. If the module is a package this will be set to search for submodules. If the module is a package this will be set to
a list (even an empty one). Otherwise it is ``None``. a list (even an empty one). Otherwise it is None.
The corresponding module attribute's name, ``__path__``, is relatively The name of the corresponding module attribute, ``__path__``, is
ambiguous. Instead of mirroring it, we use a more explicit name that relatively ambiguous. Instead of mirroring it, we use a more explicit
makes the purpose clear. name that makes the purpose clear.
**loader_state** **loader_state**
@ -405,6 +564,38 @@ or create a custom loader for each find operation.
loader_state is meant for use by the finder and corresponding loader. loader_state is meant for use by the finder and corresponding loader.
It is not guaranteed to be a stable resource for any other use. It is not guaranteed to be a stable resource for any other use.
Factory Functions
-----------------
**spec_from_file_location(name, location, \*, loader=None, submodule_search_locations=None)**
Build a spec from file-oriented information and loader APIs.
* "origin" will be set to the location.
* "has_location" will be set to True.
* "cached" will be set to the result of calling cache_from_source().
* "origin" can be deduced from loader.get_filename() (if "location" is
not passed in.
* "loader" can be deduced from suffix if the location is a filename.
* "submodule_search_locations" can be deduced from loader.is_package()
and from os.path.dirname(location) if locatin is a filename.
**from_loader(name, loader, \*, origin=None, is_package=None)**
Build a spec with missing information filled in by using loader APIs.
* "has_location" can be deduced from loader.get_data.
* "origin" can be deduced from loader.get_filename().
* "submodule_search_locations" can be deduced from loader.is_package()
and from os.path.dirname(location) if locatin is a filename.
**spec_from_module(module, loader=None)**
Build a spec based on the import-related attributes of an existing
module. The spec attributes are set to the corresponding import-
related module attributes. See the table in `Attributes`_.
Omitted Attributes and Methods Omitted Attributes and Methods
------------------------------ ------------------------------
@ -419,13 +610,13 @@ needs them (i.e. they would be an attractive nuisance).
Here are other omissions: Here are other omissions:
There is no PathModuleSpec subclass of ModuleSpec that separates out There is no "PathModuleSpec" subclass of ModuleSpec that separates out
has_location, cached, and submodule_search_locations. While that might has_location, cached, and submodule_search_locations. While that might
make the separation cleaner, module objects don't have that distinction. make the separation cleaner, module objects don't have that distinction.
ModuleSpec will support both cases equally well. ModuleSpec will support both cases equally well.
While is_package would be a simple additional attribute (aliasing While "is_package" would be a simple additional attribute (aliasing
``self.submodule_search_locations is not None``), it perpetuates the self.submodule_search_locations is not None), it perpetuates the
artificial (and mostly erroneous) distinction between modules and artificial (and mostly erroneous) distinction between modules and
packages. packages.
@ -446,7 +637,7 @@ Others:
any. any.
* Add ModuleSpec.data - a descriptor that wraps the data API of the * Add ModuleSpec.data - a descriptor that wraps the data API of the
spec's loader. spec's loader.
* Also see [3]. * Also see [cleaner_reload_support]_.
Backward Compatibility Backward Compatibility
@ -488,15 +679,16 @@ using the ``-m`` flag. In that case ``module.__spec__.name`` will
reflect the actual module name while ``module.__name__`` will be reflect the actual module name while ``module.__name__`` will be
``__main__``. ``__main__``.
Notably, the spec for each module instance will be unique to that A module's spec is not guaranteed to be identical between two modules
instance even if the information is identical to that of another spec. with the same name. Likewise there is no guarantee that successive
This won't happen in general. calls to importlib.find_spec() will return the same object or even an
equivalent object, though at least the latter is likely.
Finders Finders
------- -------
Finders are still responsible for creating the loader. That loader will Finders are still responsible for creating the loader. That loader will
now be stored in the module spec returned by ``find_spec()`` rather now be stored in the module spec returned by find_spec() rather
than returned directly. As is currently the case without the PEP, if a than returned directly. As is currently the case without the PEP, if a
loader would be costly to create, that loader can be designed to defer loader would be costly to create, that loader can be designed to defer
the cost until later. the cost until later.
@ -505,16 +697,16 @@ the cost until later.
**PathEntryFinder.find_spec(name)** **PathEntryFinder.find_spec(name)**
Finders will return ModuleSpec objects when ``find_spec()`` is Finders will return ModuleSpec objects when find_spec() is
called. This new method replaces ``find_module()`` and called. This new method replaces find_module() and
``find_loader()`` (in the ``PathEntryFinder`` case). If a loader does find_loader() (in the PathEntryFinder case). If a loader does
not have ``find_spec()``, ``find_module()`` and ``find_loader()`` are not have find_spec(), find_module() and find_loader() are
used instead, for backward-compatibility. used instead, for backward-compatibility.
Adding yet another similar method to loaders is a case of practicality. Adding yet another similar method to loaders is a case of practicality.
``find_module()`` could be changed to return specs instead of loaders. find_module() could be changed to return specs instead of loaders.
This is tempting because the import APIs have suffered enough, This is tempting because the import APIs have suffered enough,
especially considering ``PathEntryFinder.find_loader()`` was just especially considering PathEntryFinder.find_loader() was just
added in Python 3.3. However, the extra complexity and a less-than- added in Python 3.3. However, the extra complexity and a less-than-
explicit method name aren't worth it. explicit method name aren't worth it.
@ -527,6 +719,7 @@ Loaders will have a new method, exec_module(). Its only job
is to "exec" the module and consequently populate the module's is to "exec" the module and consequently populate the module's
namespace. It is not responsible for creating or preparing the module namespace. It is not responsible for creating or preparing the module
object, nor for any cleanup afterward. It has no return value. object, nor for any cleanup afterward. It has no return value.
exec_module() will be used during both loading and reloading.
exec_module() should properly handle the case where it is called more exec_module() should properly handle the case where it is called more
than once. For some kinds of modules this may mean raising ImportError than once. For some kinds of modules this may mean raising ImportError
@ -538,9 +731,9 @@ support in-place reloading.
Loaders may also implement create_module() that will return a Loaders may also implement create_module() that will return a
new module to exec. It may return None to indicate that the default new module to exec. It may return None to indicate that the default
module creation code should be used. One use case for create_module() module creation code should be used. One use case, though atypical, for
is to provide a module that is a subclass of the builtin module type. create_module() is to provide a module that is a subclass of the builtin
Most loaders will not need to implement create_module(), module type. Most loaders will not need to implement create_module(),
create_module() should properly handle the case where it is called more create_module() should properly handle the case where it is called more
than once for the same spec/module. This may include returning None or than once for the same spec/module. This may include returning None or
@ -554,22 +747,22 @@ raising ImportError.
Other changes: Other changes:
PEP 420 introduced the optional ``module_repr()`` loader method to limit PEP 420 introduced the optional module_repr() loader method to limit
the amount of special-casing in the module type's ``__repr__()``. Since the amount of special-casing in the module type's ``__repr__()``. Since
this method is part of ``ModuleSpec``, it will be deprecated on loaders. this method is part of ModuleSpec, it will be deprecated on loaders.
However, if it exists on a loader it will be used exclusively. However, if it exists on a loader it will be used exclusively.
``Loader.init_module_attr()`` method, added prior to Python 3.4's Loader.init_module_attr() method, added prior to Python 3.4's
release , will be removed in favor of the same method on ``ModuleSpec``. release , will be removed in favor of the same method on ModuleSpec.
However, ``InspectLoader.is_package()`` will not be deprecated even However, InspectLoader.is_package() will not be deprecated even
though the same information is found on ``ModuleSpec``. ``ModuleSpec`` though the same information is found on ModuleSpec. ModuleSpec
can use it to populate its own ``is_package`` if that information is can use it to populate its own is_package if that information is
not otherwise available. Still, it will be made optional. not otherwise available. Still, it will be made optional.
One consequence of ModuleSpec is that loader ``__init__`` methods will One consequence of ModuleSpec is that loader ``__init__`` methods will
no longer need to accommodate per-module state. The path-based loaders no longer need to accommodate per-module state. The path-based loaders
in ``importlib`` take arguments in their ``__init__()`` and have in importlib take arguments in their ``__init__()`` and have
corresponding attributes. However, the need for those values is corresponding attributes. However, the need for those values is
eliminated by module specs. eliminated by module specs.
@ -586,10 +779,10 @@ Other Changes
was started. For instance, with ``-m`` the spec's name will be that was started. For instance, with ``-m`` the spec's name will be that
of the run module, while ``__main__.__name__`` will still be of the run module, while ``__main__.__name__`` will still be
"__main__". "__main__".
* We add ``importlib.find_spec()`` to mirror * We add importlib.find_spec() to mirror
``importlib.find_loader()`` (which becomes deprecated). importlib.find_loader() (which becomes deprecated).
* ``importlib.reload()`` is changed to use ``ModuleSpec.load()``. * importlib.reload() is changed to use ModuleSpec.load().
* ``importlib.reload()`` will now make use of the per-module import * importlib.reload() will now make use of the per-module import
lock. lock.
@ -611,22 +804,22 @@ knowledge.
\* Other modules to look at: runpy (and pythonrun.c), pickle, pydoc, \* Other modules to look at: runpy (and pythonrun.c), pickle, pydoc,
inspect. inspect.
For instance, pickle should be updated in the __main__ case to look at For instance, pickle should be updated in the ``__main__`` case to look
``module.__spec__.name``. at ``module.__spec__.name``.
\* Add broader reloading support? See [2]_. \* Impact on some kinds of lazy loading modules. [lazy_import_concerns]_
\* Impact on some kinds of lazy loading modules. See [3]_.
References References
========== ==========
.. [1] http://mail.python.org/pipermail/import-sig/2013-August/000658.html .. [ref_files_pep] http://mail.python.org/pipermail/import-sig/2013-August/000658.html
.. [2] https://mail.python.org/pipermail/import-sig/2013-September/000735.html .. [import_system_docs] http://docs.python.org/3/reference/import.html
.. [3] https://mail.python.org/pipermail/python-dev/2013-August/128129.html .. [cleaner_reload_support] https://mail.python.org/pipermail/import-sig/2013-September/000735.html
.. [lazy_import_concerns] https://mail.python.org/pipermail/python-dev/2013-August/128129.html
Copyright Copyright