iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[PEP 451] Update the signature of find_spec() and remove supports_reload().

This commit is contained in:
Eric Snow 2013-11-01 16:19:06 -06:00
parent 3e087d47c8
commit 23dc54157b
1 changed files with 156 additions and 82 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

238
pep-0451.txt
View File

@ -278,9 +278,9 @@ finders. See the `Factory Functions`_ section below for more detail.
Other API Additions
-------------------
* importlib.find_spec(name, path=None) will work exactly the same as
importlib.find_loader() (which it replaces), but return a spec instead
of a loader.
* importlib.find_spec(name, path=None, existing=None) will work exactly
the same as importlib.find_loader() (which it replaces), but return a
spec instead of a loader.
For finders:
@ -295,8 +295,6 @@ For loaders:
over its module execution functionality.
* importlib.abc.Loader.create_module(spec) (optional) will return the
module to use for loading.
* importlib.abc.Loader.supports_reload(name) (optional) will return True
(the default) if the loader supports reloading the module.
For modules:
@ -374,13 +372,13 @@ Here's a quick breakdown of where responsibilities lie after this PEP.
finders:
* create loader
* create spec
* create/identify a loader that can load the module.
* create the spec for the module.
loaders:
* create module (optional)
* execute module
* create the module (optional).
* execute the module.
ModuleSpec:
@ -404,11 +402,9 @@ finders and loaders should change relative to this PEP:
* Implement exec_module() on loaders, if possible.
The ModuleSpec factory functions in importlib.util are intended to be
helpful for converting existing finders. from_loader() and
from_file_location() are both straight-forward utilities in this
regard. In the case where loaders already expose methods for creating
and preparing modules, ModuleSpec.from_module() may be useful to
the corresponding finder.
helpful for converting existing finders. spec_from_loader() and
spec_from_file_location() are both straight-forward utilities in this
regard.
For existing loaders, exec_module() should be a relatively direct
conversion from the non-boilerplate portion of load_module(). In some
@ -497,7 +493,7 @@ Here is the corresponding outline for reload()::
name = module.__spec__.name
except AttributeError:
name = module.__name__
spec = find_spec(name)
spec = find_spec(name, existing=module)
if sys.modules.get(name) is not module:
raise ImportError
@ -509,8 +505,6 @@ Here is the corresponding outline for reload()::
# namespace loader
_init_module_attrs(spec, module)
return module
if not spec.loader.supports_reload(name):
raise ImportError
if spec.parent and spec.parent not in sys.modules:
raise ImportError
@ -520,6 +514,19 @@ Here is the corresponding outline for reload()::
finally:
del _RELOADING[name]
A key point here is the switch to Loader.exec_module() means that
loaders will no longer have an easy way to know at execution time if it
is a reload or not. Before this proposal, they could simply check to
see if the module was already in sys.modules. Now, by the time
exec_module() is called during load (not reload) the import machinery
would already have placed the module in sys.modules. This is part of
the reason why find_spec() has
`the "existing" parameter <The "existing" parameter of find_spec()>`_.
The semantics of reload will remain essentially the same as they exist
already [reload-semantics-fix]_. The impact of this PEP on some kinds
of lazy loading modules was a point of discussion. [lazy_import_concerns]_
ModuleSpec
==========
@ -627,7 +634,7 @@ Build a spec from file-oriented information and loader APIs.
* "submodule_search_locations" can be deduced from loader.is_package()
and from os.path.dirname(location) if location is a filename.
**from_loader(name, loader, \*, origin=None, is_package=None)**
**spec_from_loader(name, loader, \*, origin=None, is_package=None)**
Build a spec with missing information filled in by using loader APIs.
@ -636,45 +643,6 @@ Build a spec with missing information filled in by using loader APIs.
* "submodule_search_locations" can be deduced from loader.is_package()
and from os.path.dirname(location) if location is a filename.
**spec_from_module(module, loader=None)**
Build a spec based on the import-related attributes of an existing
module. The spec attributes are set to the corresponding import-
related module attributes. See the table in `Attributes`_.
Omitted Attributes and Methods
------------------------------
There is no "PathModuleSpec" subclass of ModuleSpec that separates out
has_location, cached, and submodule_search_locations. While that might
make the separation cleaner, module objects don't have that distinction.
ModuleSpec will support both cases equally well.
While "is_package" would be a simple additional attribute (aliasing
self.submodule_search_locations is not None), it perpetuates the
artificial (and mostly erroneous) distinction between modules and
packages.
Conceivably, a ModuleSpec.load() method could optionally take a list of
modules with which to interact instead of sys.modules. That
capability is left out of this PEP, but may be pursued separately at
some other time, including relative to PEP 406 (import engine).
Likewise load() could be leveraged to implement multi-version
imports. While interesting, doing so is outside the scope of this
proposal.
Others:
* Add ModuleSpec.submodules (RO-property) - returns possible submodules
relative to the spec.
* Add ModuleSpec.loaded (RO-property) - the module in sys.module, if
any.
* Add ModuleSpec.data - a descriptor that wraps the data API of the
spec's loader.
* Also see [cleaner_reload_support]_.
Backward Compatibility
----------------------
@ -722,15 +690,16 @@ equivalent object, though at least the latter is likely.
Finders
-------
Finders are still responsible for creating the loader. That loader will
Finders are still responsible for identifying, an typically creating,
the loader that should be used to load a module. That loader will
now be stored in the module spec returned by find_spec() rather
than returned directly. As is currently the case without the PEP, if a
loader would be costly to create, that loader can be designed to defer
the cost until later.
**MetaPathFinder.find_spec(name, path=None)**
**MetaPathFinder.find_spec(name, path=None, existing=None)**
**PathEntryFinder.find_spec(name)**
**PathEntryFinder.find_spec(name, existing=None)**
Finders must return ModuleSpec objects when find_spec() is
called. This new method replaces find_module() and
@ -745,6 +714,42 @@ especially considering PathEntryFinder.find_loader() was just
added in Python 3.3. However, the extra complexity and a less-than-
explicit method name aren't worth it.
The "existing" parameter of find_spec()
---------------------------------------
A module object with the same name as the "name" argument (or None, the
default) should be passed in to "exising". This argument allows the
finder to build the module spec with more information than is otherwise
available. This is particularly relevant in identifying the loader to
use.
Through find_spec() the finder will always identify the loader it
will return in the spec. In the case of reload, at this point the
finder should also decide whether or not the loader supports loading
into the module-to-be-reloaded (which was passed in to find_spec() as
"existing"). This decision may entail consulting with the loader. If
the finder determines that the loader does not support reloading that
module, it should either find another loader or return None (indicating
that it could not "find" the module). This reload decision is important
since, as noted in `How Reloading Will Work`_, loaders will no longer be
able to trivially identify a reload situation on their own.
Two alternatives were presented to the "existing" parameter:
Loader.supports_reload() and adding "existing" to Loader.exec_module()
instead of find_spec(). supports_reload() was the initial approach to
the reload situation. [supports_reload]_ However, there was some
opposition to the loader-specific, reload-centric approach.
[supports_reload_considered_harmful]_
As to "existing" on exec_module(), the loader may need other information
from the existing module (or spec) during reload, more than just "does
this loader support reloading this module", that is no longer available
with the move away from load_module(). A proposal on the table was to
add something like "existing" to exec_module(). [exec_module_existing]_
However, putting "existing" on find_spec() instead is more in line with
the goals of this PEP. Furthermore, it obviates the need for
supports_reload().
Namespace Packages
------------------
@ -791,13 +796,6 @@ raising ImportError.
module attributes. The fact that load_module() does is a design flaw
that this proposal aims to correct.
**Loader.supports_reload(name)**
In cases where a module should not be reloaded, Loaders should implement
supports_reload() and have it return False. If the method is defined
and returns a false value, importlib.reload() will raise an ImportError.
Otherwise reloading proceeds as normal.
Other changes:
PEP 420 introduced the optional module_repr() loader method to limit
@ -837,24 +835,27 @@ Other Changes
* importlib.reload() will now make use of the per-module import lock.
Open Issues
===========
* In the `Finders`_ section, the PEP specifies returning None (or using
a different loader) when the found loader does not support loading into
an existing module (e.g during reload). An alternative to returning
None would be to raise ImportError with a message like "the loader does
not support reloading the module". This may actually be a better
approach since "could not find a loader" and "the found loader won't
work" are different situations that a single return value (None) may not
sufficiently represent.
Reference Implementation
========================
A reference implementation will be available at
A reference implementation is available at
http://bugs.python.org/issue18864.
Open Issues
==============
\* Impact on some kinds of lazy loading modules. [lazy_import_concerns]_
This should not be an issue since the PEP does not change the semantics
of this behavior.
Implementation Notes
====================
--------------------
\* The implementation of this PEP needs to be cognizant of its impact on
pkgutil (and setuptools). pkgutil has some generic function-based
@ -868,17 +869,90 @@ For instance, pickle should be updated in the ``__main__`` case to look
at ``module.__spec__.name``.
Rejected Additions to the PEP
=============================
There were a few proposed additions to this proposal that did not fit
well enough into its scope.
There is no "PathModuleSpec" subclass of ModuleSpec that separates out
has_location, cached, and submodule_search_locations. While that might
make the separation cleaner, module objects don't have that distinction.
ModuleSpec will support both cases equally well.
While "ModuleSpec.is_package" would be a simple additional attribute
(aliasing self.submodule_search_locations is not None), it perpetuates
the artificial (and mostly erroneous) distinction between modules and
packages.
Others left out:
* Add ModuleSpec.submodules (RO-property) - returns possible submodules
relative to the spec.
* Add ModuleSpec.loaded (RO-property) - the module in sys.module, if
any.
* Add ModuleSpec.data - a descriptor that wraps the data API of the
spec's loader.
* Also see [cleaner_reload_support]_.
The module spec `Factory Functions`_ could be classmethods on
ModuleSpec. However that would expose them on *all* modules via
``__spec__``, which has the potential to unnecessarily confuse
non-advanced Python users. The factory functions have a specific use
case, to support finder authors. See `ModuleSpec Users`_.
Likewise, several other methods could be added to ModuleSpec that expose
the specific uses of module specs by the import machinery:
* create() - a wrapper around Loader.create_module().
* exec(module) - a wrapper around Loader.exec_module().
* load() - an analogue to the deprecated Loader.load_module().
As with the factory functions, exposing these methods via
module.__spec__ is less than desireable. They would end up being an
attractive nuisance, even if only exposed as "private" attributes (as
they were in previous versions of this PEP). If someone finds a need
for these methods later, we can expose the via an appropriate API
(separate from ModuleSpec) at that point, perhaps relative to PEP 406
(import engine).
Conceivably, the load() method could optionally take a list of
modules with which to interact instead of sys.modules. Also, load()
could be leveraged to implement multi-version imports. Both are
interesting ideas, but definitely outside the scope of this proposal.
Others left out:
* Add ModuleSpec.submodules (RO-property) - returns possible submodules
relative to the spec.
* Add ModuleSpec.loaded (RO-property) - the module in sys.module, if
any.
* Add ModuleSpec.data - a descriptor that wraps the data API of the
spec's loader.
* Also see [cleaner_reload_support]_.
References
==========
.. [ref_files_pep] http://mail.python.org/pipermail/import-sig/2013-August/000658.html
.. [ref_files_pep]
http://mail.python.org/pipermail/import-sig/2013-August/000658.html
.. [import_system_docs] http://docs.python.org/3/reference/import.html
.. [cleaner_reload_support] https://mail.python.org/pipermail/import-sig/2013-September/000735.html
.. [cleaner_reload_support]
https://mail.python.org/pipermail/import-sig/2013-September/000735.html
.. [lazy_import_concerns] https://mail.python.org/pipermail/python-dev/2013-August/128129.html
.. [lazy_import_concerns]
https://mail.python.org/pipermail/python-dev/2013-August/128129.html
.. [reload-semantics-fix] http://bugs.python.org/issue19413
.. [supports_reload_considered_harmful]
https://mail.python.org/pipermail/python-dev/2013-October/129971.html
.. [exec_module_existing]
https://mail.python.org/pipermail/python-dev/2013-October/129933.html
Copyright
=========