298 lines
8.9 KiB
ReStructuredText
298 lines
8.9 KiB
ReStructuredText
PEP: 369
|
||
Title: Post import hooks
|
||
Version: $Revision$
|
||
Last-Modified: $Date$
|
||
Author: Christian Heimes <christian@python.org>
|
||
Status: Withdrawn
|
||
Type: Standards Track
|
||
Content-Type: text/x-rst
|
||
Created: 02-Jan-2008
|
||
Python-Version: 2.6, 3.0
|
||
Post-History: 02-Dec-2012
|
||
|
||
|
||
Withdrawal Notice
|
||
=================
|
||
|
||
This PEP has been withdrawn by its author, as much of the detailed design
|
||
is no longer valid following the migration to importlib in Python 3.3.
|
||
|
||
|
||
Abstract
|
||
========
|
||
|
||
This PEP proposes enhancements for the import machinery to add
|
||
post import hooks. It is intended primarily to support the wider
|
||
use of abstract base classes that is expected in Python 3.0.
|
||
|
||
The PEP originally started as a combined PEP for lazy imports and
|
||
post import hooks. After some discussion on the python-dev mailing
|
||
list the PEP was parted in two separate PEPs. [1]_
|
||
|
||
|
||
Rationale
|
||
=========
|
||
|
||
Python has no API to hook into the import machinery and execute code
|
||
*after* a module is successfully loaded. The import hooks of :pep:`302` are
|
||
about finding modules and loading modules but they were not designed to
|
||
as post import hooks.
|
||
|
||
|
||
Use cases
|
||
=========
|
||
|
||
A use case for a post import hook is mentioned in Alyssa (Nick) Coghlan's initial
|
||
posting [2]_. about callbacks on module import. It was found during the
|
||
development of Python 3.0 and its ABCs. We wanted to register classes
|
||
like decimal.Decimal with an ABC but the module should not be imported
|
||
on every interpreter startup. Alyssa came up with this example::
|
||
|
||
@imp.when_imported('decimal')
|
||
def register(decimal):
|
||
Inexact.register(decimal.Decimal)
|
||
|
||
The function ``register`` is registered as callback for the module named
|
||
'decimal'. When decimal is imported the function is called with the
|
||
module object as argument.
|
||
|
||
While this particular example isn't necessary in practice, (as
|
||
decimal.Decimal will inherit from the appropriate abstract Number base
|
||
class in 2.6 and 3.0), it still illustrates the principle.
|
||
|
||
|
||
Existing implementations
|
||
========================
|
||
|
||
PJE's peak.util.imports [3]_ implements post load hooks. My
|
||
implementation shares a lot with his and it's partly based on his ideas.
|
||
|
||
|
||
Post import hook implementation
|
||
===============================
|
||
|
||
Post import hooks are called after a module has been loaded. The hooks
|
||
are callable which take one argument, the module instance. They are
|
||
registered by the dotted name of the module, e.g. 'os' or 'os.path'.
|
||
|
||
The callable are stored in the dict ``sys.post_import_hooks`` which
|
||
is a mapping from names (as string) to a list of callables or None.
|
||
|
||
|
||
States
|
||
------
|
||
|
||
No hook was registered
|
||
''''''''''''''''''''''
|
||
|
||
sys.post_import_hooks contains no entry for the module
|
||
|
||
|
||
A hook is registered and the module is not loaded yet
|
||
'''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||
|
||
The import hook registry contains an entry
|
||
sys.post_import_hooks["name"] = [hook1]
|
||
|
||
|
||
A module is successfully loaded
|
||
'''''''''''''''''''''''''''''''
|
||
|
||
The import machinery checks if sys.post_import_hooks contains post import
|
||
hooks for the newly loaded module. If hooks are found then the hooks are
|
||
called in the order they were registered with the module instance as first
|
||
argument. The processing of the hooks is stopped when a method raises an
|
||
exception. At the end the entry for the module name set to None, even
|
||
when an error has occurred.
|
||
|
||
Additionally the new ``__notified__`` slot of the module object is set
|
||
to ``True`` in order to prevent infinity recursions when the notification
|
||
method is called inside a hook. For object which don't subclass from
|
||
``PyModule`` a new attribute is added instead.
|
||
|
||
|
||
A module can't be loaded
|
||
''''''''''''''''''''''''
|
||
|
||
The import hooks are neither called nor removed from the registry. It
|
||
may be possible to load the module later.
|
||
|
||
|
||
A hook is registered but the module is already loaded
|
||
'''''''''''''''''''''''''''''''''''''''''''''''''''''
|
||
|
||
The hook is fired immediately.
|
||
|
||
|
||
Invariants
|
||
----------
|
||
|
||
The import hook system guarantees certain invariants. XXX
|
||
|
||
|
||
Sample Python implementation
|
||
----------------------------
|
||
|
||
A Python implementation may look like::
|
||
|
||
def notify(name):
|
||
try:
|
||
module = sys.modules[name]
|
||
except KeyError:
|
||
raise ImportError("Module %s has not been imported" % (name,))
|
||
if module.__notified__:
|
||
return
|
||
try:
|
||
module.__notified__ = True
|
||
if '.' in name:
|
||
notify(name[:name.rfind('.')])
|
||
for callback in post_import_hooks[name]:
|
||
callback(module)
|
||
finally:
|
||
post_import_hooks[name] = None
|
||
|
||
XXX
|
||
|
||
|
||
C API
|
||
-----
|
||
|
||
New C API functions
|
||
'''''''''''''''''''
|
||
|
||
``PyObject* PyImport_GetPostImportHooks(void)``
|
||
Returns the dict sys.post_import_hooks or NULL
|
||
|
||
``PyObject* PyImport_NotifyLoadedByModule(PyObject *module)``
|
||
Notify the post import system that a module was requested. Returns the
|
||
a borrowed reference to the same module object or NULL if an error has
|
||
occurred. The function calls only the hooks for the module itself and not
|
||
its parents. The function must be called with the import lock acquired.
|
||
|
||
``PyObject* PyImport_NotifyLoadedByName(const char *name)``
|
||
``PyImport_NotifyLoadedByName("a.b.c")`` calls
|
||
``PyImport_NotifyLoadedByModule()`` for ``a``, ``a.b`` and ``a.b.c``
|
||
in that particular order. The modules are retrieved from
|
||
``sys.modules``. If a module can't be retrieved, an exception is raised
|
||
otherwise the a borrowed reference to ``modname`` is returned.
|
||
The hook calls always start with the prime parent module.
|
||
The caller of PyImport_NotifyLoadedByName() must hold the import lock!
|
||
|
||
``PyObject* PyImport_RegisterPostImportHook(PyObject *callable, PyObject *mod_name)``
|
||
Register a new hook ``callable`` for the module ``mod_name``
|
||
|
||
``int PyModule_GetNotified(PyObject *module)``
|
||
Returns the status of the ``__notified__`` slot / attribute.
|
||
|
||
``int PyModule_SetNotified(PyObject *module, int status)``
|
||
Set the status of the ``__notified__`` slot / attribute.
|
||
|
||
|
||
The ``PyImport_NotifyLoadedByModule()`` method is called inside
|
||
``import_submodule()``. The import system makes sure that the import lock
|
||
is acquired and the hooks for the parent modules are already called.
|
||
|
||
|
||
Python API
|
||
----------
|
||
|
||
The import hook registry and two new API methods are exposed through the
|
||
``sys`` and ``imp`` module.
|
||
|
||
``sys.post_import_hooks``
|
||
The dict contains the post import hooks::
|
||
|
||
{"name" : [hook1, hook2], ...}
|
||
|
||
``imp.register_post_import_hook(hook: "callable", name: str)``
|
||
Register a new hook *hook* for the module *name*
|
||
|
||
``imp.notify_module_loaded(module: "module instance") -> module``
|
||
Notify the system that a module has been loaded. The method is provided
|
||
for compatibility with existing lazy / deferred import extensions.
|
||
|
||
``module.__notified__``
|
||
A slot of a module instance. XXX
|
||
|
||
The when_imported function decorator is also in the imp module,
|
||
which is equivalent to::
|
||
|
||
def when_imported(name):
|
||
def register(hook):
|
||
register_post_import_hook(hook, name)
|
||
return register
|
||
|
||
imp.when_imported(name) -> decorator function
|
||
for @when_imported(name) def hook(module): pass
|
||
|
||
|
||
Open issues
|
||
===========
|
||
|
||
The when_imported decorator hasn't been written.
|
||
|
||
The code contains several XXX comments. They are mostly about error
|
||
handling in edge cases.
|
||
|
||
|
||
Backwards Compatibility
|
||
=======================
|
||
|
||
The new features and API don't conflict with old import system of Python
|
||
and don't cause any backward compatibility issues for most software.
|
||
However systems like PEAK and Zope which implement their own lazy import
|
||
magic need to follow some rules.
|
||
|
||
The post import hooks carefully designed to cooperate with existing
|
||
deferred and lazy import systems. It's the suggestion of the PEP author
|
||
to replace own on-load-hooks with the new hook API. The alternative
|
||
lazy or deferred imports will still work but the implementations must
|
||
call the ``imp.notify_module_loaded`` function.
|
||
|
||
|
||
Reference Implementation
|
||
========================
|
||
|
||
A reference implementation is already written and is available in the
|
||
*py3k-importhook* branch. [4]_ It still requires some cleanups,
|
||
documentation updates and additional unit tests.
|
||
|
||
|
||
Acknowledgments
|
||
===============
|
||
|
||
Alyssa Coghlan, for proof reading and the initial discussion
|
||
Phillip J. Eby, for his implementation in PEAK and help with my own implementation
|
||
|
||
|
||
Copyright
|
||
=========
|
||
|
||
This document has been placed in the public domain.
|
||
|
||
|
||
References
|
||
==========
|
||
|
||
.. [1] PEP: Lazy module imports and post import hook
|
||
http://permalink.gmane.org/gmane.comp.python.devel/90949
|
||
|
||
.. [2] Interest in PEP for callbacks on module import
|
||
http://permalink.gmane.org/gmane.comp.python.python-3000.devel/11126
|
||
|
||
.. [3] peak.utils.imports
|
||
http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup
|
||
|
||
.. [4] py3k-importhook branch
|
||
http://svn.python.org/view/python/branches/py3k-importhook/
|
||
|
||
|
||
..
|
||
Local Variables:
|
||
mode: indented-text
|
||
indent-tabs-mode: nil
|
||
sentence-end-double-space: t
|
||
fill-column: 70
|
||
coding: utf-8
|
||
End:
|