188 lines
7.1 KiB
Plaintext
188 lines
7.1 KiB
Plaintext
PEP: 366
|
|
Title: Main module explicit relative imports
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Nick Coghlan <ncoghlan@gmail.com>
|
|
Status: Final
|
|
Type: Standards Track
|
|
Content-Type: text/x-rst
|
|
Created: 1-May-2007
|
|
Python-Version: 2.6
|
|
Post-History: 1-May-2007
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
This PEP proposes a backwards compatible mechanism that permits
|
|
the use of explicit relative imports from executable modules within
|
|
packages. Such imports currently fail due to an awkward interaction
|
|
between PEP 328 and PEP 338 - this behaviour is the subject of at
|
|
least one open SF bug report (#1510172)[1].
|
|
|
|
With the proposed mechanism, relative imports will work automatically
|
|
if the module is executed using the ``-m`` switch. A small amount of
|
|
boilerplate will be needed in the module itself to allow the relative
|
|
imports to work when the file is executed by name.
|
|
|
|
|
|
Import Statements and the Main Module
|
|
=====================================
|
|
|
|
(This section is taken from the final revision of PEP 338)
|
|
|
|
The release of 2.5b1 showed a surprising (although obvious in
|
|
retrospect) interaction between PEP 338 and PEP 328 - explicit
|
|
relative imports don't work from a main module. This is due to
|
|
the fact that relative imports rely on ``__name__`` to determine
|
|
the current module's position in the package hierarchy. In a main
|
|
module, the value of ``__name__`` is always ``'__main__'``, so
|
|
explicit relative imports will always fail (as they only work for
|
|
a module inside a package).
|
|
|
|
Investigation into why implicit relative imports *appear* to work when
|
|
a main module is executed directly but fail when executed using ``-m``
|
|
showed that such imports are actually always treated as absolute
|
|
imports. Because of the way direct execution works, the package
|
|
containing the executed module is added to sys.path, so its sibling
|
|
modules are actually imported as top level modules. This can easily
|
|
lead to multiple copies of the sibling modules in the application if
|
|
implicit relative imports are used in modules that may be directly
|
|
executed (e.g. test modules or utility scripts).
|
|
|
|
For the 2.5 release, the recommendation is to always use absolute
|
|
imports in any module that is intended to be used as a main module.
|
|
The ``-m`` switch already provides a benefit here, as it inserts the
|
|
current directory into ``sys.path``, instead of the directory contain the
|
|
main module. This means that it is possible to run a module from
|
|
inside a package using ``-m`` so long as the current directory contains
|
|
the top level directory for the package. Absolute imports will work
|
|
correctly even if the package isn't installed anywhere else on
|
|
sys.path. If the module is executed directly and uses absolute imports
|
|
to retrieve its sibling modules, then the top level package directory
|
|
needs to be installed somewhere on sys.path (since the current directory
|
|
won't be added automatically).
|
|
|
|
Here's an example file layout::
|
|
|
|
devel/
|
|
pkg/
|
|
__init__.py
|
|
moduleA.py
|
|
moduleB.py
|
|
test/
|
|
__init__.py
|
|
test_A.py
|
|
test_B.py
|
|
|
|
So long as the current directory is ``devel``, or ``devel`` is already
|
|
on ``sys.path`` and the test modules use absolute imports (such as
|
|
``import pkg.moduleA`` to retrieve the module under test, PEP 338
|
|
allows the tests to be run as::
|
|
|
|
python -m pkg.test.test_A
|
|
python -m pkg.test.test_B
|
|
|
|
Rationale for Change
|
|
====================
|
|
|
|
In rejecting PEP 3122 (which proposed a higher impact solution to this
|
|
problem), Guido has indicated that he still isn't particularly keen on
|
|
the idea of executing modules inside packages as scripts [2]. Despite
|
|
these misgivings he has previously approved the addition of the ``-m``
|
|
switch in Python 2.4, and the ``runpy`` module based enhancements
|
|
described in PEP 338 for Python 2.5.
|
|
|
|
The philosophy that motivated those previous additions (i.e. access to
|
|
utility or testing scripts without needing to worry about name clashes in
|
|
either the OS executable namespace or the top level Python namespace) is
|
|
also the motivation behind fixing what I see as a bug in the current
|
|
implementation.
|
|
|
|
This PEP is intended to provide a solution which permits explicit
|
|
relative imports from main modules, without incurring any significant
|
|
costs during interpreter startup or normal module import.
|
|
|
|
|
|
Proposed Solution
|
|
=================
|
|
|
|
The heart of the proposed solution is a new module attribute
|
|
``__package_name__``. This attribute will be defined only in
|
|
the main module (i.e. modules where ``__name__ == "__main__"``).
|
|
|
|
For a directly executed main module, this attribute will be set
|
|
to the empty string. For a module executed using
|
|
``runpy.run_module()`` with the ``run_name`` parameter set to
|
|
``"__main__"``, the attribute will be set to
|
|
``mod_name.rpartition('.')[0]`` (i.e., everything up to
|
|
but not including the last period).
|
|
|
|
In the import machinery there is an error handling path which
|
|
deals with the case where an explicit relative reference attempts
|
|
to go higher than the top level in the package hierarchy. This
|
|
error path would be changed to fall back on the ``__package_name__``
|
|
attribute for explicit relative imports when the importing module
|
|
is called ``"__main__"``.
|
|
|
|
With this change, explicit relative imports will work automatically
|
|
from a script executed with the ``-m`` switch. To allow direct
|
|
execution of the module, the following boilerplate would be needed at
|
|
the top of the script::
|
|
|
|
if __name__ == "__main__" and not __package_name__:
|
|
__package_name__ = "<expected_pkg_name>"
|
|
|
|
Note that this boilerplate has the same disadvantage as the use of
|
|
absolute imports of sibling modules - if the script is moved to a
|
|
different package or subpackage, the boilerplate will need to be
|
|
updated manually.
|
|
|
|
With this feature in place, the test scripts in the package above
|
|
would be able to change their import lines to something along the
|
|
lines of ``import ..moduleA``. The scripts could then be
|
|
executed unmodified even if the name of the package was changed.
|
|
|
|
(Rev 47142 in SVN implemented an early variant of this proposal
|
|
which stored the main module's real module name in the
|
|
'__module_name__' attribute. It was reverted due to the fact
|
|
that 2.5 was already in beta by that time.)
|
|
|
|
|
|
Alternative Proposals
|
|
=====================
|
|
|
|
PEP 3122 proposed addressing this problem by changing the way
|
|
the main module is identified. That's a huge compatibility cost
|
|
to incur to fix something that is a pretty minor bug in the overall
|
|
scheme of things.
|
|
|
|
The advantage of the proposal in this PEP is that its only impact on
|
|
normal code is the tiny amount of time needed at startup to set the extra
|
|
attribute in the main module. The changes to the import machinery are all
|
|
in an existing error handling path, so normal imports don't incur any
|
|
performance penalty at all.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] Absolute/relative import not working?
|
|
(http://www.python.org/sf/1510172)
|
|
|
|
.. [2] Guido's rejection of PEP 3122
|
|
(http://mail.python.org/pipermail/python-3000/2007-April/006793.html)
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
End:
|