iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Major rewrite to simplify things and to incorporate python-dev feedback

This commit is contained in:
Nick Coghlan 2007-07-08 07:45:46 +00:00
parent c1e4e15c12
commit b345a5c460
1 changed files with 65 additions and 127 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

192
pep-0366.txt
View File

@ -7,8 +7,8 @@ Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 1-May-2007
Python-Version: 2.6
Post-History: 1-May-2007
Python-Version: 2.6, 3.0
Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007
Abstract
@ -17,158 +17,95 @@ Abstract
This PEP proposes a backwards compatible mechanism that permits
the use of explicit relative imports from executable modules within
packages. Such imports currently fail due to an awkward interaction
between PEP 328 and PEP 338 - this behaviour is the subject of at
least one open SF bug report (#1510172)[1], and has most likely
been a factor in at least a few queries on comp.lang.python (such
as Alan Isaac's question in [2]).
between PEP 328 and PEP 338.
With the proposed mechanism, relative imports will work automatically
if the module is executed using the ``-m`` switch. A small amount of
boilerplate will be needed in the module itself to allow the relative
By adding a new module level attribute, this PEP allows relative imports
to work automatically if the module is executed using the ``-m``switch.
A small amount of boilerplate in the module itself will allow the relative
imports to work when the file is executed by name.
Import Statements and the Main Module
=====================================
Proposed Change
===============
(This section is taken from the final revision of PEP 338)
The major proposed change is the introduction of a new module level
attribute, ``__package__``. When it is present, relative imports will
be based on this attribute rather than the module ``__name__``
attribute.
The release of 2.5b1 showed a surprising (although obvious in
retrospect) interaction between PEP 338 and PEP 328 - explicit
relative imports don't work from a main module. This is due to
the fact that relative imports rely on ``__name__`` to determine
the current module's position in the package hierarchy. In a main
module, the value of ``__name__`` is always ``'__main__'``, so
explicit relative imports will always fail (as they only work for
a module inside a package).
As with the current ``__name__`` attribute, setting ``__package__`` will
be the responsibility of the PEP 302 loader used to import a module.
Loaders which use ``imp.new_module()`` to create the module object will
have the new attribute set automatically to
``__name__.rpartition('.')[0]``.
Investigation into why implicit relative imports *appear* to work when
a main module is executed directly but fail when executed using ``-m``
showed that such imports are actually always treated as absolute
imports. Because of the way direct execution works, the package
containing the executed module is added to sys.path, so its sibling
modules are actually imported as top level modules. This can easily
lead to multiple copies of the sibling modules in the application if
implicit relative imports are used in modules that may be directly
executed (e.g. test modules or utility scripts).
``runpy.run_module`` will also set the new attribute, basing it off the
``mod_name`` argument, rather than the ``run_name`` argument. This will
allow relative imports to work correctly from main modules executed with
the ``-m`` switch.
For the 2.5 release, the recommendation is to always use absolute
imports in any module that is intended to be used as a main module.
The ``-m`` switch already provides a benefit here, as it inserts the
current directory into ``sys.path``, instead of the directory containing
the main module. This means that it is possible to run a module from
inside a package using ``-m`` so long as the current directory contains
the top level directory for the package. Absolute imports will work
correctly even if the package isn't installed anywhere else on
sys.path. If the module is executed directly and uses absolute imports
to retrieve its sibling modules, then the top level package directory
needs to be installed somewhere on sys.path (since the current directory
won't be added automatically).
Here's an example file layout::
devel/
pkg/
__init__.py
moduleA.py
moduleB.py
test/
__init__.py
test_A.py
test_B.py
So long as the current directory is ``devel``, or ``devel`` is already
on ``sys.path`` and the test modules use absolute imports (such as
``import pkg.moduleA`` to retrieve the module under test, PEP 338
allows the tests to be run as::
python -m pkg.test.test_A
python -m pkg.test.test_B
Rationale for Change
====================
In rejecting PEP 3122 (which proposed a higher impact solution to this
problem), Guido has indicated that he still isn't particularly keen on
the idea of executing modules inside packages as scripts [2]. Despite
these misgivings he has previously approved the addition of the ``-m``
switch in Python 2.4, and the ``runpy`` module based enhancements
described in PEP 338 for Python 2.5.
The philosophy that motivated those previous additions (i.e. access to
utility or testing scripts without needing to worry about name clashes in
either the OS executable namespace or the top level Python namespace) is
also the motivation behind fixing what I see as a bug in the current
implementation.
This PEP is intended to provide a solution which permits explicit
relative imports from main modules, without incurring any significant
costs during interpreter startup or normal module import.
Proposed Solution
=================
The heart of the proposed solution is a new module attribute
``__package_name__``. This attribute will be defined only in
the main module (i.e. modules where ``__name__ == "__main__"``).
For a directly executed main module, this attribute will be set
to the empty string. For a module executed using
``runpy.run_module()`` with the ``run_name`` parameter set to
``"__main__"``, the attribute will be set to
``mod_name.rpartition('.')[0]`` (i.e., everything up to
but not including the last period).
In the import machinery there is an error handling path which
deals with the case where an explicit relative reference attempts
to go higher than the top level in the package hierarchy. This
error path would be changed to fall back on the ``__package_name__``
attribute for explicit relative imports when the importing module
is called ``"__main__"``.
With this change, explicit relative imports will work automatically
from a script executed with the ``-m`` switch. To allow direct
execution of the module, the following boilerplate would be needed at
the top of the script::
When the main module is specified by its filename, then the
``__package__`` attribute will be set to the empty string. To allow
relative imports when the module is executed directly, boilerplate
similar to the following would be needed before the first relative
import statement:
if __name__ == "__main__" and not __package_name__:
__package_name__ = "<expected_pkg_name>"
__package_name__ = "<expected_pacakage_name>"
Note that this boilerplate is sufficient only if the top level package
is already accessible via sys.path. Additional code that manipulates
sys.path would be needed in order for direct execution to work
without the top level package already being on sys.path.
is already accessible via ``sys.path``. Additional code that manipulates
``sys.path`` would be needed in order for direct execution to work
without the top level package already being importable.
This approach also has the same disadvantage as the use of absolute
imports of sibling modules - if the script is moved to a different
package or subpackage, the boilerplate will need to be updated
manually.
With this feature in place, the test scripts in the package above
would be able to change their import lines to something along the
lines of ``import ..moduleA``. The scripts could then be
executed unmodified even if the name of the package was changed.
(Rev 47142 in SVN implemented an early variant of this proposal
Rationale for Change
====================
The current inability to use explicit relative imports from the main
module is the subject of at least one open SF bug report (#1510172)[1],
and has most likely been a factor in at least a few queries on
comp.lang.python (such as Alan Isaac's question in [2]).
This PEP is intended to provide a solution which permits explicit
relative imports from main modules, without incurring any significant
costs during interpreter startup or normal module import.
The section in PEP 338 on relative imports and the main module provides
further details and background on this problem.
Reference Implementation
========================
Rev 47142 in SVN implemented an early variant of this proposal
which stored the main module's real module name in the
'__module_name__' attribute. It was reverted due to the fact
that 2.5 was already in beta by that time.)
that 2.5 was already in beta by that time.
A new patch will be developed for 2.6, and forward ported to
Py3k via svnmerge.
Alternative Proposals
=====================
PEP 3122 proposed addressing this problem by changing the way
the main module is identified. That's a huge compatibility cost
the main module is identified. That's a significant compatibility cost
to incur to fix something that is a pretty minor bug in the overall
scheme of things.
scheme of things, and the PEP was rejected [3].
The advantage of the proposal in this PEP is that its only impact on
normal code is the tiny amount of time needed at startup to set the extra
attribute in the main module. The changes to the import machinery are all
in an existing error handling path, so normal imports don't incur any
performance penalty at all.
normal code is the small amount of time needed to set the extra
attribute when importing a module. Relative imports themselves should
be sped up fractionally, as the package name is stored in the module
globals, rather than having to be worked out again for each relative
import.
References
@ -177,11 +114,12 @@ References
.. [1] Absolute/relative import not working?
(http://www.python.org/sf/1510172)
.. [2] Guido's rejection of PEP 3122
.. [2] c.l.p. question about modules and relative imports
(http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/)
.. [3] Guido's rejection of PEP 3122
(http://mail.python.org/pipermail/python-3000/2007-April/006793.html)
.. [3] c.l.p. question about modules and relative imports
(http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/)
Copyright
=========