iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Added PEP 369 and PEP 370

This commit is contained in:
Christian Heimes 2008-01-14 20:42:39 +00:00
parent f4064feec0
commit da08263556
3 changed files with 469 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
pep-0000.txt
View File

@ -94,6 +94,8 @@ Index by Category
S 364 Transitioning to the Py3K Standard Library Warsaw
S 365 Adding the pkg_resources module Eby
S 368 Standard image protocol and class Mastrodomenico
S 369 Post import hooks Heimes
S 370 Per user site-packages directory Heimes
S 3108 Standard Library Reorganization Cannon
S 3134 Exception Chaining and Embedded Tracebacks Yee
S 3135 New Super Spealman, Delaney
@ -467,6 +469,8 @@ Numerical Index
SF 366 Main module explicit relative imports Coghlan
SR 367 New Super Spealman, Delaney
S 368 Standard image protocol and class Mastrodomenico
S 369 Post import hooks Heimes
S 370 Per user site-packages directory Heimes
SR 666 Reject Foolish Indentation Creighton
SR 754 IEEE 754 Floating Point Special Values Warnes
P 3000 Python 3000 GvR
@ -571,6 +575,7 @@ Owners
Griffin, Grant g2@iowegian.com
Hammond, Mark mhammond@skippinet.com.au
Harris, Peter scav@blueyonder.co.uk
Heimes, Christian christian@cheimes.de
Heller, Thomas theller@python.net
Hetland, Magnus Lie magnus@hetland.org
Hettinger, Raymond python@rcn.com

235
pep-0369.txt Normal file
View File

@ -0,0 +1,235 @@
PEP: 369
Title: Post import hooks
Version: $Revision$
Last-Modified: $Date$
Author: Christian Heimes <christian(at)cheimes(dot)de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 02-Jan-2008
Python-Version: 2.6, 3.0
Post-History:
Abstract
========
This PEP proposes enhancements for the import machinery to add
post import hooks. It is intended primarily to support the wider
use of abstract base classes that is expected in Python 3.0.
The PEP originally started as a combined PEP for lazy imports and
post import hooks. After some discussion on the python-dev mailing
list the PEP was parted in two separate PEPs. [1]_
Rationale
=========
Python has no API to hook into the import machinery and execute code
*after* a module is successfully loaded. The import hooks of PEP 302 are
about finding modules and loading modules but they were not designed to
as post import hooks.
Use cases
=========
A use case for a post import hook is mentioned in Nick Coghlan's initial
posting [2]_. about callbacks on module import. It was found during the
development of Python 3.0 and its ABCs. We wanted to register classes
like decimal.Decimal with an ABC but the module should not be imported
on every interpreter startup. Nick came up with this example::
@imp.when_imported('decimal')
def register(decimal):
Inexact.register(decimal.Decimal)
The function ``register`` is registered as callback for the module named
'decimal'. When decimal is imported the function is called with the
module object as argument.
While this particular example isn't necessary in practice, (as
decimal.Decimal will inherit from the appropriate abstract Number base
class in 2.6 and 3.0), it still illustrates the principle.
Existing implementations
========================
PJE's peak.util.imports [3]_ implements post load hooks. My
implementation shares a lot with his and it's partly based on his ideas.
Post import hook implementation
===============================
Post import hooks are called after a module has been loaded. The hooks
are callable which take one argument, the module instance. They are
registered by the dotted name of the module, e.g. 'os' or 'os.path'.
The callable are stored in the dict ``sys.post_import_hooks`` which
is a mapping from names (as string) to a list of callables or None.
States
------
No hook was registered
''''''''''''''''''''''
sys.post_import_hooks contains no entry for the module
A hook is registered and the module is not loaded yet
'''''''''''''''''''''''''''''''''''''''''''''''''''''
The import hook registry contains an entry
sys.post_import_hooks["name"] = [hook1]
A module is successfully loaded
'''''''''''''''''''''''''''''''
The import machinery checks if sys.post_import_hooks contains post import
hooks for the newly loaded module. If hooks are found then the hooks are
called in the order they were registered with the module instance as first
argument. The processing of the hooks is stopped when a method raises an
exception. At the end the entry for the module name is removed from
sys.post_import_hooks, even when an error has occured.
A module can't be loaded
''''''''''''''''''''''''
The import hooks are neither called nor removed from the registry. It
may be possible to load the module later.
A hook is registered but the module is already loaded
'''''''''''''''''''''''''''''''''''''''''''''''''''''
The hook is fired immediately.
C API
-----
New PyImport_* API functions
''''''''''''''''''''''''''''
``PyObject* PyImport_GetPostImportHooks(void)``
Returns the dict sys.post_import_hooks or NULL
``PyObject* PyImport_NotifyModuleLoaded(PyObject *module)``
Notify the post import system that a module was requested. Returns the
module or NULL if an error has occured.
``PyObject* PyImport_RegisterPostImportHook(PyObject *callable, PyObject *mod_name)``
Register a new hook ``callable`` for the module ``mod_name``
The ``PyImport_PostImportNotify()`` method is called by
``PyImport_ImportModuleLevel()``::
PyImport_ImportModuleLevel(...)
{
...
result = import_module_level(name, globals, locals, fromlist, level);
result = PyImport_PostImportNotify(result);
...
}
Python API
----------
The import hook registry and two new API methods are exposed through the
``sys`` and ``imp`` module.
sys.post_import_hooks
The dict contains the post import hooks: {"name" : [hook1, hook2], ...}
imp.register_post_import_hook(hook: "callable", name: str)
Register a new hook *hook* for the module *name*
imp.notify_module_loaded(module: "module instance") -> module
Notify the system that a module has been loaded. The method is provided
for compatibility with existing lazy / deferred import extensions.
The when_imported function decorator is also in the imp module,
which is equivalent to::
def when_imported(name):
def register(hook):
register_post_import_hook(hook, name)
return register
imp.when_imported(name) -> decorator function
for @when_imported(name) def hook(module): pass
Open issues
===========
The when_imported decorator hasn't been written.
The code contains several XXX comments. They are mostly about error
handling in edge cases.
Backwards Compatibility
=======================
The new features and API don't conflict with old import system of Python
and don't cause any backward compatibility issues for most software.
However systems like PEAK and Zope which implement their own lazy import
magic need to follow some rules.
The post import hooks carefully designed to cooperate with existing
deferred and lazy import systems. It's the suggestion of the PEP author
to replace own on-load-hooks with the new hook API. The alternative
lazy or deferred imports will still work but the implementations must
call the ``imp.notify_module_loaded`` function.
Reference Implementation
========================
A reference implementation is already written and is available in the
*py3k-importhook* branch. [4]_ It still requires some cleanups,
documentation updates and additional unit tests.
Acknowledgments
===============
Nick Coghlan, for proof reading and the initial discussion
Phillip J. Eby, for his implementation in PEAK and help with my own implementation
Copyright
=========
This document has been placed in the public domain.
References
==========
.. [1] PEP: Lazy module imports and post import hook
http://permalink.gmane.org/gmane.comp.python.devel/90949
.. [2] Interest in PEP for callbacks on module import
http://permalink.gmane.org/gmane.comp.python.python-3000.devel/11126
.. [3] peak.utils.imports
http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup
.. [4] py3k-importhook branch
http://svn.python.org/view/python/branches/py3k-importhook/
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:

229
pep-0370.txt Normal file
View File

@ -0,0 +1,229 @@
PEP: 370
Title: Per user site-packages directory
Version: $Revision$
Last-Modified: $Date$
Author: Christian Heimes <christian(at)cheimes(dot)de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 11-Jan-2008
Python-Version: 2.6, 3.0
Post-History:
Abstract
========
This PEP proposes a new a per user site-packages directory to allow
users the local installation of Python packages in their home directory.
Rationale
=========
Current Python versions don't have an unified way to install packages
into the home directory of an user (except for Mac Framework
builds). Users are either forced to ask the system administrator to
install or update a package for them or to use one of the many
workaround like Virtual Python [1]_, Working Env [2]_ or
Virtual Env [3]_.
It's not the goal of the PEP to replace the tools or to implement
isolated installations of Python. It only implements the most common
use case of an additional site-packages directory for each user.
The feature can't be implemented using the environment variable
*PYTHONPATH*. The env var just inserts a new directory to the beginning
of *sys.path* but it doesn't parse the pth files in the directory. A
full blown site-packages path is required for several applications
and Python eggs.
Specification
=============
site directory (site-packages)
A directory in ``sys.path``. In contrast to ordinary directories the pth
files in the directory are processed, too.
user site directory
A site directory inside the users' home directory. An user site
directory is specific to a Python version. The path contains
the version number (major and minor only).
Mac
``~/Library/Python/2.6/site-packages``
Unix
``~/.local/lib/python2.6/site-packages``
Windows
``%APPDATA%/Python/Python26/site-packages``
user configuration directory
Usually the parent directory of the user site directory. It's meant
for Python version specific data like config files.
Mac
``~/Library/Python/2.6``
Unix
``~/.local/lib/python2.6``
Windows
``%APPDATA%/Python/Python26``
user base directory
It's located inside the user's home directory. The user site and
use config directory are inside the base directory. On some systems
the directory may be shared with 3rd party apps.
Mac
``~/Library/Python``
Unix
``~/.local``
Windows
``%APPDATA%/Python``
user script directory
A directory for binaries and scripts. [10]_ It's shared across Python
versions and the destination directory for scripts.
Mac
``~/Library/Python/bin``
Unix
``~/.local/bin``
Windows
``%APPDATA%/Python/Scripts``
On Windows ``APPDATA`` was chosen because it is the most logical place for
application data. Microsoft recommands that software doesn't write to
``USERPROFILE`` [5]_ and ``My Documents`` is not suited for application data,
too. [8]_
On Linux ``~/.local`` was chosen in favor over ``~/.python`` because the
directory is already used by several other programs in analogy to
``/usr/local``. [7]_
Implementation
==============
The site module gets a new method ``adduserpackage()`` which adds the
appropriate directory to the search path. The directory is not added if
it doesn't exist when Python is started. However the location of the
user site directory and user base directory is stored in an internal
variable for distutils.
The user site directory is added before the system site directories
but after Python's search paths and ``PYTHONPATH``. This setup allows
the user to install a different version of a package than the system
administrator but it prevents the user from accidently overwriting a
stdlib module. Stdlib modules can still be overwritten with
``PYTHONPATH``.
For security reasons the user site directory is *not* added to
``sys.path`` when the effective user id is not equal to the process
user id [9]_. It prevents users from injecting Python code into suid
apps.
The user site directory can be suppressed with a new option ``-s`` or
the environment variable ``PYTHONNOUSERSITE``. The feature can be
disabled globally by setting ``site.ENABLE_USER_SITE`` to the value
``False``. It must be set by editing ``site.py``. It can't be altered
``in sitecustomize.py`` or later.
``distutils.command.install`` (setup.py install) gets a new argument
``--user`` to install packages in the user site directory. The required
directories are created on demand.
``distutils.sysconfig`` will get methods to access the private variables
of site. (not yet implemented)
The Windows updater needs to be updated, too. It should create an menu
item which opens the user site directory in a new explorer windows.
Backwards Compatibility
=======================
TBD
Open Questions
==============
* Are the directories for Windows, Mac and Unix fine?
* Mac: Should framework and non-framework builds of Python use the
same directories?
* The patch also adds a usecustomize hook to site. Is it useful and
should it stay?
* Should the site package directory also be ignored if process gid !=
effective gid?
* Should the Windows installer add ``%APPDATA%/Python/Scripts`` to
``PATH``?
Reference Implementation
========================
A reference implementation is available in the bug tracker. [4]_
Copyright
=========
This document has been placed in the public domain.
References
==========
.. [1] Virtual Python
http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python
.. [2] Working Env
http://pypi.python.org/pypi/workingenv.py
http://blog.ianbicking.org/workingenv-revisited.html
.. [3] Virtual Env
http://pypi.python.org/pypi/virtualenv
.. [4] reference implementation
http://bugs.python.org/issue1799
.. [5] MSDN: CSIDL
http://msdn2.microsoft.com/en-us/library/bb762494.aspx
.. [6] Initial suggestion for a per user site-packages directory
http://permalink.gmane.org/gmane.comp.python.devel/90902
.. [7] Suggestion of ~/.local/
http://permalink.gmane.org/gmane.comp.python.devel/90925
.. [8] APPDATA discussion
http://permalink.gmane.org/gmane.comp.python.devel/90932
.. [9] Security concerns and -s option
http://permalink.gmane.org/gmane.comp.python.devel/91063
.. [10] Discussion about the bin directory
http://permalink.gmane.org/gmane.comp.python.devel/91095
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End: