From 95f614c983ff963307b2a9e16271504a600729b8 Mon Sep 17 00:00:00 2001 From: Barry Warsaw Date: Tue, 13 Apr 2010 02:17:40 +0000 Subject: [PATCH] Updates --- pep-3147.txt | 140 ++++++++++++++++----------------------------------- 1 file changed, 44 insertions(+), 96 deletions(-) diff --git a/pep-3147.txt b/pep-3147.txt index 6ad39f8f4..f4929da29 100644 --- a/pep-3147.txt +++ b/pep-3147.txt @@ -8,7 +8,7 @@ Type: Standards Track Content-Type: text/x-rst Created: 2009-12-16 Python-Version: 3.2 -Post-History: 2010-01-30, 2010-02-25, 2010-03-03 +Post-History: 2010-01-30, 2010-02-25, 2010-03-03, 2010-04-12 Abstract @@ -317,21 +317,25 @@ identifier:: 'foo.580c0d0a.pyc' This isn't particularly human friendly though. Instead, this PEP -proposes to add a mapping between internal magic numbers and a -user-friendly tag. Newer versions of Python can add to this mapping -so that all later Pythons know the mapping between tags and magic -numbers. By convention, the tag will contain the Python -implementation name and version nickname, where the nickname is -generally the major version number and minor version number. Magic -numbers should not change between Python micro releases, but some -other convention can be used for changes in magic number between +proposes a *magic tag* that uniquely defines `.pyc` files for the +current version of Python. Whenever the magic number is bumped, a new +magic tag is defined which is unique among all versions and +implementations of Python. The actual contents of the magic tag is +left up to the implementation, although it is recommended that the tag +include the implementation name and a version shorthand. In general, +magic numbers never change between Python micro releases, but the +convention can be extended to handle magic number changes between pre-release development versions. -For example, CPython 3.2 would have a magic identifier tag of -`cpython-32` and write pyc files like this: `foo.cpython-32.pyc`. -When the `-O` flag is used, it would write `foo.cpython-32.pyo`. For -backports of this feature to Python 2, when the `-U` flag is used, a -file such as `foo.cpython-27u.pyc` can be written. +For example, CPython 3.2 would have a magic tag of `cpython-32` and +write pyc files like this: `foo.cpython-32.pyc`. When the `-O` flag +is used, it would write `foo.cpython-32.pyo`. For backports of this +feature to Python 2, when the `-U` flag is used, a file such as +`foo.cpython-27u.pyc` can be written. + +The magic tag is available in the `imp` module via the `get_tag()` +function. This is analogous to the `get_magic()` function already +available in that module. Alternative Python implementations @@ -363,7 +367,7 @@ Python and outside. This section enumerates some of these effects. __file__ --------- -in Python 3, when you import a module, its `__file__` attribute points +In Python 3, when you import a module, its `__file__` attribute points to its source `py` file (in Python 2, it points to the `pyc` file). A package's `__file__` points to the `py` file for its `__init__.py`. E.g.:: @@ -376,18 +380,16 @@ E.g.:: >>> baz.__file__ 'baz/__init__.py' -The implementation of this PEP would have to ensure that the same -directory level is returned from `__file__` as it currently does so -that the common idiom above continues to work. +Nothing in this PEP would change the semantics of `__file__`. -As part of this PEP, we will add an `__cached__` attribute to modules, -which will always point to the actual `pyc` file that was read or -written. When the environment variable `$PYTHONDONTWRITEBYTECODE` is -set, or the `-B` option is given, or if the source lives on a -read-only filesystem, then the `__cached__` attribute will point to -the location that the `pyc` file *would* have been written to if it -didn't exist. This location of course includes the `__pycache__` -subdirectory in its path. +This PEP proposes the addition of an `__cached__` attribute to +modules, which will always point to the actual `pyc` file that was +read or written. When the environment variable +`$PYTHONDONTWRITEBYTECODE` is set, or the `-B` option is given, or if +the source lives on a read-only filesystem, then the `__cached__` +attribute will point to the location that the `pyc` file *would* have +been written to if it didn't exist. This location of course includes +the `__pycache__` subdirectory in its path. For alternative Python implementations which do not support `pyc` files, the `__cached__` attribute may point to whatever information @@ -397,19 +399,19 @@ use multiple compiled files to create the module, in which case `__cached__` may be a tuple. The exact contents of `__cached__` are Python implementation specific. -Alternative implementations for which this scheme does not make sense -should set the `__cached__` attribute to `None`. +It is recommended that when nothing sensible can be calculated, +implementations should set the `__cached__` attribute to `None`. py_compile and compileall ------------------------- -Python comes with two modules, `py_compile` [15]_ and `compileall` [16]_ -which support compiling Python modules external to the built-in import -machinery. `py_compile` in particular has intimate knowledge of byte -compilation, so these will have to be updated to understand the new -layout. It's possible that `compileall` could be modified to support -legacy byte-code only file system layout. +Python comes with two modules, `py_compile` [15]_ and `compileall` +[16]_ which support compiling Python modules external to the built-in +import machinery. `py_compile` in particular has intimate knowledge +of byte compilation, so these will be updated to understand the new +layout. The `-b` flag is added to `compileall` for writing legacy +`.pyc` byte-compiled file path names. bdist_wininst and the Windows installer @@ -459,6 +461,14 @@ an environment variable called `$PYTHONENABLECACHEDIR` or the command line switch `-Xenablecachedir` to enable the feature. +Makefiles and other dependency tools +------------------------------------ + +Makefiles and other tools which calculate dependencies on `.pyc` files +(e.g. to byte-compile the source if the `.pyc` is missing) will have +to be updated to check the new paths. + + Alternatives ============ @@ -543,68 +553,6 @@ A Rietveld code review issue [24]_ has been opened as of 2010-04-01 (no, this is not an April Fools joke :). -Open issues -=========== - -__pycache__ vs. __cachepy__ ------------------------------ - -Minor point, but __pycache__ sorts after __init__.py alphabetically so -that might be a little jarring (see the directory layout examples -above). It seems that `ls(1)` on Linux at least also sorts the files -alphabetically, ignoring the leading underscores. - -Should we name the cache directory something like `__cachepy__` so -that it sorts before `__init__.py`? OTOH, many graphical file system -navigators sort directories before plain files anyway, so maybe it -doesn't matter. - -Here are some sample `ls(1) -l` output. First, with `__pycache__`:: - - % ls -l - total 8 - -rw-rw-r-- 1 user user 0 2010-03-03 08:29 alpha.py - drwxrwxr-x 2 user user 4096 2010-03-03 08:28 beta/ - -rw-rw-r-- 1 user user 0 2010-03-03 08:28 __init__.py - -rw-rw-r-- 1 user user 0 2010-03-03 08:28 one.py - drwxrwxr-x 2 user user 4096 2010-03-03 08:28 __pycache__/ - -rw-rw-r-- 1 user user 0 2010-03-03 08:28 two.py - -Now, with `__cachepy__`:: - - % ls -l - total 8 - -rw-rw-r-- 1 user user 0 2010-03-03 08:29 alpha.py - drwxrwxr-x 2 user user 4096 2010-03-03 08:28 beta/ - drwxrwxr-x 2 user user 4096 2010-03-03 08:28 __cachepy__/ - -rw-rw-r-- 1 user user 0 2010-03-03 08:28 __init__.py - -rw-rw-r-- 1 user user 0 2010-03-03 08:28 one.py - -rw-rw-r-- 1 user user 0 2010-03-03 08:28 two.py - - -__cached__ vs. __compiled__ ----------------------------- - -Guido says: "I still prefer __compiled__ over __cached__ but I don't -feel strong about it." - -Barry likes `__cached__` because it the more general term seems to fit -in better with future possible use cases such as JIT output from -Unladen Swallow. - - -On-demand creation ------------------- - -The original use case was to support multiple versions of Python on -the same system. This could be accomplished solely through -pre-creation of `__pycache__` directories during the compilation phase -of installation, without also supporting on-demand creation of the -`__pycache__` directories. It seems however the consensus to be that -on-demand creation is useful enough to keep since it reduces clutter -in source directories. - - References ==========