171 lines
5.7 KiB
Plaintext
171 lines
5.7 KiB
Plaintext
|
PEP: 339
|
|||
|
|
Title: How to Change CPython's Bytecode
|
|||
|
|
Version: $Revision$
|
|||
|
|
Last-Modified: $Date$
|
|||
|
|
Author: Brett Cannon <brett@python.org>
|
|||
|
|
Status: Active
|
|||
|
|
Type: Informational
|
|||
|
|
Content-Type: text/x-rst
|
|||
|
|
Created: 02-Feb-2005
|
|||
|
|
Post-History: 02-Feb-2005
|
|||
|
|
|
|||
|
|
|
|||
|
|
Abstract
|
|||
|
|
========
|
|||
|
|
|
|||
|
|
Python source code is compiled down to something called bytecode. This
|
|||
|
|
bytecode must implement enough semantics to perform the actions required by the
|
|||
|
|
Language Reference [#lang_ref]_. As such, knowing how to add, remove, or change
|
|||
|
|
the bytecode is important to do properly when changing the abilities of the
|
|||
|
|
Python language.
|
|||
|
|
This PEP covers how to accomplish this in the CPython implementation of the
|
|||
|
|
language (referred to as simply "Python" for the rest of this PEP).
|
|||
|
|
|
|||
|
|
.. warning::
|
|||
|
|
The guidelines outlined in this PEP apply to Python 2.4 and earlier.
|
|||
|
|
Current plans for Python 2.5 will lead to a significant change in how
|
|||
|
|
Python's bytecode is handled.
|
|||
|
|
This PEP will be updated once these planned changes are committed into
|
|||
|
|
CVS.
|
|||
|
|
|
|||
|
|
|
|||
|
|
Rationale
|
|||
|
|
=========
|
|||
|
|
|
|||
|
|
While changing Python's bytecode is not a frequent occurence, it still happens.
|
|||
|
|
Having the required steps documented in a single location should make
|
|||
|
|
experimentation with the bytecode easier since it is not necessarily obvious
|
|||
|
|
what the steps are to change the bytecode.
|
|||
|
|
|
|||
|
|
This PEP, paired with PEP 306 [#PEP-306]_, should provide enough basic
|
|||
|
|
guidelines for handling any changes performed to the Python language itself in
|
|||
|
|
terms of syntactic changes that introduce new semantics.
|
|||
|
|
|
|||
|
|
|
|||
|
|
Checklist
|
|||
|
|
=========
|
|||
|
|
|
|||
|
|
This is a rough checklist of what files need to change and how they are
|
|||
|
|
involved with the bytecode. All paths are given from the viewpoint of
|
|||
|
|
``/cvsroot/python/dist/src`` from CVS). This list should not be considered
|
|||
|
|
exhaustive nor to cover all possible situations.
|
|||
|
|
|
|||
|
|
- ``Include/opcode.h``
|
|||
|
|
This include file lists all known opcodes and associates each opcode
|
|||
|
|
name with
|
|||
|
|
a unique number. When adding a new opcode it is important to take note
|
|||
|
|
of the ``HAVE_ARGUMENT`` value. This ``#define``'s value specifies the
|
|||
|
|
value at which all opcodes greater than ``HAVE_ARGUMENT`` are expected
|
|||
|
|
to take an argument to the opcode.
|
|||
|
|
|
|||
|
|
- ``Lib/opcode.py``
|
|||
|
|
Lists all of the opcodes and their associated value. Used by the dis
|
|||
|
|
module [#dis]_ to map bytecode values to their names.
|
|||
|
|
|
|||
|
|
- ``Python/ceval.c``
|
|||
|
|
Contains the main interpreter loop. Code to handle the evalution of an
|
|||
|
|
opcode goes here.
|
|||
|
|
|
|||
|
|
- ``Python/compile.c``
|
|||
|
|
To make sure an opcode is actually used, this file must be altered.
|
|||
|
|
The emitting of all bytecode occurs here.
|
|||
|
|
|
|||
|
|
- ``Lib/compiler/pyassem.py``, ``Lib/compiler/pycodegen.py``
|
|||
|
|
The 'compiler' package [#compiler]_ needs to be altered to also reflect
|
|||
|
|
any changes to the bytecode.
|
|||
|
|
|
|||
|
|
- ``Doc/lib/libdis.tex``
|
|||
|
|
The documentation [#opcode_list]_ for the dis module contains a complete
|
|||
|
|
list of all the opcodes.
|
|||
|
|
|
|||
|
|
- ``Python/import.c``
|
|||
|
|
Defines the magic word (named ``MAGIC``) used in .pyc files to detect if
|
|||
|
|
the bytecode used matches the one used by the version of Python running.
|
|||
|
|
This number needs to be changed to make sure that the running
|
|||
|
|
interpreter does not try to execute bytecode that it does not know
|
|||
|
|
about.
|
|||
|
|
|
|||
|
|
|
|||
|
|
Suggestions for bytecode development
|
|||
|
|
====================================
|
|||
|
|
|
|||
|
|
A few things can be done to make sure that development goes smoothly when
|
|||
|
|
experimenting with Python's bytecode. One is to delete all .py(c|o) files
|
|||
|
|
after each semantic change to Python/compile.c . That way all files will use
|
|||
|
|
any bytecode changes.
|
|||
|
|
|
|||
|
|
Make sure to run the entire testing suite [#test-suite]_. Since the
|
|||
|
|
``regrtest.py`` driver recompiles all source code before a test is run it acts
|
|||
|
|
a good test to make sure that no existing semantics are broken.
|
|||
|
|
|
|||
|
|
Running parrotbench [#parrotbench]_ is also a good way to make sure existing
|
|||
|
|
semantics are not broken; this benchmark is practically a compliance test.
|
|||
|
|
|
|||
|
|
|
|||
|
|
Previous experiments
|
|||
|
|
====================
|
|||
|
|
This section lists known bytecode experiments that have not gone into Python.
|
|||
|
|
|
|||
|
|
Skip Montanaro presented a paper at a Python workshop on a peephole optimizer
|
|||
|
|
[#skip-peephole]_.
|
|||
|
|
|
|||
|
|
Michael Hudson has a non-active SourceForge project named Bytecodehacks
|
|||
|
|
[#Bytecodehacks]_ that provides functionality for playing with bytecode
|
|||
|
|
directly.
|
|||
|
|
|
|||
|
|
An opcode to combine the functionality of LOAD_ATTR/CALL_FUNCTION was created
|
|||
|
|
named CALL_ATTR [#CALL_ATTR]_. Currently only works for classic classes and
|
|||
|
|
for new-style classes rough benchmarking showed an actual slowdown thanks to
|
|||
|
|
having to support both classic and new-style classes.
|
|||
|
|
|
|||
|
|
|
|||
|
|
References
|
|||
|
|
==========
|
|||
|
|
|
|||
|
|
.. [#lang_ref] Python Language Reference, van Rossum & Drake
|
|||
|
|
(http://docs.python.org/ref/ref.html)
|
|||
|
|
|
|||
|
|
.. [#PEP-306] PEP 306, How to Change Python's Grammar, Hudson
|
|||
|
|
(http://www.python.org/peps/pep-0306.html)
|
|||
|
|
|
|||
|
|
.. [#dis] dis Module
|
|||
|
|
(http://docs.python.org/lib/module-dis.html)
|
|||
|
|
|
|||
|
|
.. [#compiler] 'compiler' Package
|
|||
|
|
(http://docs.python.org/lib/module-compiler.html)
|
|||
|
|
|
|||
|
|
.. [#test-suite] 'test' Package
|
|||
|
|
(http://docs.python.org/lib/module-test.html)
|
|||
|
|
|
|||
|
|
.. [#opcode_list] Python Byte Code Instructions
|
|||
|
|
(http://docs.python.org/lib/bytecodes.html)
|
|||
|
|
|
|||
|
|
.. [#parrotbench] Parrotbench
|
|||
|
|
(ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz,
|
|||
|
|
http://mail.python.org/pipermail/python-dev/2003-December/041527.html)
|
|||
|
|
|
|||
|
|
.. [#skip-peephole] Skip Montanaro's Peephole Optimizer Paper
|
|||
|
|
(http://www.foretec.com/python/workshops/1998-11/proceedings/papers/montanaro/montanaro.html)
|
|||
|
|
|
|||
|
|
.. [#Bytecodehacks] Bytecodehacks Project
|
|||
|
|
(http://bytecodehacks.sourceforge.net/bch-docs/bch/index.html)
|
|||
|
|
|
|||
|
|
.. [#CALL_ATTR] CALL_ATTR opcode
|
|||
|
|
(http://www.python.org/sf/709744)
|
|||
|
|
|
|||
|
|
|
|||
|
|
Copyright
|
|||
|
|
=========
|
|||
|
|
|
|||
|
|
This document has been placed in the public domain.
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
..
|
|||
|
|
Local Variables:
|
|||
|
|
mode: indented-text
|
|||
|
|
indent-tabs-mode: nil
|
|||
|
|
sentence-end-double-space: t
|
|||
|
|
fill-column: 70
|
|||
|
|
End:
|