116 lines
3.3 KiB
ReStructuredText
116 lines
3.3 KiB
ReStructuredText
PEP: 306
|
|
Title: How to Change Python's Grammar
|
|
Version: $Revision$
|
|
Last-Modified: $Date$
|
|
Author: Michael Hudson <mwh@python.net>, Jack Diederich <jackdied@gmail.com>, Alyssa Coghlan <ncoghlan@gmail.com>, Benjamin Peterson <benjamin@python.org>
|
|
Status: Withdrawn
|
|
Type: Informational
|
|
Content-Type: text/x-rst
|
|
Created: 29-Jan-2003
|
|
Post-History: 30-Jan-2003
|
|
|
|
|
|
Note
|
|
====
|
|
|
|
This PEP has been moved to the Python dev guide [1]_.
|
|
|
|
|
|
Abstract
|
|
========
|
|
|
|
There's more to changing Python's grammar than editing
|
|
``Grammar/Grammar`` and ``Python/compile.c``. This PEP aims to be a
|
|
checklist of places that must also be fixed.
|
|
|
|
It is probably incomplete. If you see omissions, just add them if
|
|
you can -- you are not going to offend the author's sense of
|
|
ownership. Otherwise submit a bug or patch and assign it to mwh.
|
|
|
|
This PEP is not intended to be an instruction manual on Python
|
|
grammar hacking, for several reasons.
|
|
|
|
|
|
Rationale
|
|
=========
|
|
|
|
People are getting this wrong all the time; it took well over a
|
|
year before someone noticed [2]_ that adding the floor division
|
|
operator (``//``) broke the ``parser`` module.
|
|
|
|
|
|
Checklist
|
|
=========
|
|
|
|
- ``Grammar/Grammar``: OK, you'd probably worked this one out :)
|
|
|
|
- ``Parser/Python.asdl`` may need changes to match the ``Grammar``. Run
|
|
``make`` to regenerate ``Include/Python-ast.h`` and
|
|
``Python/Python-ast.c``.
|
|
|
|
- ``Python/ast.c`` will need changes to create the AST objects
|
|
involved with the ``Grammar`` change. ``Lib/compiler/ast.py`` will
|
|
need matching changes to the pure-python AST objects.
|
|
|
|
- ``Parser/pgen`` needs to be rerun to regenerate ``Include/graminit.h``
|
|
and ``Python/graminit.c``. (make should handle this for you.)
|
|
|
|
- ``Python/symbtable.c``: This handles the symbol collection pass
|
|
that happens immediately before the compilation pass.
|
|
|
|
- ``Python/compile.c``: You will need to create or modify the
|
|
``compiler_*`` functions to generate opcodes for your productions.
|
|
|
|
- You may need to regenerate ``Lib/symbol.py`` and/or ``Lib/token.py``
|
|
and/or ``Lib/keyword.py``.
|
|
|
|
- The ``parser`` module. Add some of your new syntax to ``test_parser``,
|
|
bang on ``Modules/parsermodule.c`` until it passes.
|
|
|
|
- Add some usage of your new syntax to ``test_grammar.py``.
|
|
|
|
- The ``compiler`` package. A good test is to compile the standard
|
|
library and test suite with the ``compiler`` package and then check
|
|
it runs. Note that this only needs to be done in Python 2.x.
|
|
|
|
- If you've gone so far as to change the token structure of
|
|
Python, then the ``Lib/tokenizer.py`` library module will need to
|
|
be changed.
|
|
|
|
- Certain changes may require tweaks to the library module
|
|
``pyclbr``.
|
|
|
|
- Documentation must be written!
|
|
|
|
- After everything's been checked in, you're likely to see a new
|
|
change to ``Python/Python-ast.c``. This is because this
|
|
(generated) file contains the SVN version of the source from
|
|
which it was generated. There's no way to avoid this; you just
|
|
have to submit this file separately.
|
|
|
|
|
|
References
|
|
==========
|
|
|
|
.. [1] CPython Developer's Guide: Changing CPython's Grammar
|
|
https://devguide.python.org/grammar/
|
|
|
|
.. [2] SF Bug #676521, parser module validation failure
|
|
https://bugs.python.org/issue676521
|
|
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This document has been placed in the public domain.
|
|
|
|
|
|
|
|
..
|
|
Local Variables:
|
|
mode: indented-text
|
|
indent-tabs-mode: nil
|
|
sentence-end-double-space: t
|
|
fill-column: 70
|
|
End:
|