推荐的编译器解析树数据结构

Question

我正在尝试构建一个基于 python 的自定义解析树数据结构。我知道 python 包含 ast、parser、tokenize 等模块。这些模块旨在使解析 python 语法相对容易，但是由于这些模块中的大多数都包含很少的文档，所以我正在努力解决它们。所以我想我有两个问题：

1.) 如何使用像 ast 这样的模块来获取解析树？ 2.) 您建议使用什么类型的树数据结构来保存此信息，以便我以后查看/编辑它？

任何帮助将不胜感激。

Answer 1

在 Python 终端，输入help(ast)：

>>> import ast
>>> help(ast)
Help on module ast:

NAME
    ast

FILE
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py

MODULE DOCS
    http://docs.python.org/library/ast

DESCRIPTION
    ast
    ~~~

    The `ast` module helps Python applications to process trees of the Python
    abstract syntax grammar.  The abstract syntax itself might change with
    each Python release; this module helps to find out programmatically what
    the current grammar looks like and allows modifications of it.

    An abstract syntax tree can be generated by passing `ast.PyCF_ONLY_AST` as
    a flag to the `compile()` builtin function or by using the `parse()`
    function from this module.  The result will be a tree of objects whose
    classes all inherit from `ast.AST`.
    …

这告诉我们调用parse() 方法。让我们试试吧：

#!/usr/bin/env python2.7

import ast
import sys

with open(sys.argv[0], 'r') as my_source:
    my_ast = ast.parse(my_source.read())

print ast.dump(my_ast)

输出有点乱，但是把它粘贴到你的编辑器中，自动缩进，你就可以得到详细的 AST：

Module(body=[Import(names=[alias(name='ast',
                                 asname=None)]),
             Import(names=[alias(name='sys',
                                 asname=None)]),
             With(context_expr=Call(func=Name(id='open',
                                              ctx=Load()),
                                    args=[Subscript(value=Attribute(value=Name(id='sys',
                                                                               ctx=Load()),
                                                                    attr='argv',
                                                                    ctx=Load()),
                                                    slice=Index(value=Num(n=0)),
                                                    ctx=Load()),
                                          Str(s='r')],
                                    keywords=[],
                                    starargs=None,
                                    kwargs=None),
                  optional_vars=Name(id='my_source',
                                     ctx=Store()),
                  body=[Assign(targets=[Name(id='my_ast',
                                             ctx=Store())],
                               value=Call(func=Attribute(value=Name(id='ast',
                                                                    ctx=Load()),
                                                         attr='parse',
                                                         ctx=Load()),
                                          args=[Call(func=Attribute(value=Name(id='my_source',
                                                                               ctx=Load()),
                                                                    attr='read',
                                                                    ctx=Load()),
                                                     args=[],
                                                     keywords=[],
                                                     starargs=None,
                                                     kwargs=None)],
                                          keywords=[],
                                          starargs=None,
                                          kwargs=None))]),
             Print(dest=None,
                   values=[Call(func=Attribute(value=Name(id='ast',
                                                          ctx=Load()),
                                               attr='dump',
                                               ctx=Load()),
                                args=[Name(id='my_ast',
                                           ctx=Load())],
                                keywords=[],
                                starargs=None,
                                kwargs=None)],
                   nl=True)])

要用代码戳东西，用python -i foo.py 运行上面的脚本来得到一个my_ast 对象，你可以用dir() 交互式地戳它：

>>> dir(my_ast)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_attributes', '_fields', 'body']
>>> my_ast.body
[<_ast.Import object at 0x1055b3590>, <_ast.Import object at 0x1055b3610>, <_ast.With object at 0x1055b3690>]
>>> with_stmt = my_ast.body[2]
>>> dir(with_stmt)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_attributes', '_fields', 'body', 'col_offset', 'context_expr', 'lineno', 'optional_vars']
>>> with_stmt.lineno
6
>>> with_stmt.body
[<_ast.Assign object at 0x1055b3910>]
>>> assign1 = with_stmt.body[0]
>>> dir(assign1)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_attributes', '_fields', 'col_offset', 'lineno', 'targets', 'value']
>>> assign1.value
<_ast.Call object at 0x1055b3990>