{
    "info": {
        "author": "Michael Chow",
        "author_email": "michael@datacamp.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: GNU Affero General Public License v3",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# antlr-ast\n\n[![Build Status](https://travis-ci.org/datacamp/antlr-ast.svg?branch=master)](https://travis-ci.org/datacamp/antlr-ast)\n[![codecov](https://codecov.io/gh/datacamp/antlr-ast/branch/master/graph/badge.svg)](https://codecov.io/gh/datacamp/antlr-ast)\n\nThis package allows you to use ANTLR grammars and use the parser output to generate an abstract syntax tree (AST).\n\n## Install\n\n```bash\npip install antlr-ast\n```\n\n**Note:** this package is not python2 compatible.\n\n## Running Tests\n\n```bash\n# may need:\n# pip install pytest\npy.test\n```\n\n## Usage\n\nUsing `antlr-ast` involves four steps:\n\n1. Using ANTLR to define a grammar and to generate the necessary Python files to parse this grammar\n2. Using `parse` to get the ANTLR runtime output based on the generated grammar files\n3. Using `process_tree` on the output of the previous step\n    1. A `BaseAstVisitor` (customisable by providing a subclass) transforms the ANTLR output to a serializable tree of `BaseNode`s,\n       dynamically created based on the rules in the ANTLR grammar\n    2. A `BaseNodeTransformer` subclass can be used to transform each kind of node\n    3. The simplify option can be used to shorten paths in the tree by skipping nodes that only have a single descendant\n4. Using the resulting tree\n\nThe next sections go into more detail about these steps.\n\nTo visualize the process of creating and transforrming these parse trees, you can use [this ast-viewer](https://github.com/datacamp/ast-viewer).\n\n### Using ANTLR\n\n**Note: For this part of this tutorial you need to know how to parse code**  \nSee the ANTLR [getting started guide](https://github.com/antlr/antlr4/blob/4.7.2/doc/getting-started.md) if you have never installed ANTLR.  \nThe [ANTLR Mega Tutorial](https://tomassetti.me/antlr-mega-tutorial/#python-setup) has useful Python examples.\n\n[This page explains how to write ANTLR parser rules](https://github.com/antlr/antlr4/blob/master/doc/parser-rules.md).  \nThe rule definition below is an example with descriptive names for important ANTLR parser grammar elements:\n\n```g4\nrule_name: rule_element? rule_element_label='literal'    #RuleAlternativeLabel\n         | TOKEN+                                        #RuleAlternativeLabel\n         ;\n```\n\nRule element and alternative labels are optional.  \n`+`, `*`, `?`, `|` and `()` have the same meaning as in RegEx.\n\nBelow, we'll use a simple grammar to explain how `antlr-ast` works.\nThis grammar can be found in `/tests/Expr.g4`.\n\n```g4\ngrammar Expr;\n\n// parser\n\nexpr:   left=expr op=('+'|'-') right=expr       #BinaryExpr\n    |   NOT expr                                #NotExpr\n    |   INT                                     #Integer\n    |   '(' expr ')'                            #SubExpr\n    ;\n\n// lexer\n\nINT :   [0-9]+ ;         // match integers\nNOT :   'not' ;\n\nWS  :   [ \\t]+ -> skip ; // toss out whitespace\n```\n\nANTLR can use the grammar above to generate a parser in a number of languages.\nTo generate a Python parser, you can use the following command.\n\n```bash\nantlr4 -Dlanguage=Python3 -visitor /tests/Expr.g4\n```\n\nThis will generate a number of files in the `/tests/` directory, including a Lexer (`ExprLexer.py`),\na parser (`ExprParser.py`), and a visitor (`ExprVisitor.py`).\n\nYou can use and import these directly in Python. For example, from the root of this repo:\n\n```bash\nfrom tests import ExprVisitor\n```\n\nTo easily use the generated files, they are put in the `antlr_py` package.\nThe `__init__.py` file exports the generated files under an alias that doesn't include the name of the grammar.\n\n### Base nodes\n\nA `BaseNode` subclass has fields for all rule elements and labels for all rule element labels in its corresponding grammar rule.\nBoth fields and labels are available as properties on `BaseNode` instances.\nLabels take precedence over fields if the names would collide.\n\nThe name of a `BaseNode` is the name of the corresponding ANTLR grammar rule, but starting with an uppercase character.\nIf rule alternative labels are specified for an ANTLR rule, these are used instead of the rule name.\n\n### Transforming nodes\n\nTypically, there is no 1-to-1 mapping between ANTLR rules and the concepts of a language: the rule hierarchy is more nested.\nTransformations can be used to make the initial tree of BaseNodes based on ANTLR rules more similar to an AST.\n\n#### Transformer\n\nThe `BaseNodeTransformer` will walk over the tree from the root node to the leaf nodes.\nWhen visiting a node, it is possible to transform it.\nThe tree is updated with transformed node before continuing the walk over the tree.\n\nTo define a node transform, add a static method to the `BaseNodeTransformer` subclass passed to `process_tree`.\n\n- The name of the method you should define follows this pattern: `visit_<BaseNode>`,\n  where `<BaseNode>` should be replaced by the name of the `BaseNode` subclass to transform.\n- The method should return the transformed node.\n\nThis is a simple example:\n\n```python\nclass Transformer(BaseNodeTransformer):\n    @staticmethod\n    def visit_My_antlr_rule(node):\n        return node.name_of_part\n```\n\n#### Custom nodes\n\nA custom node can represent a part of the parsed language, a type of node present in an AST.\n\nTo make it easy to return a custom node, you can define `AliasNode` subclasses.\nNormally, fields of `AliasNode`s are like symlinks to navigate the tree of `BaseNode`s.\n\nInstances of custom nodes are created from a `BaseNode`.\nFields and labels of the source `BaseNode` are also available on the `AliasNode`.\nIf an `AliasNode` field name collides with these, it takes precedence when accessing that property.\n\nThis is what a custom node looks like:\n\n```python\nclass NotExpr(AliasNode):\n    _fields_spec = [\"expr\", \"op=NOT\"]\n```\n\nThis code defines a custom node, `NotExpr` with an `expr` and an `op` field.\n\n##### Field specs\n\nThe `_fields_spec` class property is a list that defines the fields the custom node should have.\n\nThis is how a field spec in this list is used when creating an custom node from a `BaseNode` (the source node):\n\n- If a field spec does not exist on the source node, it is set to `None`\n- If multiple field specs define the same field, the first one that isn't `None` is used\n- If a field spec is just a name, it is copied from the source node\n- If a field spec is an assignment, the left side is the name of the field on the `AliasNode`\n  and the right side is the path that should be taken starting in the source node to get the node\n  that should be the value for the field on the custom node.\n  Parts of this path are separated using `.`\n\n##### Connecting to the transformer\n\nTo use this custom node, add a method to the transformer:\n\n```python\nclass Transformer(BaseNodeTransformer):\n    # ...\n\n    # here the BaseNode name is the same as the custom node name\n    # but that isn't required\n    @staticmethod\n    def visit_NotExpr(node):\n        return NotExpr.from_spec(node)\n```\n\nInstead of defining methods on the transformer class to use custom nodes, it's possible to do this automatically:\n\n```python\nTransformer.bind_alias_nodes(alias_nodes)\n```\n\nTo make this work, the `AliasNode` classes in the list should have a `_rules` class property\nwith a list of the `BaseNode` names it should transform.\n\nThis is the result:\n\n```python\nclass NotExpr(AliasNode):\n    _fields_spec = [\"expr\", \"op=NOT\"]\n    _rules = [\"NotExpr\"]\n\nclass Transformer(BaseNodeTransformer):\n    pass\n\nalias_nodes = [NotExpr]\nTransformer.bind_alias_nodes(alias_nodes)\n```\n\nAn item in `_rules` can also be a tuple.\nIn that case, the first item in the tuple is a `BaseNode` name\nand the second item is the name of a class method of the custom node.\n\nIt's not useful in the example above, but it is equivalent to this:\n\n```python\nclass NotExpr(AliasNode):\n    _fields_spec = [\"expr\", \"op=NOT\"]\n    _rules = [(\"NotExpr\", \"from_not\")]\n\n    @classmethod\n    def from_not(cls, node):\n        return cls.from_spec(node)\n\nclass Transformer(BaseNodeTransformer):\n    pass\n\nalias_nodes = [NotExpr]\nTransformer.bind_alias_nodes(alias_nodes)\n```\n\n### Using the final tree\n\nIt's easy to use a tree that has a mix of `AliasNode`s and dynamic `BaseNode`s:\nthe whole tree is just a nested Python object.\n\nWhen searching nodes in a tree, the priority of nodes can be taken into account.\nBy default, `BaseNode`s have priority 3 and `AliasNode`s have priority 2.\n\nWhen writing code to work with trees, it can be affected by changes in the grammar, the transforms and the custom nodes.\nThe grammar is the most likely to change.\n\nTo make grammar updates have no impact on your code, don't rely on `BaseNode`s.\nYou can still check whether the `AliasNode` parent node of a `BaseNode` has the correct fields set\nand search for nested `AliasNode`s in a subtree.\n\nIf you do rely on `BaseNode`s, code could break by the addition of `AliasNode`s that replace some of these\nif a field name collides with a field name on a used `BaseNode`.",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/datacamp/antlr-ast",
        "keywords": "",
        "license": "",
        "maintainer": "Jeroen Hermans",
        "maintainer_email": "content-engineering@datacamp.com",
        "name": "antlr-ast",
        "package_url": "https://pypi.org/project/antlr-ast/",
        "platform": "",
        "project_url": "https://pypi.org/project/antlr-ast/",
        "project_urls": {
            "Homepage": "https://github.com/datacamp/antlr-ast"
        },
        "release_url": "https://pypi.org/project/antlr-ast/0.8.1/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "AST shaping for antlr parsers",
        "version": "0.8.1"
    },
    "last_serial": 5896209,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ef32a27c33e71be527a5db474f9f0d84",
                    "sha256": "243835d5dfdd51f292db6cf21c5ada7d58fbd3313a90a8caa337794f0173238c"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "ef32a27c33e71be527a5db474f9f0d84",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2284,
                "upload_time": "2017-04-13T21:15:19",
                "url": "https://files.pythonhosted.org/packages/27/2c/b802603c2aa7608e2171013b87675b1fd3bda13e4ab5b0e380be94bdf38d/antlr-ast-0.0.1.tar.gz"
            }
        ],
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8c1a72d86c962bc378c080d0f229fb82",
                    "sha256": "01d2320240de266415d944d63c41c1a94f289175d1f440e23cf0271116d932a8"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8c1a72d86c962bc378c080d0f229fb82",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2554,
                "upload_time": "2017-04-18T13:44:50",
                "url": "https://files.pythonhosted.org/packages/23/8a/0d39ebd5e721efead1a3eba6e60c150a13388dc87930ae0de52854b6553e/antlr-ast-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "310bdb9b28648640a82d5e9df633f2f4",
                    "sha256": "82b7bae907eaa5586ae0d9b48db434091091b8347b441051cb9b4f85b3fb5834"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "310bdb9b28648640a82d5e9df633f2f4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3027,
                "upload_time": "2017-05-08T18:06:33",
                "url": "https://files.pythonhosted.org/packages/46/ff/5681ff86b69f498b64f4cee64b2ff91eb807dc012c0cfb51bb5dc9cd84fd/antlr-ast-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "77221c13f5bd8874a1e1e22fb74b66f9",
                    "sha256": "46b619e554306cfdfd5b84aa36cef48131312ce06ac90372eff3215181b41d76"
                },
                "downloads": -1,
                "filename": "antlr_ast-0.2.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "77221c13f5bd8874a1e1e22fb74b66f9",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 15871,
                "upload_time": "2018-10-22T21:14:16",
                "url": "https://files.pythonhosted.org/packages/26/bb/f6af177709961846bed1826ef597ea631e8e59621f53e48c8d2bc757212d/antlr_ast-0.2.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f9b79b5ee74453a2bbbce79f45a7692",
                    "sha256": "942f5bf0fc88169a6c3e0c2177091cbb3e2043409fc42529b6906249d6d9cbf3"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "9f9b79b5ee74453a2bbbce79f45a7692",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5197,
                "upload_time": "2018-10-22T21:14:18",
                "url": "https://files.pythonhosted.org/packages/f8/94/2c9bd70cc08024505ed50e3627824e87f21fbdc3fc76ba8d0faa5ceabcdc/antlr-ast-0.2.1.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5668b5589af9466759039c770695d2ba",
                    "sha256": "295cc863b75049f0a879ca1db1d63e39086edb007f64193dcb2ea4431a69f9ff"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "5668b5589af9466759039c770695d2ba",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19877,
                "upload_time": "2019-01-02T11:58:52",
                "url": "https://files.pythonhosted.org/packages/9f/49/e53d733bb477454cbde635640cd7e6959d98fdcf40c2d48c3a1acf3bfe25/antlr-ast-0.3.0.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "80adc686c533e1bebbf21e641a94e7c6",
                    "sha256": "d1cd16aaff3e721e78c451db3a74311ee2a021164c6b384008672efe6a49d8b4"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "80adc686c533e1bebbf21e641a94e7c6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17132,
                "upload_time": "2019-01-02T15:32:14",
                "url": "https://files.pythonhosted.org/packages/5d/81/8e20e2e6c218a952f03a165a0541ec01f6d4bea2370d59f9270c141a323f/antlr-ast-0.4.0.tar.gz"
            }
        ],
        "0.4.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "90cae3d8e2c43aacefc832d2b1da8b4c",
                    "sha256": "953704e168f3026776e4a7b7e8fd709ba2563b00073847d3358212a1f15b3e33"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.4.1.tar.gz",
                "has_sig": false,
                "md5_digest": "90cae3d8e2c43aacefc832d2b1da8b4c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20172,
                "upload_time": "2019-01-02T16:34:40",
                "url": "https://files.pythonhosted.org/packages/97/5b/89eb3ea3dcfe7520de43e75361cb74ef742eceb2cd5ad69e8a39a56044d5/antlr-ast-0.4.1.tar.gz"
            }
        ],
        "0.4.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d1666c8713f1461ba49dba14e1376c39",
                    "sha256": "9699371d66319428db58836ceb0d17a46a810a99713e91d1680709f6cad78aef"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.4.2.tar.gz",
                "has_sig": false,
                "md5_digest": "d1666c8713f1461ba49dba14e1376c39",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20191,
                "upload_time": "2019-01-25T16:40:40",
                "url": "https://files.pythonhosted.org/packages/ac/a7/08df20e214f1e3b16706995834fb89f7756299c78fab55c23a4bb5105bab/antlr-ast-0.4.2.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e64046c77fb255cd268c6fb1efb4d159",
                    "sha256": "506a487731c164defcdcadcb50b81a7ce4af97518658be457c6de7830a0e29f0"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "e64046c77fb255cd268c6fb1efb4d159",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 26269,
                "upload_time": "2019-03-11T18:20:40",
                "url": "https://files.pythonhosted.org/packages/a6/8a/a2444ebfe87c9fa6938fc5d339a90d9dc246df82883370af3c844f6502db/antlr-ast-0.5.0.tar.gz"
            }
        ],
        "0.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fe5f84065b007e01778cb462f2ce6ff0",
                    "sha256": "e675245733f4aa86bdcd0556fa73e3027ca6aa17a66fedf75a87182f4950a90e"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "fe5f84065b007e01778cb462f2ce6ff0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29073,
                "upload_time": "2019-08-20T11:57:06",
                "url": "https://files.pythonhosted.org/packages/e7/a1/49e675cd622984d927921a3fe143fea2f9859c61f6855476b288b7df7c71/antlr-ast-0.7.0.tar.gz"
            }
        ],
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3e7e6c7b9487fb7f2510aba10cf830c4",
                    "sha256": "b915d645bb33367ce0d23bafd70e077a3f2b0c260b88174d7fcd00301b632ce6"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3e7e6c7b9487fb7f2510aba10cf830c4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29254,
                "upload_time": "2019-09-03T11:09:46",
                "url": "https://files.pythonhosted.org/packages/8c/f1/af3944621e51b270c7b17b4876d9c5da1e6d62943a5fdc58d2dda8059f85/antlr-ast-0.8.0.tar.gz"
            }
        ],
        "0.8.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "55393dcd1342974e1d47bc8e6810cc02",
                    "sha256": "534ff94889de3561de0a126e76e003bc3a992e9ac0c2f4c7ea8563e651e695ff"
                },
                "downloads": -1,
                "filename": "antlr-ast-0.8.1.tar.gz",
                "has_sig": false,
                "md5_digest": "55393dcd1342974e1d47bc8e6810cc02",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29393,
                "upload_time": "2019-09-27T13:47:18",
                "url": "https://files.pythonhosted.org/packages/06/0e/85ce7fe4d672995947e6582f22cd76841044fe0faaa1960bd07ab401acf9/antlr-ast-0.8.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "55393dcd1342974e1d47bc8e6810cc02",
                "sha256": "534ff94889de3561de0a126e76e003bc3a992e9ac0c2f4c7ea8563e651e695ff"
            },
            "downloads": -1,
            "filename": "antlr-ast-0.8.1.tar.gz",
            "has_sig": false,
            "md5_digest": "55393dcd1342974e1d47bc8e6810cc02",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 29393,
            "upload_time": "2019-09-27T13:47:18",
            "url": "https://files.pythonhosted.org/packages/06/0e/85ce7fe4d672995947e6582f22cd76841044fe0faaa1960bd07ab401acf9/antlr-ast-0.8.1.tar.gz"
        }
    ]
}