{ "info": { "author": "Jan Max Meyer", "author_email": "info@pynetree.org", "bugtrack_url": null, "classifiers": [], "description": "**pynetree**: *A light-weight parsing toolkit written in pure Python*\n\n\nAbout\n=====\n\n**pynetree** is both a Python library and a command-line utility for\nparsing.\n\nParsing is the process of transferring input matching a particular\ngrammar, like e.g. a program source code, into a well-formed data\nstructure. This data structure is a so called abstract syntax tree\n(AST). pynetree is a tool doing all this for you: It takes a grammar\ndescription, runs a parser on provided input and generates the abstract\nsyntax tree on a successful parse. This AST can than be taken to perform\nsubsequent tasks, like writing an interpreter, compiler or any other\nkind of software requiring a parser.\n\nThe following Python example using pynetree defines a simple\ntwo-function calculator as an expression language, runs a parser on it,\nand dumps out the generated abstract syntax tree.\n\n.. code:: python\n\n import pynetree\n\n p = pynetree.Parser(\"\"\"\n %skip /\\s+/;\n @int /\\d+/;\n\n factor: int | '(' expr ')';\n\n @mul: term '*' factor;\n term: mul | factor;\n\n @add: expr '+' term;\n expr$: add | term;\n \"\"\")\n\n p.parse(\"1 + 2 * ( 3 + 4 ) * 5\").dump()\n\nWhen this program is run from a console, a proper abstract syntax tree\nwill be generated and printed, which shows the hierarchical structure of\nthe parsed expression.\n\n::\n\n add\n int (1)\n div\n mul\n int (2)\n add\n int (3)\n int (4)\n int (5)\n\npynetree also provides a handy command-line tool to rapidly prototype\ngrammars. The next command just generates the same parser as the example\nprogram from above.\n\n::\n\n $ pynetree \"@int /[0-9]+/; f: int | '(' e ')'; t: @mul( t '*' f ) | f; e: @add( e '+' t ) | t;\"\n\nRequirements\n============\n\npynetree is written in pure Python. It runs natively with any Python >=\n2.5.\n\nThe only import done so far is the build-in ``re`` module for regular\nexpression support. Nothing more is required!\n\nFeatures\n========\n\npynetree so far provides\n\n- A top-down packrat parser with support of direct and indirect left\n recursive grammars.\n- Mostly linear parsing time, even for left-recursive grammars due a\n powerful memorization algorithm.\n- Grammars can be expressed as dict objects or by a BNF-grammar.\n- Support functions for generating and traversing abstract syntax trees\n (AST).\n- Lexical analysis can be performed via regular expressions (re),\n string or by Python callables.\n\nPlease check out http://pynetree.org to get help and the newest updates\non the pynetree project.\n\nThe pynetree project is under heavy development, so that changes in API,\nfunction names, syntax or semantics may occur and need existing projects\nto be ported.\n\nIt has been developed in the course of implementing a top-down parser\nsupporting left-recursive grammars. Therefore, pynetree is a parser that\nimplements a modified version of the packrat parsing algorithm, but with\nthe approach to provide true BNF-styled grammars, as known from other\nparser development tools and frameworks.\n\nGetting started\n===============\n\npynetree is not a parser generator in classic terms like yacc or bison.\nIt can be seen as a library to directly express and parse the desired\ngrammar within Python code.\n\nCommand-line interface\n----------------------\n\nNevertheless, a command-line interface is provided for rapidly grammar\nprototyping and testing.\n\n::\n\n usage: pynetree.py [-h] [-d] [-v] [-V] grammar [input [input ...]]\n\n pynetree - a light-weight parsing toolkit written in Python.\n\n positional arguments:\n grammar Grammar to create a parser from.\n input Input to be processed by the parser.\n\n optional arguments:\n -h, --help show this help message and exit\n -d, --debug Verbose, and print debug output\n -v, --verbose Print processing information during run\n -V, --version show program's version number and exit\n\n 'grammar' and 'input' can be either supplied as strings or files.\n\nUsing it as a library\n---------------------\n\nFor using pynetree in a Python script, it simply is required to create\nan object of the class ``pynetree.Parser``.\n\nThe Parser class requires a BNF-grammar to describe the language that\nshall be parsed as parameter. This grammar can be expressed in two\ndifferent ways:\n\n1. As dict specifying the non-terminals as keys and their left-hand\n sides as the values (where the left-hand sides can be provided as\n list of strings). This requires much more configuration on the\n parsers token and emitting behavior, but is the Python-specific way.\n\n2. The other, more flexible method for specifying a grammar is\n pynetree's own grammar definition language, which itself internally\n is implemented using pynetree.\n\n This language is oriented on the classic BNF-style, but supports\n several syntactical elements to quickly define a language and its\n behavior for token definition and abstract syntax tree (AST)\n construction.\n\nAfter the grammar is generally defined, the parser can be fine-tuned or\nextended to various features.\n\n- ``pynetree.Parser.token()`` is used to define named terminal symbols,\n which can be regular expression patterns, static strings or\n callables.\n- ``pynetree.Parser.ignore()`` is used for the definition of whitespace\n tokens, which are generally allowed between all other tokens.\n- ``pynetree.Parser.emit()`` is used to define symbols (both\n non-terminal or terminal) that are emitted as nodes in AST. Terminal\n symbols will always define leafs in the AST, where non-terminals can\n emit leafs if no sub-ordered symbols are emitted. (In a full parse\n tree, non-terminals will never be leafs, but nodes).\n\nThe final parsing of a string is performed by the function\n``Parser.parse()``. This function returns the AST for the parsed input.\nAST are consisting of ``pynetree.Node`` objects or - in case of a\nsequence of multiple elements in the same level - lists of\n``pynetree.Node`` objects.\n\nTo walk on such an AST, the function ``pynetree.Node.dump()`` can be\nused to print the AST in a well-formatted style, or\n``pynetree.Parser.traverse()`` to possible call emitting functions on\nevery node. It is also helpful to use these functions as templates for\nother, more specialized tree traversal and walker functions.\n\n- ``pynetree.Parser.parse()`` parses an input string on the given\n grammar and returns and AST. The AST is represented as a hierarchy of\n ``pynetree.Node`` objects.\n- ``pynetree.Node.dump()`` allows for dumping ASTs returned by\n ``pynetree.Parser.parse()`` in a well-formed style.\n- ``pynetree.Parser.traverse()`` walks along an abstract syntax tree\n generated by ``pynetree.Parser.parse()``, and performs function calls\n to perform top-down, pass-by and bottom-up tree traversal\n possibilities.\n\nWhen higher AST traversal features are required for a pynetree parser,\nit is recommended to sub-class ``pynetree.Parser`` into a more specific\nclass, serving as some kind of compiler or interpreter, like this\nexample:\n\n.. code:: python\n\n import pynetree\n\n class Calculator(pynetree.Parser):\n stack = []\n\n def __init__(self):\n super(Calculator, self).__init__(\n \"\"\"\n %ignore /\\s+/;\n @INT /\\d+/;\n\n f: INT | '(' e ')';\n\n @mul: t \"*\" f;\n @div: t \"/\" f;\n t: mul | div | f;\n\n @add: e \"+\" t;\n @sub: e \"-\" t;\n e: add | sub | t;\n\n @calc$: e;\n \"\"\")\n\n def post_INT(self, node):\n self.stack.append(float(node.match))\n\n def post_add(self, node):\n self.stack.append(self.stack.pop() + self.stack.pop())\n\n def post_sub(self, node):\n x = self.stack.pop()\n self.stack.append(self.stack.pop() - x)\n\n def post_mul(self, node):\n self.stack.append(self.stack.pop() * self.stack.pop())\n\n def post_div(self, node):\n x = self.stack.pop()\n self.stack.append(self.stack.pop() / x)\n\n def post_calc(self, node):\n print(self.stack.pop())\n\n c = Calculator()\n c.traverse(c.parse(\"1337 - 42 + 23\"))\n\nPlease do also take a look at the many examples provided with pynetree\nto get familiar with these functions and possibilities.\n\nAuthor\n======\n\npynetree is developed and maintained by Jan Max Meyer, Phorward Software\nTechnologies.\n\nIt is the result of a several years experience in parser development\ntools, and is currently worked out as some kind of sister-project of the\n`Phorward Toolkit/libphorward `__,\na parsing a lexical analysis toolkit implementing LR/LALR parsers and\nfocusing on the C programming language. Therefore, the BNF-styled\ngrammar definition language of both pynetree and libphorward are similar\nand provide the same interface for both parser development tools.\n\nLicense\n=======\n\nCopyright (C) 2015-2017 by Jan Max Meyer, Phorward Software\nTechnologies.\n\nYou may use, modify and distribute this software under the terms and\nconditions of the MIT license. The full license terms are provided in\nthe LICENSE file.\n\nTHIS SOFTWARE IS PROVIDED BY JAN MAX MEYER (PHORWARD SOFTWARE\nTECHNOLOGIES) AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,\nBUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND\nFITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JAN\nMAX MEYER (PHORWARD SOFTWARE TECHNOLOGIES) BE LIABLE FOR ANY DIRECT,\nINDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)\nHOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,\nSTRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN\nANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE\nPOSSIBILITY OF SUCH DAMAGE.\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://pynetree.org", "keywords": "parsing parser parse packrat left-recursive ast syntax compiler", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pynetree", "package_url": "https://pypi.org/project/pynetree/", "platform": "", "project_url": "https://pypi.org/project/pynetree/", "project_urls": { "Homepage": "http://pynetree.org" }, "release_url": "https://pypi.org/project/pynetree/0.6/", "requires_dist": null, "requires_python": "", "summary": "A light-weight parsing toolkit written in pure Python.", "version": "0.6" }, "last_serial": 3069202, "releases": { "0.5.0": [ { "comment_text": "", "digests": { "md5": "52ae98b2eae4be34733efe21d64add6c", "sha256": "a356c3e7d0fc0b3f824673114870bcdf192e280601b794cc69e608e81581105e" }, "downloads": -1, "filename": "pynetree-0.5.0.tar.gz", "has_sig": false, "md5_digest": "52ae98b2eae4be34733efe21d64add6c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11497, "upload_time": "2017-08-02T12:50:17", "url": "https://files.pythonhosted.org/packages/0c/1c/fcb30b9ac85f0991f1af94a3a7869044222429e5c7555f590fbe80826164/pynetree-0.5.0.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "4060a96c86d688290201e72c0e1a8dc3", "sha256": "242303ab5b3cbcf0829acce30110f14942bc8a301446c3143933225eec48968e" }, "downloads": -1, "filename": "pynetree-0.5.2.tar.gz", "has_sig": false, "md5_digest": "4060a96c86d688290201e72c0e1a8dc3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12521, "upload_time": "2017-08-02T13:07:28", "url": "https://files.pythonhosted.org/packages/d8/5d/7fcf43cd05fb93698b36ccfd4d09bff3d5edf9133f7fec33f481af25e2e2/pynetree-0.5.2.tar.gz" } ], "0.6": [ { "comment_text": "", "digests": { "md5": "345f6281198b2b226ab0eb50ce698fe7", "sha256": "78477c33a06d88fcc43877265e28ab109fdde7e9bf3947bf80d795f4303e98db" }, "downloads": -1, "filename": "pynetree-0.6.tar.gz", "has_sig": false, "md5_digest": "345f6281198b2b226ab0eb50ce698fe7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12472, "upload_time": "2017-08-03T08:43:27", "url": "https://files.pythonhosted.org/packages/c2/04/5bfcc77e808d348679aa3a04e8255edbdd79fb5acd93ef499b918d660b22/pynetree-0.6.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "345f6281198b2b226ab0eb50ce698fe7", "sha256": "78477c33a06d88fcc43877265e28ab109fdde7e9bf3947bf80d795f4303e98db" }, "downloads": -1, "filename": "pynetree-0.6.tar.gz", "has_sig": false, "md5_digest": "345f6281198b2b226ab0eb50ce698fe7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12472, "upload_time": "2017-08-03T08:43:27", "url": "https://files.pythonhosted.org/packages/c2/04/5bfcc77e808d348679aa3a04e8255edbdd79fb5acd93ef499b918d660b22/pynetree-0.6.tar.gz" } ] }