{ "info": { "author": "F\u00e1bio Mac\u00eado Mendes", "author_email": "fabiomacedomendes@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License (GPL)", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Software Development :: Libraries" ], "description": "Ox is simple \"compiler of compilers\" framework based on the excellent PLY_\nlibrary.\n\n.. _PLY: http://www.dabeaz.com/ply/\n\n\nWhy Ox?\n=======\n\nPLY is a great library which is a reasonably efficient pure Python\nimplementation of Yacc/Bison. We think, however, its API is a little bit awkward\nand does a lot of strange magic under the hood. Ox wraps main PLY functionality\ninto a more functional and straightforward API that aims to be more explicit while\nstill being easier to use.\n\nPLY was designed to be a Python replacement for Yacc/Bison and does not offer\nany functionality to work as a general framework for building compilers. Ox\nis a minimalistic framework and provides a few extra bells and whistles (but\nit will never be nowhere near a Python replacement for, say, LLVM).\n\nOx is mature enough to be useful for production code, but just like PLY, it was\ncreated as a tool for a introductory compilers course. One explicit pedagogical\ngoal of Ox is to make the boundaries of the different compilation phases very\nexplicit and easily pluggable into each other. This approach is good for\nteaching, but it does not lead to the most efficient or robust\nimplementations of real compilers. Ox, as most compiler generators, is good for\nquick experimentation but it is limited in terms of performance and, more\nimportantly, Ox parsers generally fail to provide nice error messages for\nsyntax errors.\n \n \nWhat about the name?\n====================\n\nPLY is a Pythonic implementation/interpretation of Yacc. The most widespread\nYacc implementation is of course GNU Bison. We decided to keep the bovine \ntheme alive and call it Ox.\n\n\nConcepts\n========\n \nCompilation is usually broken in a few steps:\n\n1) Tokenization/lexical analysis: a string of source code is broken into a \n list of tokens. Ox lexers are any function that receives a string of source\n code and return a list (or any iterable) of tokens.\n2) Parsing: the list of tokens is converted into a syntax tree. In Ox, the parser\n is derived from a grammar in BNF form. It receives a list of tokens and\n outputs an arbitrary parse tree.\n3) Semantic analysis: the parse tree is scanned for semantic errors (e.g. \n invalid variable names, invalid type signatures, etc). The parse tree may\n be converted to different representations in this process.\n4) Code optimization: many optimizations are applied in order to generate \n efficient internal representations. This is highly dependent on the target\n language and runtime and it tends to be the largest part of a real compiler.\n5) Code generation: the intermediate representation is used to emit code in the\n target language. The target language is often a low level language such as\n assembly or machine code. Nothing prevents us from emmiting Python or\n Javascript, however.\n\nOx is mostly concerned with steps 1 and 2. The library has very limited support\nsteps 3 onwards, but in general they tend to be very application specific and\na general tool such as Ox can offer little help.\n\nUsage\n=====\n\nOx can build a lexer function by simply providing a list of token names\nassociated with their corresponding regular expressions:\n\n.. code-block:: python\n\n import ox\n \n lexer = ox.make_lexer([\n ('NUMBER', r'\\d+(\\.\\d*)?'),\n ('PLUS', r'\\+'),\n ('MINUS', r'\\-'),\n ('MUL', r'\\*'),\n ('DIV', r'\\/'),\n ])\n\n\nThis declares a tokenizer function that receives a string of source code and\nreturns a list of tokens:\n \n>>> lexer('21 + 21')\n[NUMBER('21'), PLUS('+'), NUMBER('21')]\n \nThe next step, of course, is to pass this list of tokens to a parser in order to \ngenerate the parse tree. We can easily declare a parser in Ox from a mapping \nof grammar rules to handler functions.\n\nEach handler function receives a number of inputs from its corresponding\ngrammar rule and return an AST node. In the example bellow, we return tuples\nto build our AST as a LISP-like S-expressions.\n\n.. code-block:: python\n\n binop = lambda x, op, y: (op, x, y)\n identity = lambda x: x\n \nNow the rules:\n\n.. code-block:: python\n\n parser = ox.make_parser([\n ('expr : expr PLUS term', binop),\n ('expr : expr MINUS term', binop),\n ('expr : term', identity),\n ('term : term MUL atom', binop),\n ('term : term DIV atom', binop),\n ('term : atom', identity),\n ('atom : NUMBER', float),\n ])\n\nThe parser takes a list of tokens and convert it to an AST:\n\n>>> parser(lexer('2 + 2 * 20'))\n('+', 2.0, ('*', 2.0, 20.0))\n\n\nThe AST makes it easy to analyze and evaluate an expression. We can\nwrite a simple evaluator as follows:\n\n.. code-block:: python\n\n import operator as op\n\n operations = {'+': op.add, '-': op.sub, '*': op.mul, '/': op.truediv}\n \n def eval(node):\n if isinstance(node, tuple):\n head, *tail = node\n func = operations[head]\n args = (eval(x) for x in tail)\n return func(*args)\n else:\n return node\n\n\nThe eval function receives an AST, but we can easily compose it with the other\nfunctions in order to accept string inputs. (Ox functions understand sidekick's \npipeline operators. The arrow operator ``>>`` composes two functions by passing\nthe output of each function to the function in the pipeline following the arrow\ndirection).\n\n>>> eval_input = lexer >> parser >> eval\n>>> eval_input('2 + 2 * 20')\n42.0\n\nWe can call this function in a loop to have a nice calculator written with only\na few lines of Python code!\n\n.. code-block:: python\n\n def eval_loop():\n expr = input('expr: ')\n print('result:', eval_input(expr))\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/fabiommendes/ox/", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "ox-parser", "package_url": "https://pypi.org/project/ox-parser/", "platform": "any", "project_url": "https://pypi.org/project/ox-parser/", "project_urls": { "Homepage": "https://github.com/fabiommendes/ox/" }, "release_url": "https://pypi.org/project/ox-parser/0.2.4/", "requires_dist": null, "requires_python": "", "summary": "A simplified API to PLY (Python Lex & Yacc).", "version": "0.2.4" }, "last_serial": 3849526, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "2575a9ec1f6428d1efd76dc21a4ba527", "sha256": "5166646710cdfe27981bea4991566c8f4057d0e79ef54be4262c8baf53b7f86d" }, "downloads": -1, "filename": "ox-parser-0.1.0.tar.gz", "has_sig": false, "md5_digest": "2575a9ec1f6428d1efd76dc21a4ba527", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12607, "upload_time": "2017-08-08T01:31:34", "url": "https://files.pythonhosted.org/packages/80/ce/984a22182d44950bf013dbe442c74079d171621022cbf6964f3f54d6f577/ox-parser-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "123e1aec4cb6070f2d22ae88eddd2690", "sha256": "f0b83fbcc91444aff7f3111ed82547435152d8366e05840ff0e13b531e766fe5" }, "downloads": -1, "filename": "ox-parser-0.1.1.tar.gz", "has_sig": false, "md5_digest": "123e1aec4cb6070f2d22ae88eddd2690", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12703, "upload_time": "2017-08-08T01:54:05", "url": "https://files.pythonhosted.org/packages/27/be/910f0f18da43acc55036d7d860f5989eb336627c6f9b1a322d722a066f9c/ox-parser-0.1.1.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "7c2c11aa7df61cc8f719a1ef32b2a0c6", "sha256": "ba97beb6c4f8f2e09793c5c3573776c98337123106dc5fee0f2100df67629d1e" }, "downloads": -1, "filename": "ox-parser-0.2.1.tar.gz", "has_sig": false, "md5_digest": "7c2c11aa7df61cc8f719a1ef32b2a0c6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 40299, "upload_time": "2018-05-08T16:55:41", "url": "https://files.pythonhosted.org/packages/ff/7e/24761c083c4ba19b10059917176e67f0c28436ffaf1ea8ff636370d791c1/ox-parser-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "bcc148f6e641f6c4d7ee24ff04269081", "sha256": "ddb8e12052bc23304a17b7c71eebc6ba6493d6550da6cf0927fd13bc70db96b0" }, "downloads": -1, "filename": "ox-parser-0.2.2.tar.gz", "has_sig": false, "md5_digest": "bcc148f6e641f6c4d7ee24ff04269081", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 40475, "upload_time": "2018-05-10T03:49:46", "url": "https://files.pythonhosted.org/packages/e4/5f/a1303c1e16430a2519a3ac9126053b4f8201bbb1a8232ccbbbb36a9b4103/ox-parser-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "8bfb0cad7a1ad229114c2089436d6283", "sha256": "57dde4d0a9eafc7b0d4cc7dcc8f2c0dda97c1b7be672f7251ce53513b00a4c82" }, "downloads": -1, "filename": "ox-parser-0.2.3.tar.gz", "has_sig": false, "md5_digest": "8bfb0cad7a1ad229114c2089436d6283", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 40487, "upload_time": "2018-05-10T04:01:40", "url": "https://files.pythonhosted.org/packages/19/96/5d60d59d2042b09c0ecdc4b9760f27286bdf3512d54dc16c4ba44c4936cd/ox-parser-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "f806eaa865f584cc2cdaac8c695ebe14", "sha256": "bfac422986043e944de77968ee32858d28afd9895ffe739d54cea9bd106fb71e" }, "downloads": -1, "filename": "ox-parser-0.2.4.tar.gz", "has_sig": false, "md5_digest": "f806eaa865f584cc2cdaac8c695ebe14", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 40488, "upload_time": "2018-05-10T05:25:42", "url": "https://files.pythonhosted.org/packages/4a/a5/02cc7483e23572e918567b1c4b5866c6ffb4f626b72d9319db5f245bd40c/ox-parser-0.2.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f806eaa865f584cc2cdaac8c695ebe14", "sha256": "bfac422986043e944de77968ee32858d28afd9895ffe739d54cea9bd106fb71e" }, "downloads": -1, "filename": "ox-parser-0.2.4.tar.gz", "has_sig": false, "md5_digest": "f806eaa865f584cc2cdaac8c695ebe14", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 40488, "upload_time": "2018-05-10T05:25:42", "url": "https://files.pythonhosted.org/packages/4a/a5/02cc7483e23572e918567b1c4b5866c6ffb4f626b72d9319db5f245bd40c/ox-parser-0.2.4.tar.gz" } ] }