{ "info": { "author": "Tom A. Thorogood", "author_email": "tom@tomthorogood.com", "bugtrack_url": null, "classifiers": [], "description": "# What is pyboro? #\n\npyboro is a regular expression lexer built with python.\n\npyboro is suitable for parsing syntax and building symbol tables,\nand it's easy to use this information to extend into building your\nown code generation and so on.\n\nThe name comes from 'borosilicate', which is the type of glass used\nin making [Pyrex](http://en.wikipedia.org/wiki/Pyrex) glassware. The repository\nwas originally called pyRex, but this conflicted with similarly named packages \nalready existing in PyPi.\n\n# Installation #\n\n pip install pyboro\n easy_install pyboro\n\nThen:\n import pyboro\nOr:\n from pyboro import Lexer\n from pyboro import Consumer\n\n# The Lexer Module #\n\nThe Lexer module (`pyboro.Lexer`) is used to define your symbol\ntables. You'll create `ParseMap` objects for each of the\nregular expressions you want to be able to parse. \n\nThese are set up in the following way:\n\n my_parser = pyboro.Lexer.ParseMap((\n (\"token name\", \"regular expression\", handler),\n #...\n ))\n\nIn the above, \"token name\" is a brief title for the regular expression.\nFor those that will be ignored, the name is irrevant for functionality,\nbut good for people who might have to maintain your code. For tokens that will\nbe preserved, the token name is how you will access the matched expression.\n \n # `let x = 17;`, where the ParseMap searches for a variable name and an assignment:\n \n result = my_parser.parse(input_string)\n print(result['variable name']) #prints 'x'\n print(result['assigment']) #prints '17'\n\n\n\"regular expression\" is the regular expression matches the title.\n\n\"handler\" is either a function that you create, OR a ParseMap constant.\n\nThe ParseMap class has two constants:\n\n pyboro.Lexer.ParseMap.IGNORE # consumes the input matching the regex, but does not store it\n pyboro.Lexer.ParseMap.LITERAL # consumes and stores the input exactly as its found\n\nIf you use the name of a function for the handler, it must take a single string as an argument,\nand output a single string. This allows you to transform or verify inputs further. Note that in python,\nbasic types are also functions. This makes it easy to convert the input strings into integers, etc\n\n (\"integer assignment\", regex_foo, int)\n \n\nFor a more in-depth look at the Lexer, read through [the tutorial](#tutorial).\n\n# The Consumer Module #\n\nWhile the Lexer module is used to set up your symbol tables, the Consumer module is used to actually\nconsume input and return result tables.\n\nA Consumer object requires a single argument a list of ParseMaps:\n\n my_consumer = Consumer.Consumer([my_parser,my_other_parser,my_third_parser])\n\nWhenever an input string is given to that object, it will iterate over the input, checking against\neach of the ParseMaps in order. It will do this until either a) all of the input is consumed (hooray!)\nor b) a syntax error is found (boo).\n\nIt will return the results from each of the ParseMaps, along with a reference to the map that found it. \n \n my_consumer = Consumer.Consumer([EmailAddress, Name, FavoriteColor])\n \n results = None\n with open(\"input_file.txt\") as f: \n results = my_consumer.parse(f.read()) #will throw an error if the entire file cannot be parsed cleanly\n \n for result in results:\n if result is EmailAddress:\n print result[\"address\"]\n\nOptional arguments to the conumser module:\n\n help: A text to be displayed if an error is found\n formatting_func: a function which takes input, formats it, and returns it.\n\nThis will produce helpful error messages like:\n\n\"Syntax error near 'dafsdfdfdjfkdljkj 12;', expecting valid type.\"\n\nwhere 'dafsdfdfdjfkdljkj 12;' is the output of formatting\\_func, and \n\"valid type\" is the help message.\n\n\n# Brief Tutorial of Lexing#\n\nBefore you begin using pyboro, you should have an idea how to use\nregular expressions with Python's `re` module. If you need help with\nthis, check out the [official documentation](http://docs.python.org/library/re.html).\n\nNext, you'll need an idea of what you'll be parsing. For this example,\nlet's say that you want to parse this line:\n\n int aNumber = 17;\n\nWe have five things here: \n\n+ `int` : a variable declaration \n+ `aNumber` : an identifier\n+ `=` : an operator\n+ `17` : an assignment to the variable\n\nWe also have whitespace between these things. \n\nWhat are the regular expressions for these things?\n\n+ `int`\n + the literal 'int'\n+ `[ \\t]+`\n + at least one space or tab\n+ `[_a-zA-Z][_a-zA-Z0-9]*`\n + A string of at least one character. It must start with an underscore or letter, but may have numbers after it.\n+ `[ \\t]*`\n + any number of spaces or tabs\n+ `\\=`\n + The assignment operator\n+ `[ \\t]*`\n + any number of spaces or tabs\n+ `[^;]+`\n + at least one character that isn't a semicolon (more on this in a second)\n+ `[ \\t]*`\n + any number of spaces or tabs\n+ `;`\n + a semicolon.\n\nThe reason we're allowing any number of things (except semicolons) in\nthe assignment is because we may not have a literal integer here. We may\nhave a function call, for instance, which returns an integer. We'll deal\nwith whatever that is at a later time (not in this example).\n\nNow that we've done that, we can create a \n\n`ParseMap`\n\nTo create a `ParseMap` of everything above, we do it like this:\n\n```python\na_parser = Lexer.ParseMap((\n (\"type_declaration\" , \"int\", Lexer.ParseMap.LITERAL),\n (\"whitespace\" , \"[ \\t]+\", Lexer.ParseMap.IGNORE),\n (\"identifier\" , \"[_a-zA-Z][_a-zA-Z0-9]*\", Lexer.ParseMap.LITERAL),\n (\"whitespace\" , \"[ \\t]*\", Lexer.ParseMap.IGNORE),\n (\"assignment_op\" , \"\\=\", Lexer.ParseMap.IGNORE),\n (\"assignment\" , \"[^;]+\", validate_as_integer),\n (\"whitespace\" , \"[ \\t]*\", Lexer.ParseMap.IGNORE),\n (\"statement_end\" , \";\", Lexer.ParseMap.IGNORE)\n))\n```\n\nWe give the ParseMap a tuple of tuples, each of which has the following:\n\n1. A short description of what the regex represents\n2. A regular expression\n3. Something that tells the parser what to do with the token found\n\nLet's examine the third column:\n\n1. `Lexer.ParseMap.LITERAL` tells the parseMap to store the data found exactly as it is stated.\n2. `Lexer.ParseMap.IGNORE` will discard the information \n3. `validate_as_integer` is a function that maybe we've already written, which takes in the \ntoken as a string, and returns some other value which will be stored. \n\nWe can then feed the parser input like so:\n results = a_parser.parse(input_string)\n\n`results` will be an `OrderedDict` whose keys are the descriptions above,\nand whose values are the parsed values (or literal values). Therefore, in this\ninstance, `results` will look like this:\n\n```python\nresults['type_declaration'] = 'int'\nresults['identifier'] = 'aNumber'\nresults['assignment'] = 17\n```", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://www.github.com/tomthorogood/pyboro", "keywords": null, "license": "GPLv3", "maintainer": null, "maintainer_email": null, "name": "pyboro", "package_url": "https://pypi.org/project/pyboro/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/pyboro/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://www.github.com/tomthorogood/pyboro" }, "release_url": "https://pypi.org/project/pyboro/0.4.1/", "requires_dist": null, "requires_python": null, "summary": "pyboro is a utility for lexing and consuming input by setting up regular expression tables and building syntax maps.", "version": "0.4.1" }, "last_serial": 867097, "releases": { "0.1.0a": [ { "comment_text": "", "digests": { "md5": "a715442cc2172a080d49874afbb0f294", "sha256": "affbeac16d7765f5b26047d2a5308602e558622798bae16257aac76d205b5f4f" }, "downloads": -1, "filename": "pyboro-0.1.0a-py2.7.egg", "has_sig": false, "md5_digest": "a715442cc2172a080d49874afbb0f294", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 13640, "upload_time": "2013-03-21T16:12:47", "url": "https://files.pythonhosted.org/packages/fb/27/5368299242d9d95143b391c0768bdab4f960a58deeaa845346cbf7ec4d9a/pyboro-0.1.0a-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "2c61722d3d86b634238f3f04db492a51", "sha256": "7dd14ade9d291dcdcfee090d15b4b1aee46e8dad2ef09ce9c9d44447a09e1dc4" }, "downloads": -1, "filename": "pyboro-0.1.0a.tar.gz", "has_sig": false, "md5_digest": "2c61722d3d86b634238f3f04db492a51", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7898, "upload_time": "2013-03-21T16:12:45", "url": "https://files.pythonhosted.org/packages/71/4b/e74de0551f764e432b7e789fa537c0fac7c612d8eb8460ae36eb49a14885/pyboro-0.1.0a.tar.gz" } ], "0.2.0a": [ { "comment_text": "", "digests": { "md5": "e596e37c13b5430f074773fb3491a94e", "sha256": "0b49e9f8e20865c7e89868861b0eba550b71d7cc618f2eef3c6c4768ef897a8d" }, "downloads": -1, "filename": "pyboro-0.2.0a-py2.7.egg", "has_sig": false, "md5_digest": "e596e37c13b5430f074773fb3491a94e", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 13728, "upload_time": "2013-07-30T23:08:59", "url": "https://files.pythonhosted.org/packages/51/9b/1f61e35ab913e172a92d4cce272675a7bf60c25cf07d6f8ff9dd4d0587b1/pyboro-0.2.0a-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "74a4b8ab6a3c7fbe1fa2720ba2479817", "sha256": "291d6646e9ea89ef20ef7b51db4c7c9c23a7143c0c676c20bfc0b1c996af7892" }, "downloads": -1, "filename": "pyboro-0.2.0a.tar.gz", "has_sig": false, "md5_digest": "74a4b8ab6a3c7fbe1fa2720ba2479817", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8086, "upload_time": "2013-07-30T23:08:56", "url": "https://files.pythonhosted.org/packages/d1/4e/a96a2b074ddc98e20806e014b6a28e44405bfa0a48f61745b954f000b24d/pyboro-0.2.0a.tar.gz" } ], "0.2.0b": [ { "comment_text": "", "digests": { "md5": "bd59348d0b0f327c7c911beaa67a7624", "sha256": "8812f92dcc1a72ce10c4a30b8eaef0813df58ae3f8278d719eecaab303f02cb6" }, "downloads": -1, "filename": "pyboro-0.2.0b-py2.7.egg", "has_sig": false, "md5_digest": "bd59348d0b0f327c7c911beaa67a7624", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 13718, "upload_time": "2013-07-30T23:10:32", "url": "https://files.pythonhosted.org/packages/82/97/9df223ec7c66c516f0f0f9c82d8196ab56ff3bbcbaa375b57fa22930160d/pyboro-0.2.0b-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "8d8946d035a0d94df3d8956adb0e7577", "sha256": "9cbe82658e8d4f2c5a7317f40e1e00cae57dbc13f68bfc62db28033070782fa6" }, "downloads": -1, "filename": "pyboro-0.2.0b.tar.gz", "has_sig": false, "md5_digest": "8d8946d035a0d94df3d8956adb0e7577", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8090, "upload_time": "2013-07-30T23:10:29", "url": "https://files.pythonhosted.org/packages/fe/88/3e14af939f7ebb197a9958b02f6198963d03a9e70b3b3bff2a2afd795f92/pyboro-0.2.0b.tar.gz" } ], "0.2.0c": [ { "comment_text": "", "digests": { "md5": "0c5d1706bc83224884021d907c67bd85", "sha256": "bdb599219dbcd29b2ecd2b2684e9d4ea10afc3afaa23d64913d4890e1d0402e3" }, "downloads": -1, "filename": "pyboro-0.2.0c-py2.7.egg", "has_sig": false, "md5_digest": "0c5d1706bc83224884021d907c67bd85", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 13734, "upload_time": "2013-07-30T23:50:09", "url": "https://files.pythonhosted.org/packages/5d/36/dcf0d59ef3d236c783587bbafc3016ad038c42658171455df36bdf1cca0d/pyboro-0.2.0c-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "6457989bd73efdf17beb081c6e74b681", "sha256": "8bd5901aadea62351427101060a05983a4c45c667d04cbeda6f75a3d12cc5958" }, "downloads": -1, "filename": "pyboro-0.2.0c.tar.gz", "has_sig": false, "md5_digest": "6457989bd73efdf17beb081c6e74b681", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8096, "upload_time": "2013-07-30T23:50:07", "url": "https://files.pythonhosted.org/packages/5f/e5/d6c67065abbcfc22e34d540dea5e2d63546f2d11fc7ac4797960ab265d56/pyboro-0.2.0c.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "a1c71d2ed49eeadd2f9c6bda807df3f3", "sha256": "6433f6f35e8cdaae05af196588751c7f3bc089621bdf2096a3f33608b298a20d" }, "downloads": -1, "filename": "pyboro-0.3.0-py2.7.egg", "has_sig": false, "md5_digest": "a1c71d2ed49eeadd2f9c6bda807df3f3", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 13955, "upload_time": "2013-09-04T21:00:48", "url": "https://files.pythonhosted.org/packages/bf/99/d452ba6fe624d57dc0b7435d9a0b57c26ed1a1413ac85a29137b2ee05173/pyboro-0.3.0-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "7a341cca1307ba796381f3d1713497ee", "sha256": "08d1a7406deeb2bdd772ba32d019e543febae08110edc8fc383e595a17df0892" }, "downloads": -1, "filename": "pyboro-0.3.0.tar.gz", "has_sig": false, "md5_digest": "7a341cca1307ba796381f3d1713497ee", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8298, "upload_time": "2013-09-04T21:00:26", "url": "https://files.pythonhosted.org/packages/da/98/6cf3c4b367c895b7dec992d39b907e3c03e7736c7ef42d5ed3dfdbcdca66/pyboro-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "a08d04b6df9cc346d896d6c8ad6bcd01", "sha256": "7ea64e48387b4b95aee83fc74b815311dd75ea457257ccc20d4a313ea4246467" }, "downloads": -1, "filename": "pyboro-0.4.0-py2.7.egg", "has_sig": false, "md5_digest": "a08d04b6df9cc346d896d6c8ad6bcd01", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 14010, "upload_time": "2013-09-11T20:23:32", "url": "https://files.pythonhosted.org/packages/24/3a/0b5508c9f69aac4d8d42b5b756c4c58b00070b439110c1a8138b7ac1b58d/pyboro-0.4.0-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "46d4b4cce58cf2d0dad0c620203caf47", "sha256": "522e9c97d9a1bebaee8d6ec7c602109d7411e266358632b287d436c544923e20" }, "downloads": -1, "filename": "pyboro-0.4.0.tar.gz", "has_sig": false, "md5_digest": "46d4b4cce58cf2d0dad0c620203caf47", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8334, "upload_time": "2013-09-11T20:23:29", "url": "https://files.pythonhosted.org/packages/0a/4b/02597eee0bf59a07e2d28ad8829a72983696c70cd625c38d2c65a5a86e50/pyboro-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "5e057ecaa2fb411d18ff98ae0d652d96", "sha256": "8ca61fa8db33bfc28eb4fa2bdee556db1a01927778491e082182e8dc68e0fbb4" }, "downloads": -1, "filename": "pyboro-0.4.1-py2.7.egg", "has_sig": false, "md5_digest": "5e057ecaa2fb411d18ff98ae0d652d96", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 14002, "upload_time": "2013-09-16T18:51:58", "url": "https://files.pythonhosted.org/packages/a9/63/93eab1f031da12b6d7f1c85625750bbbf9300532407e639a1e5799ab82ee/pyboro-0.4.1-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "776cbe9e2599b0a72d819e86c4789f54", "sha256": "14421a0c83e5902ede09d9a023db63ebb1f0b5ac7cfde30f79c70bc1d7e54ba3" }, "downloads": -1, "filename": "pyboro-0.4.1.tar.gz", "has_sig": false, "md5_digest": "776cbe9e2599b0a72d819e86c4789f54", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8343, "upload_time": "2013-09-16T18:51:56", "url": "https://files.pythonhosted.org/packages/94/2b/4c3b1a5d3f7eb55117ee32928b009be7ba5a1bde9fd4064c58bec79d3530/pyboro-0.4.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5e057ecaa2fb411d18ff98ae0d652d96", "sha256": "8ca61fa8db33bfc28eb4fa2bdee556db1a01927778491e082182e8dc68e0fbb4" }, "downloads": -1, "filename": "pyboro-0.4.1-py2.7.egg", "has_sig": false, "md5_digest": "5e057ecaa2fb411d18ff98ae0d652d96", "packagetype": "bdist_egg", "python_version": "2.7", "requires_python": null, "size": 14002, "upload_time": "2013-09-16T18:51:58", "url": "https://files.pythonhosted.org/packages/a9/63/93eab1f031da12b6d7f1c85625750bbbf9300532407e639a1e5799ab82ee/pyboro-0.4.1-py2.7.egg" }, { "comment_text": "", "digests": { "md5": "776cbe9e2599b0a72d819e86c4789f54", "sha256": "14421a0c83e5902ede09d9a023db63ebb1f0b5ac7cfde30f79c70bc1d7e54ba3" }, "downloads": -1, "filename": "pyboro-0.4.1.tar.gz", "has_sig": false, "md5_digest": "776cbe9e2599b0a72d819e86c4789f54", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8343, "upload_time": "2013-09-16T18:51:56", "url": "https://files.pythonhosted.org/packages/94/2b/4c3b1a5d3f7eb55117ee32928b009be7ba5a1bde9fd4064c58bec79d3530/pyboro-0.4.1.tar.gz" } ] }