{ "info": { "author": "Sri Panyam", "author_email": "sri.panyam@gmail.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Topic :: Software Development :: Code Generators", "Topic :: Software Development :: Compilers" ], "description": "\nastgen - A tool for generating Abstract Syntaxt Trees\n=====================================================\n\nIntroduction\n------------\n\nThis tools allows language and compiler developers to generate Abstract Syntax Trees that can be \nused by parsers (or parser generators) to record the result of a parse phase. The input is a set\nof python classes which describe the methods and members of the AST classes specific to the parser\nand outputs language specific (so far Java and C++) classes with the definitions of the various\nAST classes.\n\nInstallation\n------------\n\n```\npip install astgen\n```\n\nor from github\n\n```\ngit clone [git-repo-url] astgen\ncd astgen\npython setup.py install\n```\n\nGetting Started\n---------------\n\nUpon installation, a script called \"astgen\" is created. To see its usage, invoke it with the -h option:\n\n```\nbash-3.2$ astgen -h\nUsage: astgen [options]\n\nOptions:\n -h, --help show this help message and exit\n -o OUTPUTDIR, --outputdir=OUTPUTDIR\n The output folder in which all generted files are\n written to.\n -l LAYOUT_BACKEND, --layout_backend=LAYOUT_BACKEND\n The backend to decide the package layout of the code\n being generated, eg single file, multiple files etc\n -p PLATFORM_BACKEND, --platform_backend=PLATFORM_BACKEND\n The backend to help with the platform to target code\n genration for, eg java, python, stl, C etc\n -c BACKEND_CONFIG, --data=BACKEND_CONFIG\n Data required any custom data that can be used/passed\n to the platform or layout backends.\n -m MODEL_FILE, --modelfile=MODEL_FILE\n Input model file containing AST definitions for which\n AST code is to be generated.\n```\n\n\nThe MODEL_FILE and BACKEND_CONFIG parameters are mandatory while the Platform parameter defaults to C++ (ie outputs are C++ classes) and a two file layout is used where all node declarations are writen to the header (.h) file and the definitions are written to the implementation file (.cpp).\n\nSample Usage\n\n------------\nTo try out the /alculator example:\n\n```\ncd samples/calculator\nastgen -o /tmp/calculator -m model.py -c cpp_config.py\n```\n\nThis would now generate all the files for the various AST nodes\nassociated with the sample in the folder /tmp/calculator\n\nModels\n------\n\nModels define the structure of AST nodes that are to be generated. A model looks similar to:\n\n```\nclass NodeClass(ASTNode):\n properties = dict(member1 = MemberType1,\n member2 = MemberType2,...)\n```\n\nNodes can also inherit from other defined nodes, eg:\n\n```\n\nOperator = EnumType(\"Operator\", \"PLUS\", \"MINUS\", \"MUL\", \"DIV\")\n\nclass Expression(ASTNode): pass\n\nclass BinaryExpression(Expresssion):\n \"\"\"\n Has a binary operator and left and right sub expressions.\n \"\"\"\n properties = dict(op = Operator, \n lhs = \"Expression\",\n rhs = \"Expression\")\n\nclass UnaryExpression(Expression):\n \"\"\"\n Has a prefix operator and an expression\n \"\"\"\n properties = dict(op = Operator, \n child = \"Expression\")\n```\n\nTypes will be discussed in next section.\n\nConfig\n------\n\nThe config file (specified with the -c parameter) contains overrides and other data used by the various parts of astgen while generating the AST code. The different parameters depend on the layout and platform backends used and will be described in the respective sections.\n\nTypes\n------\n\nIn the Models section, nodes and properties were discussed. Each property must have a type. The type value of a property can be a String which indicates a reference to a value of another defined class (in the above case an \"Expression\") or it can be one of the following types:\n\n\n#### BasicType(\"type\")\n\nDefines a native type that is platform specific and will resolved by the PlatformBackend object. For instance:\n\n```\nInteger = BasicType(\"int\")\n```\n\ndefines a type called Integer which will is of type \"int\" and can be equal to an int variable in C/C++ or in Java. \n\nNote that this does not mean that only primitive types can be defined in this manner. The use of BasicType is only a hint that the underlying type (and its string representation that is rendered in the final output) is determined by the PlatformBackend. For a more complicated examle the following can also be defined:\n\n```\nPointListPtr = BasicType(\"PointListPtr\")\n```\n\nand in return, this could be resolved to (in C++) as:\n\n```\nstd::shared_ptr>\n```\n\nHow these types are resolved will be discussed in the PlatformBackend section.\n\n#### EnumType(TypeName, EnumVal1, EnumVal2, EnumVal3....)\n\nDefines an Enum type similar to the following:\n\n```\nenum TypeName\n{\n EnumVal1,\n EnumVal2,\n EnumVal3\n ...\n}\n```\n\n#### PairOf(Type1, Type2, TypeName = '')\n\nDefines a Pair type over the given two sub types. In C++ this would be similar to:\n\n```\ntypedef std::pair TypeName\n```\n\nIf the TypeName is not specified, a name is automatically generated and used.\n\n#### ListOf(BaseType, TypeName = '')\n\nDefines a type that is the List of a BaseType typed values. In C++ this would be similar to:\n\n```\ntypedef std::list TypeName\n```\n\nIf the TypeName is not specified, a name is automatically generated and used.\n\n#### MapOf(KeyType, ValueType)\n\n\nDefines a type that is the Map with the key and value types given by KeyType and ValueType respectively. In C++ this would be similar to:\n\n```\ntypedef std::map TypeName\n```\n\nIf the TypeName is not specified, a name is automatically generated and used.\n\nPlatforms\n---------\n\nThe Platform objects are responsible for handling and delegating all concerns related to the specific platform the ASTs are being generated for (eg C++, Java etc). For now a Platform object only provides one method:\n\n```\ndef evalType(self, typeobj): pass\n```\n\nThis method is responsible for returning the string representation of a type object (eg those discussed in the Types section) specific to the platform. To achieve the results above, the astgen.platforms.CPlusPlus backend looks like this:\n\n```\nclass CPlusPlus(astgen.ASTPlatform):\n def evalType(self, typeobj):\n if type(typeobj) is str:\n return typeobj + \"Ptr\"\n elif type(typeobj) is astgen.BasicType:\n if typeobj.typename == \"boolean\": return \"bool\"\n if typeobj.typename == \"string\": return \"std::string\"\n return typeobj.typename\n elif type(typeobj) is astgen.ListOf:\n return \"std::list<%s>\" % self.evalType(typeobj.base_type)\n elif type(typeobj) is astgen.PairOf:\n return \"std::pair<%s,%s>\" % (self.evalType(typeobj.type1), self.evalType(typeobj.type2))\n elif type(typeobj) is astgen.MapOf:\n return \"std::map<%s,%s>\" % (self.evalType(typeobj.key_type), self.evalType(typeobj.value_type))\n elif type(typeobj) is astgen.EnumType:\n return typeobj.enum_name\n return super(CPlusPlus, self).evalType(typeobj)\n```\n\nCustom platform backends can be provided to the astgen script with the -p parameter.\n\nAlternatively, it can be defined in the Config(.py) file itself.\n\nSo far the following (hopefully self explanatory) platforms are defined:\n\n#### astgen.platforms.CPlusPlus\n\n#### astgen.platforms.Java\n\nLayouts\n-------\n\nRegardless of the platform, there would be several ways to layout the generate nodes. For instance, for C++ alone, the following (and several more) layouts are possible:\n\n - Monolithic header file: A single .h file that would contain all class definitions along with their implementations.\n - Two files: Broken down into one header file (containing all the interface/class declarations) and an implementation file (containing all the class/implementation definitions).\n - Two files per class: Each class with its own header and implementation files.\n\nThere could be several other layouts depending on the needs of the project. All these layouts inherit from the ASTLayout base. The ASTLayout base has the following methods:\n\n```\nclass ASTLayout(object):\n \"\"\"\n Given an AST node generates the code for the node. This can be used to \n generate AST code for different languages or platforms.\n \"\"\"\n\n def orderNodes(self, nodes):\n \"\"\"\n Called to order all nodes in any way as deemed necessary.\n \"\"\"\n pass\n\n def generationStarted(self, nodelist):\n \"\"\"\n Called before starting node generation for any of the nodes.\n \"\"\"\n pass\n\n def generationFinished(self, nodelist):\n \"\"\"\n Called after the code generation of all nodes has completed.\n \"\"\"\n pass\n\n def nodeStarted(self, node):\n \"\"\"\n Called before the generation of code for a particular node.\n \"\"\"\n pass\n\n def renderNodes(self, nodelist):\n \"\"\"\n Called to render all nodes\n \"\"\"\n pass\n\n def renderNode(self, node):\n \"\"\"\n Called to render a particular node.\n \"\"\"\n pass\n\n def nodeFinished(self, node):\n \"\"\"\n Called after the generation of code for a particular node.\n \"\"\"\n pass\n\n```\n\nShown below are the laout classes that are defined so far (along with the parameters they accept in the config file):\n\n##### astgen.layouts.OneFileLayout\n\nThe monolithic one file layout where all class are declared and defined in the same file. The following options in the config file are accepted:\n\n- HEADER_OUTPUT: The output file to which all classes will be written to.\n- HEADER_TEMPLATE: The (jinja2) template file that will be used to render all the classes (defaults to \"cpp_header\").\n\n##### astgen.layouts.TwoFilesLayout\n\nThe two files layout where all class interface declarations are written to a header and implementations/definitions are written to an implementation file (eg .cpp, .m etc). The following options in the config file are accepted:\n\n- HEADER_OUTPUT: The output file to which all classes declarations will be written to.\n- HEADER_TEMPLATE: The (jinja2) template file that will be used to render all the classes (defaults to \"cpp_header\").\n- IMPLEMENTATION_OUTPUT: The output file to which all classes definitions/implementations will be written to.\n- IMPLEMENTATION_TEMPLATE: The (jinja2) template file that will be used to render all the classes implementaitons/definitions (defaults to \"cpp_implementation\").\n\n##### astgen.layouts.TwoFilesPerNodeLayout\n\nIn this layout, each node is written to its own file (typical\n.h and ). Additionally, a forward defs file, a\npublic headers file and a header file for all enums is also generated.\nThese are denoted by:\n\n- FWDDEFS_OUTPUT: The output file to which all forward definitions are written (optional).\n- ENUMS_OUTPUT: The output file to which all enums are written (optional).\n- PUBLIC_OUTPUT: The output file to which all include headers are written to (optional).\n\n- FWDDEFS_TEMPLATE: The (jinja2) template file that will be used to render the forward definitions (if used).\n- ENUMS_TEMPLATE: The (jinja2) template file that will be used to render the enum definitions (if used).\n- PUBLIC_TEMPLATE: The (jinja2) template file that will be used to render the public header includes (if used).\n\nNote that the above are all optional and not required in all cases (for\ninstance for Java none of the above are required).\n\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/panyam/astgen/", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "astgen", "package_url": "https://pypi.org/project/astgen/", "platform": "", "project_url": "https://pypi.org/project/astgen/", "project_urls": { "Homepage": "http://github.com/panyam/astgen/" }, "release_url": "https://pypi.org/project/astgen/0.1.6.13/", "requires_dist": [ "jinja2 (>=2.7)" ], "requires_python": "", "summary": "A generator for Abstract Syntax Trees.", "version": "0.1.6.13" }, "last_serial": 4803271, "releases": { "0.1.6.10": [ { "comment_text": "", "digests": { "md5": "bbf7f3e6c1b7adc2f67783e5ae4a37e0", "sha256": "35891e5f95224de5dc9765a05c8680dc5adb73dac0aa207db217e7fa5a370fec" }, "downloads": -1, "filename": "astgen-0.1.6.10.tar.gz", "has_sig": false, "md5_digest": "bbf7f3e6c1b7adc2f67783e5ae4a37e0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14337, "upload_time": "2014-08-13T06:52:03", "url": "https://files.pythonhosted.org/packages/32/f5/3883bb83dcf567c38bf7599cec9adba7a064b4174356219a64aa4732257c/astgen-0.1.6.10.tar.gz" } ], "0.1.6.11": [ { "comment_text": "", "digests": { "md5": "793a3491309ece1f79b2e291683b0b46", "sha256": "7225bcc206695a056d3121c5f1d201dd0ca1fae77bd9e5715d7694880216ab60" }, "downloads": -1, "filename": "astgen-0.1.6.11.tar.gz", "has_sig": false, "md5_digest": "793a3491309ece1f79b2e291683b0b46", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14331, "upload_time": "2014-08-13T06:54:58", "url": "https://files.pythonhosted.org/packages/f1/43/6948800aafd6a59a6f257f93def064e304db0fdcab982d8eeb0b198409d1/astgen-0.1.6.11.tar.gz" } ], "0.1.6.12": [ { "comment_text": "", "digests": { "md5": "e3581284e80c90b20ac74e9a2f7683dd", "sha256": "07388b433894dd65d387cd48537bb56b20cff84d958c565fd5ce5831762335d8" }, "downloads": -1, "filename": "astgen-0.1.6.12-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e3581284e80c90b20ac74e9a2f7683dd", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19458, "upload_time": "2019-02-10T20:50:55", "url": "https://files.pythonhosted.org/packages/fe/4f/cb77bff635926bba0b9ec5ef99c506ea78862d15a30a2b64f5ef67f3c5d1/astgen-0.1.6.12-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d2b37106d0ab2e233556527703feec5d", "sha256": "56dbd28957427e2cfeca565871d829dd02cb6e89ad0c6b24f99760be81a67a94" }, "downloads": -1, "filename": "astgen-0.1.6.12.tar.gz", "has_sig": false, "md5_digest": "d2b37106d0ab2e233556527703feec5d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15135, "upload_time": "2014-08-13T06:56:46", "url": "https://files.pythonhosted.org/packages/41/10/932497daee137dd4b71950d902062c83a655908a61e6e907008105fee682/astgen-0.1.6.12.tar.gz" } ], "0.1.6.13": [ { "comment_text": "", "digests": { "md5": "2f24a5a78bb8a71be722a237af7e81e2", "sha256": "11e08a3e348a76a8a7e7bb346ff5c4177993dd24819257fbde60b2fc08d2aa00" }, "downloads": -1, "filename": "astgen-0.1.6.13-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "2f24a5a78bb8a71be722a237af7e81e2", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19455, "upload_time": "2019-02-10T20:51:47", "url": "https://files.pythonhosted.org/packages/2d/37/60257947beebe1a28755b7b82cfa2d28cc4a4bf9780ba06bcc812fe3f2bb/astgen-0.1.6.13-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ce8da9c00933a3f7e3a052d36b898c87", "sha256": "91df1b736cb6d7829fec12b6c6e5bed03c6c486948eb54f638501166cfb68f4b" }, "downloads": -1, "filename": "astgen-0.1.6.13.tar.gz", "has_sig": false, "md5_digest": "ce8da9c00933a3f7e3a052d36b898c87", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19545, "upload_time": "2019-02-10T20:51:49", "url": "https://files.pythonhosted.org/packages/a3/3b/08c069779e5deeb455b0bf2f531fab1b9d7987f900dcfb01a55b26dc7ef1/astgen-0.1.6.13.tar.gz" } ], "0.1.6.2": [ { "comment_text": "", "digests": { "md5": "1bce04984b8620fd4dbf371e936a54ca", "sha256": "81c599b4d0007b02c6a4eb42cb6c9b7d941f04eb1457fb16a202f68f8f2200cf" }, "downloads": -1, "filename": "astgen-0.1.6.2.tar.gz", "has_sig": false, "md5_digest": "1bce04984b8620fd4dbf371e936a54ca", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14033, "upload_time": "2014-08-05T06:37:04", "url": "https://files.pythonhosted.org/packages/cd/0f/802a1501a142011c3349aea6fb5ae2f39be36b8bf3a3920d938f73c393fb/astgen-0.1.6.2.tar.gz" } ], "0.1.6.3": [ { "comment_text": "", "digests": { "md5": "041ee900588e41d389fc864c04cce533", "sha256": "6e1c113901c948249ddba0e7fca588b17b9ca4d60ab182039823926b294537f1" }, "downloads": -1, "filename": "astgen-0.1.6.3.tar.gz", "has_sig": false, "md5_digest": "041ee900588e41d389fc864c04cce533", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14265, "upload_time": "2014-08-05T07:42:24", "url": "https://files.pythonhosted.org/packages/2d/1b/19b257478778ee1643c90c23d545a07139df167b05a2553fe879de39a42c/astgen-0.1.6.3.tar.gz" } ], "0.1.6.4": [ { "comment_text": "", "digests": { "md5": "76227e05c9390ebef903a615f48c5dbb", "sha256": "e609c139e8a908494c384b067fae14bbb60f1325343d0d4d9e1c6ba2d68c32d6" }, "downloads": -1, "filename": "astgen-0.1.6.4.tar.gz", "has_sig": false, "md5_digest": "76227e05c9390ebef903a615f48c5dbb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14199, "upload_time": "2014-08-10T08:07:49", "url": "https://files.pythonhosted.org/packages/40/db/f26e9474648636a26ec9048d071487fd1035e7e565091cc84dc5812088d7/astgen-0.1.6.4.tar.gz" } ], "0.1.6.5": [ { "comment_text": "", "digests": { "md5": "80bf3c0b4547fc1593254f3de3293665", "sha256": "25c77bbc2dc2ad753d366e0ad22cbd584ca8aeb21f4edc6a75af3b3c23768bf9" }, "downloads": -1, "filename": "astgen-0.1.6.5.tar.gz", "has_sig": false, "md5_digest": "80bf3c0b4547fc1593254f3de3293665", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14220, "upload_time": "2014-08-13T06:43:55", "url": "https://files.pythonhosted.org/packages/35/29/31486de92545f40d268ea1cbd006230680a9c704194fe9586925b80d2f4d/astgen-0.1.6.5.tar.gz" } ], "0.1.6.6": [ { "comment_text": "", "digests": { "md5": "64177409e9eac7a21b80519dd0d1bb3f", "sha256": "60511716386b235d75e5022a84e9e8ad36f9a198f70e9493c4d37b67d8c2282b" }, "downloads": -1, "filename": "astgen-0.1.6.6.tar.gz", "has_sig": false, "md5_digest": "64177409e9eac7a21b80519dd0d1bb3f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14231, "upload_time": "2014-08-13T06:45:31", "url": "https://files.pythonhosted.org/packages/f0/ce/cc0fa579ee49151bb32cecbadd9cea77d93bc9f826b968289c64442802b5/astgen-0.1.6.6.tar.gz" } ], "0.1.6.7": [ { "comment_text": "", "digests": { "md5": "e133e601abdf6b475a629dafa486db79", "sha256": "c8e1ff0cd571f78fd91ab85c694cd728fbbffdb5148ee274624c221841323584" }, "downloads": -1, "filename": "astgen-0.1.6.7.tar.gz", "has_sig": false, "md5_digest": "e133e601abdf6b475a629dafa486db79", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14244, "upload_time": "2014-08-13T06:46:59", "url": "https://files.pythonhosted.org/packages/f9/cc/1365711eae1a6dad8216684bf202d997f5409ab394ceac9acce19249fbfc/astgen-0.1.6.7.tar.gz" } ], "0.1.6.9": [ { "comment_text": "", "digests": { "md5": "bfdf0becb2df949e69f28e2931f2adab", "sha256": "37ce1e5c9a71f0715528751953d55b71b24116d96406270316ad0dccc975d144" }, "downloads": -1, "filename": "astgen-0.1.6.9.tar.gz", "has_sig": false, "md5_digest": "bfdf0becb2df949e69f28e2931f2adab", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14224, "upload_time": "2014-08-13T06:49:40", "url": "https://files.pythonhosted.org/packages/47/af/52d9cd098b2797e48f519a3297a7af5e617e4ed87988cda42dcbbd865eb1/astgen-0.1.6.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2f24a5a78bb8a71be722a237af7e81e2", "sha256": "11e08a3e348a76a8a7e7bb346ff5c4177993dd24819257fbde60b2fc08d2aa00" }, "downloads": -1, "filename": "astgen-0.1.6.13-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "2f24a5a78bb8a71be722a237af7e81e2", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19455, "upload_time": "2019-02-10T20:51:47", "url": "https://files.pythonhosted.org/packages/2d/37/60257947beebe1a28755b7b82cfa2d28cc4a4bf9780ba06bcc812fe3f2bb/astgen-0.1.6.13-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ce8da9c00933a3f7e3a052d36b898c87", "sha256": "91df1b736cb6d7829fec12b6c6e5bed03c6c486948eb54f638501166cfb68f4b" }, "downloads": -1, "filename": "astgen-0.1.6.13.tar.gz", "has_sig": false, "md5_digest": "ce8da9c00933a3f7e3a052d36b898c87", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19545, "upload_time": "2019-02-10T20:51:49", "url": "https://files.pythonhosted.org/packages/a3/3b/08c069779e5deeb455b0bf2f531fab1b9d7987f900dcfb01a55b26dc7ef1/astgen-0.1.6.13.tar.gz" } ] }