{ "info": { "author": "Joe Jevnik and Scott Sanderson", "author_email": "joejev@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "License :: OSI Approved :: GNU General Public License v2 (GPLv2)", "Natural Language :: English", "Operating System :: POSIX", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Software Development :: Pre-processors" ], "description": "``codetransformer``\n===================\n\n|build status| |documentation|\n\nBytecode transformers for CPython inspired by the ``ast`` module's\n``NodeTransformer``.\n\nWhat is ``codetransformer``?\n----------------------------\n\n``codetransformer`` is a library that allows us to work with CPython's bytecode\nrepresentation at runtime. ``codetransformer`` provides a level of abstraction\nbetween the programmer and the raw bytes read by the eval loop so that we can\nmore easily inspect and modify bytecode.\n\n``codetransformer`` is motivated by the need to override parts of the python\nlanguage that are not already hooked into through data model methods. For example:\n\n* Override the ``is`` and ``not`` operators.\n* Custom data structure literals.\n* Syntax features that cannot be represented with valid python AST or source.\n* Run without a modified CPython interpreter.\n\n``codetransformer`` was originally developed as part of lazy_ to implement\nthe transformations needed to override the code objects at runtime.\n\nExample Uses\n------------\n\nOverloading Literals\n~~~~~~~~~~~~~~~~~~~~\n\nWhile this can be done as an AST transformation, we will often need to execute\nthe constructor for the literal multiple times. Also, we need to be sure that\nany additional names required to run our code are provided when we run. With\n``codetransformer``, we can pre compute our new literals and emit code that is\nas fast as loading our unmodified literals without requiring any additional\nnames be available implicitly.\n\nIn the following block we demonstrate overloading dictionary syntax to result in\n``collections.OrderedDict`` objects. ``OrderedDict`` is like a ``dict``;\nhowever, the order of the keys is preserved.\n\n.. code-block:: python\n\n >>> from codetransformer.transformers.literals import ordereddict_literals\n >>> @ordereddict_literals\n ... def f():\n ... return {'a': 1, 'b': 2, 'c': 3}\n >>> f()\n OrderedDict([('a', 1), ('b', 2), ('c', 3)])\n\nThis also supports dictionary comprehensions:\n\n.. code-block:: python\n\n >>> @ordereddict_literals\n ... def f():\n ... return {k: v for k, v in zip('abc', (1, 2, 3))}\n >>> f()\n OrderedDict([('a', 1), ('b', 2), ('c', 3)])\n\nThe next block overrides ``float`` literals with ``decimal.Decimal``\nobjects. These objects support arbitrary precision arithmetic.\n\n.. code-block:: python\n\n >>> from codetransformer.transformers.literals import decimal_literals\n >>> @decimal_literals\n ... def f():\n ... return 1.5\n >>> f()\n Decimal('1.5')\n\nPattern Matched Exceptions\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nPattern matched exceptions are a good example of a ``CodeTransformer`` that\nwould be very complicated to implement at the AST level. This transformation\nextends the ``try/except`` syntax to accept instances of ``BaseException`` as\nwell subclasses of ``BaseException``. When excepting an instance, the ``args``\nof the exception will be compared for equality to determine which exception\nhandler should be invoked. For example:\n\n.. code-block:: python\n\n >>> @pattern_matched_exceptions()\n ... def foo():\n ... try:\n ... raise ValueError('bar')\n ... except ValueError('buzz'):\n ... return 'buzz'\n ... except ValueError('bar'):\n ... return 'bar'\n >>> foo()\n 'bar'\n\nThis function raises an instance of ``ValueError`` and attempts to catch it. The\nfirst check looks for instances of ``ValueError`` that were constructed with an\nargument of ``'buzz'``. Because our custom exception is raised with ``'bar'``,\nthese are not equal and we do not enter this handler. The next handler looks for\n``ValueError('bar')`` which does match the exception we raised. We then enter\nthis block and normal python rules take over.\n\nWe may also pass their own exception matching function:\n\n.. code-block:: python\n\n >>> def match_greater(match_expr, exc_type, exc_value, exc_traceback):\n ... return math_expr > exc_value.args[0]\n\n >>> @pattern_matched_exceptions(match_greater)\n ... def foo():\n ... try:\n ... raise ValueError(5)\n ... except 4:\n ... return 4\n ... except 5:\n ... return 5\n ... except 6:\n ... return 6\n >>> foo()\n 6\n\nThis matches on when the match expression is greater in value than the first\nargument of any exception type that is raised. This particular behavior would be\nvery hard to mimic through AST level transformations.\n\nCore Abstractions\n-----------------\n\nThe three core abstractions of ``codetransformer`` are:\n\n1. The ``Instruction`` object which represents an opcode_ which may be paired\n with some argument.\n2. The ``Code`` object which represents a collection of ``Instruction``\\s.\n3. The ``CodeTransformer`` object which represents a set of rules for\n manipulating ``Code`` objects.\n\nInstructions\n~~~~~~~~~~~~\n\nThe ``Instruction`` object represents an atomic operation that can be performed\nby the CPython virtual machine. These are things like ``LOAD_NAME`` which loads\na name onto the stack, or ``ROT_TWO`` which rotates the top two stack elements.\n\nSome instructions accept an argument, for example ``LOAD_NAME``, which modifies\nthe behavior of the instruction. This is much like a function call where some\nfunctions accept arguments. Because the bytecode is always packed as raw bytes,\nthe argument must be some integer (CPython stores all arguments two in bytes).\nThis means that things that need a more rich argument system (like ``LOAD_NAME``\nwhich needs the actual name to look up) must carry around the actual arguments\nin some table and use the integer as an offset into this array. One of the key\nabstractions of the ``Instruction`` object is that the argument is always some\npython object that represents the actual argument. Any lookup table management\nis handled for the user. This is helpful because some arguments share this table\nso we don't want to add extra entries or forget to add them at all.\n\nAnother annoyance is that the instructions that handle control flow use their\nargument to say what bytecode offset to jump to. Some jumps use the absolute\nindex, others use a relative index. This also makes it hard if you want to add\nor remove instructions because all of the offsets must be recomputed. In\n``codetransformer``, the jump instructions all accept another ``Instruction`` as\nthe argument so that the assembler can manage this for the user. We also provide\nan easy way for new instructions to \"steal\" jumps that targeted another\ninstruction so that can manage altering the bytecode around jump targets.\n\nCode\n~~~~\n\n``Code`` objects are a nice abstraction over python's\n``types.CodeType``. Quoting the ``CodeType`` constructor docstring:\n\n::\n\n code(argcount, kwonlyargcount, nlocals, stacksize, flags, codestring,\n constants, names, varnames, filename, name, firstlineno,\n lnotab[, freevars[, cellvars]])\n\n Create a code object. Not for the faint of heart.\n\nThe ``codetransformer`` abstraction is designed to make it easy to dynamically\nconstruct and inspect these objects. This allows us to easy set things like the\nargument names, and manipulate the line number mappings.\n\nThe ``Code`` object provides methods for converting to and from Python's code\nrepresentation:\n\n1. ``from_pycode``\n2. ``to_pycode``.\n\nThis allows us to take an existing function, parse the meaning from it, modify\nit, and then assemble this back into a new python code object.\n\n.. note::\n\n ``Code`` objects are immutable. When we say \"modify\", we mean create a copy\n with different values.\n\nCodeTransformers\n----------------\n\nThis is the set of rules that are used to actually modify the ``Code``\nobjects. These rules are defined as a set of ``patterns`` which are a DSL used\nto define a DFA for matching against sequences of ``Instruction`` objects. Once\nwe have matched a segment, we yield new instructions to replace what we have\nmatched. A simple codetransformer looks like:\n\n.. code-block:: python\n\n from codetransformer import CodeTransformer, instructions\n\n class FoldNames(CodeTransformer):\n @pattern(\n instructions.LOAD_GLOBAL,\n instructions.LOAD_GLOBAL,\n instructions.BINARY_ADD,\n )\n def _load_fast(self, a, b, add):\n yield instructions.LOAD_FAST(a.arg + b.arg).steal(a)\n\nThis ``CodeTransformer`` uses the ``+`` operator to implement something like\n``CPP``\\s token pasting for local variables. We read this pattern as a sequence\nof two ``LOAD_GLOBAL`` (global name lookups) followed by a ``BINARY_ADD``\ninstruction (``+`` operator call). This will then call the function with the\nthree instructions passed positionally. This handler replaces this sequence with\na single instruction that emits a ``LOAD_FAST`` (local name lookup) that is the\nresult of adding the two names together. We then steal any jumps that used to\ntarget the first ``LOAD_GLOBAL``.\n\nWe can execute this transformer by calling an instance of it on a\nfunction object, or using it like a decorator. For example:\n\n.. code-block:: python\n\n >>> @FoldNames()\n ... def f():\n ... ab = 3\n ... return a + b\n >>> f()\n 3\n\n\nLicense\n-------\n\n``codetransformer`` is free software, licensed under the GNU General Public\nLicense, version 2. For more information see the ``LICENSE`` file.\n\n\nSource\n------\n\nSource code is hosted on github at\nhttps://github.com/llllllllll/codetransformer.\n\n\n.. _lazy: https://github.com/llllllllll/lazy_python\n.. _opcode: https://docs.python.org/3.5/library/dis.html#opcode-NOP\n.. |build status| image:: https://travis-ci.org/llllllllll/codetransformer.svg?branch=master\n :target: https://travis-ci.org/llllllllll/codetransformer\n.. |documentation| image:: https://readthedocs.org/projects/codetransformer/badge/?version=stable\n :target: http://codetransformer.readthedocs.io/en/stable/?badge=stable\n :alt: Documentation Status", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/llllllllll/codetransformer", "keywords": "", "license": "GPL-2", "maintainer": "", "maintainer_email": "", "name": "codetransformer", "package_url": "https://pypi.org/project/codetransformer/", "platform": "", "project_url": "https://pypi.org/project/codetransformer/", "project_urls": { "Homepage": "https://github.com/llllllllll/codetransformer" }, "release_url": "https://pypi.org/project/codetransformer/0.7.2/", "requires_dist": null, "requires_python": "", "summary": "Python code object transformers", "version": "0.7.2" }, "last_serial": 5311010, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "4055c9367a931864377ccc59cf2850cd", "sha256": "62dd0a7629ce77c808adfa2039c50b56f40c063d2715ec720b59a51d2eebaf7d" }, "downloads": -1, "filename": "codetransformer-0.1.0.tar.gz", "has_sig": false, "md5_digest": "4055c9367a931864377ccc59cf2850cd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4439, "upload_time": "2015-03-02T05:23:23", "url": "https://files.pythonhosted.org/packages/9e/6e/c1c93e33ba170a97ba2714a5d247536cd3a7f73082015fdbcd249037fc56/codetransformer-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "e4364bbb83faafeef112617b9f51de61", "sha256": "07d7dd8db62032dce219c23bbbadceba3c336c25e274090c1d5bde6d47c5a6d7" }, "downloads": -1, "filename": "codetransformer-0.1.1.tar.gz", "has_sig": false, "md5_digest": "e4364bbb83faafeef112617b9f51de61", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4445, "upload_time": "2015-03-02T05:30:28", "url": "https://files.pythonhosted.org/packages/c7/03/d815e216af89a4537c7ced7727c265135d44e3b764951e0e04e41d6c19b5/codetransformer-0.1.1.tar.gz" } ], "0.1.1.1": [ { "comment_text": "", "digests": { "md5": "3cc9fc1d41b21c424e7053c699560a4f", "sha256": "b9ae209b2cbbb947c4481424d00332f0e4bc7d2d3478259b4445e441a7867f9c" }, "downloads": -1, "filename": "codetransformer-0.1.1.1.tar.gz", "has_sig": false, "md5_digest": "3cc9fc1d41b21c424e7053c699560a4f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4447, "upload_time": "2015-03-02T05:37:08", "url": "https://files.pythonhosted.org/packages/fc/e6/806e9248552f3e3e352fb5b2418dc7341ce1fcdc1710326d5e05b97f604a/codetransformer-0.1.1.1.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "98bcf2b6753c6578d47d598039a8cb81", "sha256": "4eac97c74ed095f553dca7123e8163838bb83adf4db4fdc73740e83e13b8e8f4" }, "downloads": -1, "filename": "codetransformer-0.2.0.tar.gz", "has_sig": false, "md5_digest": "98bcf2b6753c6578d47d598039a8cb81", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4620, "upload_time": "2015-04-17T02:34:26", "url": "https://files.pythonhosted.org/packages/a1/38/820d0332298d4df9a313e3cda6dad6b69b5a45d2f2c0801723aff1f49b68/codetransformer-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "45d26cfb1611d1e19053fcd71e290b89", "sha256": "a646dd11c2427cad2d61c779fefa48a82eb578a25f78d61b201af5d572b92f8b" }, "downloads": -1, "filename": "codetransformer-0.3.0.tar.gz", "has_sig": false, "md5_digest": "45d26cfb1611d1e19053fcd71e290b89", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5594, "upload_time": "2015-04-27T01:40:52", "url": "https://files.pythonhosted.org/packages/11/fd/6f0a579f5694977243ad8b356e3222127c98f022c699d1073d84414c8627/codetransformer-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "915f4f1b4933fd301848bdf3dc62cb69", "sha256": "d4f015eca600364a09a3c97766b93ef32ad51f8a65063bcd6e322fb95e4947ff" }, "downloads": -1, "filename": "codetransformer-0.4.0.tar.gz", "has_sig": false, "md5_digest": "915f4f1b4933fd301848bdf3dc62cb69", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5605, "upload_time": "2015-05-03T02:31:54", "url": "https://files.pythonhosted.org/packages/13/1a/215ddfcb4652b8cdebaad59265c7f5a4fc0f685e706016d62a4c72029de2/codetransformer-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "3807486659a42322021842092e2b3014", "sha256": "de38d26045dd2a06bcc5ced5d745e018d3994bc1108d47fb7707c7f3cd83f24f" }, "downloads": -1, "filename": "codetransformer-0.4.1.tar.gz", "has_sig": false, "md5_digest": "3807486659a42322021842092e2b3014", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7302, "upload_time": "2015-05-17T02:06:43", "url": "https://files.pythonhosted.org/packages/e7/9e/9147da5a4853f03ab167de26bc56c1ba291bd72f405a3456a21a34f3e366/codetransformer-0.4.1.tar.gz" } ], "0.4.2": [ { "comment_text": "", "digests": { "md5": "2087403d5cc9f8d1563074a49bef9d21", "sha256": "4a19a3617eed67aadec8b195f5128ff6514a350ca37f2a20c86ac77c7ba04d4f" }, "downloads": -1, "filename": "codetransformer-0.4.2.tar.gz", "has_sig": false, "md5_digest": "2087403d5cc9f8d1563074a49bef9d21", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7415, "upload_time": "2015-05-17T04:19:20", "url": "https://files.pythonhosted.org/packages/a5/36/779bb8feff79133ea3b981674ebf7a5d738bb1237677465b46b2bacc5e76/codetransformer-0.4.2.tar.gz" } ], "0.4.3": [ { "comment_text": "", "digests": { "md5": "ead9570b8a99ad8517b17498364a8a95", "sha256": "915e82f63e600272f2f8785a9bfbe9fec2afaf45b67a331b587926653e4a3cc5" }, "downloads": -1, "filename": "codetransformer-0.4.3.tar.gz", "has_sig": false, "md5_digest": "ead9570b8a99ad8517b17498364a8a95", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7439, "upload_time": "2015-06-15T23:02:17", "url": "https://files.pythonhosted.org/packages/59/2e/cea2db7b55abc0349ea823c3bda7b807c7f1ee0502a9f71f7cad609b347b/codetransformer-0.4.3.tar.gz" } ], "0.4.4": [ { "comment_text": "", "digests": { "md5": "8b4fc47af68ab26dd94f13d890517a08", "sha256": "ed99777a6a78679ce536e278e80057ebfddb1d244e284b30340174fd73c53313" }, "downloads": -1, "filename": "codetransformer-0.4.4.tar.gz", "has_sig": false, "md5_digest": "8b4fc47af68ab26dd94f13d890517a08", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7731, "upload_time": "2015-06-16T00:01:19", "url": "https://files.pythonhosted.org/packages/49/c7/ca9eb69c031469e077e3c259217a182d84f4109c0adcb71b967bda4862d8/codetransformer-0.4.4.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "4756d06281a9da668b2fc9158157b7f8", "sha256": "48943e04b17d0133dfb01f6dbdf730d92593e9c10c62e6a1b8994e9261a51211" }, "downloads": -1, "filename": "codetransformer-0.5.0.tar.gz", "has_sig": false, "md5_digest": "4756d06281a9da668b2fc9158157b7f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11635, "upload_time": "2015-06-28T01:31:18", "url": "https://files.pythonhosted.org/packages/a2/d0/ea14b12cfde31b5bf1a61a42e6080c5f295479b0dff423aaeebb395ca53a/codetransformer-0.5.0.tar.gz" } ], "0.6.0": [ { "comment_text": "", "digests": { "md5": "5e422c681e1ac6abfe20444971a029ff", "sha256": "a04af0307465ecb0b40104b9dc44b4ac5d4cc520f374be4a63e7a41c2defbbdb" }, "downloads": -1, "filename": "codetransformer-0.6.0.tar.gz", "has_sig": false, "md5_digest": "5e422c681e1ac6abfe20444971a029ff", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 50979, "upload_time": "2016-02-03T06:31:45", "url": "https://files.pythonhosted.org/packages/ed/35/d036925009d467d81b43b8ff36a6d128ff1b0f6ce27836c0bba0f03d5c49/codetransformer-0.6.0.tar.gz" } ], "0.7.0": [ { "comment_text": "", "digests": { "md5": "557a655e64a190182cc0101bbcda133e", "sha256": "8cf95b96d22bd1b7047162f0a0bc6293d526b70e9d4edf38a932921d76993e0b" }, "downloads": -1, "filename": "codetransformer-0.7.0.tar.gz", "has_sig": false, "md5_digest": "557a655e64a190182cc0101bbcda133e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 72820, "upload_time": "2017-05-24T22:35:59", "url": "https://files.pythonhosted.org/packages/18/56/d2207feca7a457e3bcc894b180c2c0438a38443b3509567ea9d39d448291/codetransformer-0.7.0.tar.gz" } ], "0.7.1": [ { "comment_text": "", "digests": { "md5": "b571ec56a452a303e8bb8cd378748bd6", "sha256": "89651eb75c8e86ec44978768b3cca21e334dcdf8244f5ffcc7f75937c49a9697" }, "downloads": -1, "filename": "codetransformer-0.7.1.tar.gz", "has_sig": false, "md5_digest": "b571ec56a452a303e8bb8cd378748bd6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 72965, "upload_time": "2017-05-25T00:28:19", "url": "https://files.pythonhosted.org/packages/30/02/b392ae5c61a5d816760ddb88717e86bbe4d4102a225e4c86d74f7d0ddd86/codetransformer-0.7.1.tar.gz" } ], "0.7.2": [ { "comment_text": "", "digests": { "md5": "1ca91458bc5a82d9f3f1a5b468af785c", "sha256": "442aca89cc961f8c9234dd17e35d36cdeaca670f167610e67cef232426705118" }, "downloads": -1, "filename": "codetransformer-0.7.2.tar.gz", "has_sig": false, "md5_digest": "1ca91458bc5a82d9f3f1a5b468af785c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 73598, "upload_time": "2017-09-10T10:20:27", "url": "https://files.pythonhosted.org/packages/7b/de/7b1dc2314b04b7d3a6ab3d1e0c2e24b3db51f9147dbcde5bd3840475368b/codetransformer-0.7.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1ca91458bc5a82d9f3f1a5b468af785c", "sha256": "442aca89cc961f8c9234dd17e35d36cdeaca670f167610e67cef232426705118" }, "downloads": -1, "filename": "codetransformer-0.7.2.tar.gz", "has_sig": false, "md5_digest": "1ca91458bc5a82d9f3f1a5b468af785c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 73598, "upload_time": "2017-09-10T10:20:27", "url": "https://files.pythonhosted.org/packages/7b/de/7b1dc2314b04b7d3a6ab3d1e0c2e24b3db51f9147dbcde5bd3840475368b/codetransformer-0.7.2.tar.gz" } ] }