{ "info": { "author": "David R. MacIver", "author_email": "david@drmaciver.com", "bugtrack_url": null, "classifiers": [], "description": "===========\nTestmachine\n===========\n\nTestmachine is a new way of testing. It is a python library, and intended for\ntesting python programs, but the core concept is language agnostic and should be\neasy to port to other languages.\n\nIt's strongly inspired by Quickcheck and other similar approaches to fuzz testing,\nbut with a twist that essentially turns the idea inside out and makes it much more\npowerful.\n\nThe core concept is that testmachine doesn't generate data, it generates \n*programs*. This is based on ideas from the stateful testing system found in \n`Scalacheck `_, (as well as my subsequent\nport of it to Python in `hypothesis `_),\nbut with testmachine it applies equally well to stateful and non-stateful systems\nalike.\n\nPrograms generated by statemachine are defined by composing a number of user\nprovided rules. These rules will normally fall into one of three categories:\n\n- Functions which generate random values (like an Arbitrary in quickcheck)\n- Assertions about values (like a Property in quickcheck)\n- Rules for *combining* values (this is hard to do in quickcheck)\n\nIn reality all three of these are represented by the same type internally and\nit will sometimes be convenient to use rules which are hybrids of these (e.g.\noperations which also do make some assertions, or functions which return a random\nassertion), but for the most part this separation is convenient.\n\nYou define these rules as python functions, and testmachine then takes them \nand attempts to find a combination which will produce an exception. If it finds\none it then does its best to minimize it and compiles the final output into a\nsimple text format that will usually be interpretable as valid python (it's\ngenerated by simple string expansion from patterns. If your values have reprs\nwhich are valid python and you don't do anything too unusual then the output\nwill be valid python. Otherwise it may take some editing but should be readable\non its own).\n\n--------------\nUsage examples\n--------------\n\n1. `Demonstrating that floating point addition is non-associative `_\n2. `Trying (and failing) to show that integer addition is not commutative `_\n3. `Finding lists with duplicate elements `_\n4. `Constructing unbalanced trees `_\n\n------\nStatus\n------\n\nThis code should be considered pretty experimental right now. You *can* use it\nand it will *probably* work, but I make no promises about the API being stable.\nIt's very much still a work in progress as I explore a new idea.\n\n---------\nInternals\n---------\n\nHow does it work?\n\nAt its core, a Testmachine describes a language of programs that can be executed\non a machine with multiple named stacks of values. It generates random programs\nin an attempt to find one which causes an error.\n\nEach program is a sequence of operations. Each of which may manipulate the stacks\nin an arbitrary way. If an operation throws an exception, testmachine has now \nfound a bug.\n\nBecause of its definition in terms of stack operations it's very easy to then\nminimize the program: A program is just a list of operations with no explicit\ndependencies between them (all data dependency goes via the stacks), so you can\njust delete instructions from it, remove any instructions which find themselves\nin an invalid state (e.g. if not enough values are on the stacks they need) and \nsee if the program still fails. If it does, you've shrunk the program. We then\niterate this process greedily until we can no longer shrink the program further.\n\nAlthough this does not produce program which is guaranteed to be globally minimal,\nin practice it generally seems to do extremely well at producing short example\nprograms.\n\nOnce this has complete, we have an example program against the stack machine.\nBut this isn't very readable, so we want to compile it to a simpler python output.\n\nWe do this by maintaining a set of shadow stacks with variable names,\none for each of stacks in our machine. By tracking which variables are read\nand written in an operation we convert the stack operations into a set of \nvariable reads and writes. The result is a sequence of variable reads and assignments\nin SSA form which is equivalent to the stack program. We then do some simple string\ntemplating which generates output that looks quite like a python program.\n\n---------\n Testing\n---------\n\nThis version of testmachine has been tested using Python series 2.6, 2.7,\n3.2, 3.3 and pypy. Builds are checked with `travis`_:\n\n.. _travis: https://travis-ci.org/DRMacIver/testmachine\n\n.. image:: https://travis-ci.org/DRMacIver/testmachine.png?branch=master\n :target: https://travis-ci.org/DRMacIver/testmachine\n\n.. image:: https://coveralls.io/repos/DRMacIver/testmachine/badge.png?branch=master\n :target: https://coveralls.io/r/DRMacIver/testmachine?branch=master", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/DRMacIver/testmachine", "keywords": null, "license": "LICENSE.txt", "maintainer": null, "maintainer_email": null, "name": "testmachine", "package_url": "https://pypi.org/project/testmachine/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/testmachine/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/DRMacIver/testmachine" }, "release_url": "https://pypi.org/project/testmachine/0.0.5/", "requires_dist": null, "requires_python": null, "summary": "Stack based automatic testcase generation", "version": "0.0.5" }, "last_serial": 994313, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "c958ace00e1e37ce969aff21869ed2f2", "sha256": "f9941bacd89a163af7abe4bbefa3b96ff05e33d18881050bd33d3b612c01a75e" }, "downloads": -1, "filename": "testmachine-0.0.1.tar.gz", "has_sig": false, "md5_digest": "c958ace00e1e37ce969aff21869ed2f2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6222, "upload_time": "2014-01-08T14:57:46", "url": "https://files.pythonhosted.org/packages/30/fe/1d255dba37cd8673f1781abd88e666aad0cab4f00788456d875cd6ef2253/testmachine-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "1e25f96bf7dc3aa114e4695769e5940f", "sha256": "9ffd74825042f007dd578f8bd2e833a42f5c9efe9c511e19c7fdcaaa2b42fbf7" }, "downloads": -1, "filename": "testmachine-0.0.2.tar.gz", "has_sig": false, "md5_digest": "1e25f96bf7dc3aa114e4695769e5940f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6700, "upload_time": "2014-01-08T16:27:41", "url": "https://files.pythonhosted.org/packages/6d/82/fe128a6a39a48fef12294b6c070407ca0d43b96a9df3e1f348ca3fe8fb4a/testmachine-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "504c587fe59546326aa37c2b69029f2e", "sha256": "befeb4625e55dd2b0f4f9977b4b965d313f50e8c2701eaf1a4b9986c7f765db5" }, "downloads": -1, "filename": "testmachine-0.0.3.tar.gz", "has_sig": false, "md5_digest": "504c587fe59546326aa37c2b69029f2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8383, "upload_time": "2014-01-09T08:51:20", "url": "https://files.pythonhosted.org/packages/54/71/ebccb386a0c97108d72e5f42703f9cc4a49749f2c0d8212ce82c0fb447f5/testmachine-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "82ddee7b3b789d23877385dd8e43f704", "sha256": "d4ac308501d5e9b06af0a264cc5202925d26c957e34481282110784764031b63" }, "downloads": -1, "filename": "testmachine-0.0.4.tar.gz", "has_sig": false, "md5_digest": "82ddee7b3b789d23877385dd8e43f704", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9067, "upload_time": "2014-01-12T11:02:59", "url": "https://files.pythonhosted.org/packages/c2/be/a433c253376b62872951e3bb02ec8d6fd6dc8022d789899249b2fcdd99ba/testmachine-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "23b3ae4e36e60997af422d2e781ce2a6", "sha256": "32f1822ad1ff548aa8f9b3ff314f14ef550967726d147715fbe9d30a7cd93bb9" }, "downloads": -1, "filename": "testmachine-0.0.5.tar.gz", "has_sig": false, "md5_digest": "23b3ae4e36e60997af422d2e781ce2a6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9638, "upload_time": "2014-02-08T11:27:12", "url": "https://files.pythonhosted.org/packages/85/6a/2b455c7a4f5cea5c242ee1b7ef89ec3892b795fd5c12b8987f36d6acdcad/testmachine-0.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "23b3ae4e36e60997af422d2e781ce2a6", "sha256": "32f1822ad1ff548aa8f9b3ff314f14ef550967726d147715fbe9d30a7cd93bb9" }, "downloads": -1, "filename": "testmachine-0.0.5.tar.gz", "has_sig": false, "md5_digest": "23b3ae4e36e60997af422d2e781ce2a6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9638, "upload_time": "2014-02-08T11:27:12", "url": "https://files.pythonhosted.org/packages/85/6a/2b455c7a4f5cea5c242ee1b7ef89ec3892b795fd5c12b8987f36d6acdcad/testmachine-0.0.5.tar.gz" } ] }