{ "info": { "author": "Kevin L. Mitchell", "author_email": "kevin.mitchell@rackspace.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4" ], "description": "=================\nTimid Test Runner\n=================\n\nTimid is a command line tool for running tests. It differs from\nPython tools like tox in that it is not limited to Python. It uses a\nYAML file to describe how to build the environment to run the test\nin.\n\nWhy Timid?\n==========\n\nTimid is intended to provide a very flexible test description\nlanguage. It provides functionality for setting up various aspects of\nthe test environment, as well as for actually invoking the test\ncommand. While not too dissimilar from a Python tool like tox, Timid\ndoes not make any assumptions about what that environment should look\nlike; in particular, it does not create a virtual environment unless\nthe test description includes the appropriate commands to do so. This\nmakes it suitable for running any set of tests.\n\nAnother aspect of Timid is the ability to reference any subset of test\nsteps from other files. This enables easy reuse for complicated test\ndescriptions, and even means a library of test description fragments\nmay be easily established and used. Timid also allows the working\ndirectory to be directly set from the command line, allowing the test\ndescriptions to be separated from the actual code to test. Finally,\nTimid is extremely extensible; new test step actions and modifiers may\nbe created by including a class in the appropriate entrypoint\nnamespaces (\"timid.actions\" and \"timid.modifiers\", respectively), and\nextensions (namespace \"timid.extensions\") may also be created that can\nperform specific tasks under control of the command line--for\ninstance, an extension could allow a Timid test to run on a Github\npull request, setting test status information using the Github status\nAPI.\n\nBasic Test Description Syntax\n=============================\n\nTest descriptions are YAML files consisting of a list of dictionaries,\nwhere each dictionary describes a *step* in the testing process. Each\nstep consists of one *action* and zero or more *modifiers* which alter\nthe action in some way. A step may also have a *name*, which is used\nto identify the step in the output, and a *description*. The action\nand the modifiers are identified by the keys of the step dictionary;\nthe values of those keys identify the options for that action or\nmodifier. The options may be any legal YAML entity, such as a string,\ninteger, boolean, list, or a dictionary; the documentation for each of\nthe actions and modifiers will describe what that action or modifier\nexpects.\n\nThe test description could also be a smaller component of a YAML file\ncontaining a dictionary; each value of this dictionary must be a list\nof dictionaries, as described above. This could be used to describe\nseveral different but related tests for a single project (e.g., style\ntests, unit tests, functional tests, and integration tests), or it\ncould be used to provide a library of test steps that can be included\nusing the \"include\" action.\n\nTemplating and Expressions\n--------------------------\n\nMany actions and modifiers allow Jinja2-style templates to be\nspecified for values, which enhances reusability of test description\ncomponents. Jinja2-style expressions can also be used; an example\nwould be the \"when\" modifier, which provides simple conditional\ncontrol of an action. Template variables can be set on the command\nline, read from a YAML file, or set up directly in the test\ndescription.\n\nSecurity\n--------\n\nTimid provides a way to mark both template variables and environment\nvariables as being \"sensitive\". This is to allow security-sensitive\ndata, such as usernames and passwords, to be used, while ensuring that\nthat sensitive data is scrubbed from any verbose or debugging output\nfrom Timid itself. For template variables, this can only be done from\nthe test description file, but environment variables can also be\nmarked sensitive by listing them in the ``TIMID_SENSITIVE``\nenvironment variable, separated by your system's path separator\ncharacter. (On UNIX and Linux systems, this character would be the\n\":\" character.) The ``TIMID_SENSITIVE`` environment variable will\nalso be present in the environment of any subordinate processes,\nupdated with any additional environment variables marked as sensitive\nby the test description; this can be used by test scripts to omit\nsensitive information from the environment in debugging output.\n\nExtending Timid\n===============\n\nAs mentioned previously, Timid uses Python entrypoints for simple\nextensibility. Each action or modifier in a test description is\nlooked up for in the \"timid.actions\" or \"timid.modifiers\" namespaces,\nrespectively. These entrypoints must map to subclasses of\n``timid.Action`` or ``timid.Modifier``, as appropriate. Timid also\nprovides extensions, which allow extending the actual command line\ninterface and other per-step behavior; these are listed in the\n\"timid.extensions\" interface, and the entrypoints must map to\nsubclasses of ``timid.Extension``.\n\nCreating a New Action\n---------------------\n\nActions perform the actual test step. Creating a new action is a\nmatter of extending ``timid.Action``. In the new action class, the\n``schema`` class attribute must be set to a JSONSchema description of\nthe expected configuration; and the ``__call__()`` method, taking as\nits sole argument a *context* object, must be defined to implement the\nactual action; it should return an instance of ``timid.StepResult``.\nThe ``timid.Action`` class declares a ``__init__()`` method taking\nfour arguments (a context object, the name of the action (the key read\nfrom the test description), the configuration for the action (the\nvalue for that key), and a *step address* object); it validates the\nconfiguration, then stores the last three arguments in the ``name``,\n``config``, and ``step_addr`` attributes, respectively. (The context\nobject should not be stored; it will be passed in to the\n``__call__()`` method.)\n\nThere are two types of actions. By default, all ``timid.Action``\nsubclasses are instantiated while reading the test description, then\ntheir ``__call__()`` methods are invoked in order during the actual\ntest run--or not invoked at all, if a syntax check is being\nperformed. However, it is possible to create a \"step action\", an\naction invoked immediately after it is read from the test description;\nthis is used, for instance, to implement the \"include\" step, which\nreads steps from another file and inserts them in place of the\n\"include\" step. These are implemented by setting the ``step_action``\nclass attribute to ``True`` and having ``__call__()`` return a list of\n``timid.Step`` objects, instead of a ``timid.StepResult`` object.\n\nCreating a New Modifier\n-----------------------\n\nModifiers modify a step in some fashion, such as by running the step\nin a loop or applying a conditional prior to invoking the step.\nCreating a new action is a matter of extending ``timid.Modifier``.\nLike actions, the new subclass must have a ``schema`` class attribute\nset to a JSONSchema description of the expected configuration, and a\n``__init__()`` method identical to that for ``timid.Action`` is also\nimplemented; however, modifiers do not have a ``__call__()`` method,\nand the class attribute ``priority`` must be set to an integer value.\nThe ``priority`` attribute controls the order in which modifiers are\napplied while running a step, with lower values invoked before higher\nvalues.\n\nA modifier actually consists of a set of hook functions. The\n``timid.Modifier`` superclass contains default implementations of\nthese hook functions, so a developer need only override the ones\nneeded to implement the modifier.\n\nThe first hook is the ``action_conf()`` hook, which takes 5 arguments:\na context object, the class implementing the modified action, the name\nof the action (key read from the test description), the configuration\nfor the *action* (the ``__init__()`` method receives the configuration\nfor the *modifier*), and a *step address* object. The hook function\nmust return the configuration that should be passed to the action\nclass, giving the modifier the opportunity to alter the configuration.\n\nThe remaining two hooks are the ``pre_call()`` and ``post_call()``\nmethods, which are invoked prior to and after calling the action's\n``__call__()`` method, respectively. The ``pre_call()`` method can\nreturn a ``timid.StepResult`` object, which aborts further processing\n(including the call to the action) and proceeds with invoking any\n``post_call()`` methods. The ``post_call()`` method receives the\n``timid.StepResult`` object and can modify it or even replace it\nentirely, by returning a different object. The ``pre_call()`` method\ntakes 4 arguments: a context object, a \"pre_mod\" list, a \"post_mod\"\nlist, and the instance of the ``timid.Action`` subclass. The\n``post_call()`` method gets 5 arguments: a context object, the result\nof the call (an instance of ``timid.StepResult``), the instance of the\n``timid.Action`` subclass, a \"post_mod\" list, and a \"pre_mod\" list.\nThe \"pre_mod\" and \"post_mod\" lists are lists of ``timid.Modifier``\ninstances that have lower priority and higher priority, respectively.\nIt should also be noted that ``post_call()`` is called in the inverse\norder of ``pre_call()``.\n\nContext Objects\n---------------\n\nThe context object passed to the actions and modifier methods provides\nseveral services throughout Timid. The ``verbose`` attribute contains\nan integer value controlling the verbosity of Timid's output (0 means\nno output at all), and ``debug`` is a boolean indicating whether\ndebugging is enabled. The ``variables`` attribute contains a\ndictionary of template variables, and ``environment`` contains the\nenvironment variables. (The environment dictionary-like object also\nallows control of the current working directory, by setting its\n``cwd`` attribute, and its ``call()`` method should be used to invoke\nexternal programs.)\n\nTimid provides an interface to Jinja2, and two utility methods on the\ncontext object facilitate this: the ``template()`` method takes a\nstring and returns a callable of one argument that will render the\ntemplate, and ``expression()`` works similarly for Jinja2\nexpressions. (It is safe to pass objects other than strings to these\ntwo methods as well; the result will still be a callable of one\nargument, but no template expansion will be performed.) The context\nobject should be passed to the callable returned by ``template()`` and\n``expression()``.\n\nThe usual way to use the ``template()`` and ``expression()`` methods\nis to override the ``__init__()`` method of the ``timid.Action`` or\n``timid.Modifier`` subclass; the method should invoke the superclass's\nversion of ``__init__()`` (using a ``super()`` expression), and would\nthen process the configuration, saving the callables produced by\ncalling ``template()`` and ``expression()``. Then, where the values\nare used in the action's ``__call__()`` or the modifier's hook\nmethods, simply pass the context to the callable and use the result as\nthe actual value to use.\n\nStep Addresses\n--------------\n\nTo aid debugging, each action or modifier has a *step address* object\nassociated with it. The address has three attributes: the filename\nfrom which the step was read (``fname``); the 0-based index of the\nstep within the file (``idx``); and the key for the list containing\nthe steps (``key``). (This latter attribute will be ``None`` if the\nfile was a simple list of steps.) The object also has a\nstraightforward string representation which includes the filename,\nkey, and step index (1-based; that is, if ``idx`` is 3, the string\nwill identify the step as step 4).\n\nCreating an Extension\n---------------------\n\nExtensions are the most powerful extensibility mechanism in Timid.\nCreating one is a matter of extending ``timid.Extension`` and\nimplementing the desired hook methods, similar to creating a new\nmodifier, except that a Timid extension must implement an\n``activate()`` class method if it actually intends to do anything.\nAdditionally, a ``timid.Extension`` subclass must set the ``priority``\nclass attribute to a numerical value, just like a ``timid.Modifier``\nsubclass; extension hook functions will be called in the order\ndictated by the priorities.\n\nThe first hook method that an extension may implement is the\n``prepare()`` method. This must be a class method, and will receive\nas its sole argument an ``argparse.ArgumentParser`` instance, which\nthe extension may use to declare new command line options. All\nextensions will have their ``prepare()`` method called during Timid\ninitialization.\n\nOnce the command line has been processed by\n``argparse.ArgumentParser``, each extension's ``activate()`` method\nwill be called with a context object and an ``argparse.Namespace``\ncontaining the results of the command line processing. This method\nmust also be a class method, and must return either ``None`` or an\ninstance of the ``timid.Extension`` subclass; if it returns ``None``,\nthe extension is treated as inactive and no other hook methods will be\ncalled.\n\nThe remaining hook methods are all instance methods, called on the\nobject returned by the ``activate()`` method. The ``read_steps()``\nmethod is called with a context object and a list of ``timid.Step``\ninstances; the extension may perform any in-place modifications to the\nlist of steps that are appropriate. The ``pre_step()`` and\n``post_step()`` methods are called before and after executing a step,\nrespectively; ``pre_step()`` is called with a context object, the\n``timid.Step`` instance, and the index of the step, and may return a\n``True`` value to cause the step to be skipped. The ``post_step()``\nmethod is called with the same arguments, and a fourth argument, which\nwill be a ``timid.StepResult`` object, which it may alter in place;\nthe return value of ``post_step()`` is ignored. Note that\n``post_step()`` is called in extension order, in contrast to the\n``post_call()`` method of ``timid.Modifier`` instances.\n\nThe final hook function is the ``finalize()`` method, which is called\njust before the command line tool exits. It is called with a context\nobject and the result, which will typically be ``None`` for success,\nor a text string indicating an error. (It could also be called with\nan ``Exception`` instance if an error occurred.) This method's return\nvalue will replace the result.\n\nDebugging Extensions\n--------------------\n\nThe implementation of extensions explicitly ignores exceptions raised\nby a given extension. This would make it difficult to debug a newly\ndeveloped extension, so Timid provides a debugging mechanism: the\n``TIMID_EXTENSION_DEBUG`` environment variable may be set to an\ninteger value, with larger values resulting in more verbose debugging.\nIf ``TIMID_EXTENSION_DEBUG`` is present in the environment with no\nvalue, or with a non-integer value, the debugging level will be set to\n1; a debugging level of 0 (or any negative value) is exactly the same\nas if ``TIMID_EXTENSION_DEBUG`` was not present in the environment at\nall.\n\nNote that this environment variable is checked directly from the\nenvironment, unlike the ``TIMID_SENSITIVE`` environment variable.\nThis means that the value used by extension debugging cannot be\naltered by any instructions in the test description; only child\nprocesses can be affected by such instructions. Even command line\nenvironment variable manipulations are ignored for the purposes of\nextension debugging. This design decision was made so that debugging\ncould be enabled before even calling the extension ``prepare()``\nmethod, which is called before any argument processing is done, and\nthus prior to reading any test description files.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/rackerlabs/timid", "keywords": null, "license": "Apache License (2.0)", "maintainer": null, "maintainer_email": null, "name": "timid", "package_url": "https://pypi.org/project/timid/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/timid/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/rackerlabs/timid" }, "release_url": "https://pypi.org/project/timid/0.1.2/", "requires_dist": null, "requires_python": null, "summary": "Timid test runner", "version": "0.1.2" }, "last_serial": 1916224, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "6625424e1de9583e55f73f8b70d76992", "sha256": "1a40f1e4f9947c13d1688dde5a1bf77cb167db971ddc2921bdce79455a93b7e6" }, "downloads": -1, "filename": "timid-0.1.0.tar.gz", "has_sig": false, "md5_digest": "6625424e1de9583e55f73f8b70d76992", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 57673, "upload_time": "2015-06-26T15:24:32", "url": "https://files.pythonhosted.org/packages/49/83/28e638150abc9b5be9ac6ab430a3fb8e74b9ac961a61dc4bef34ac19b4a7/timid-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "5b110a32881c3ca68f72fb46fd08dfa2", "sha256": "dc15d9cc4e2c42c31a2d7c8057dc7df5a3582661e4bcef2543a2cacad6c13871" }, "downloads": -1, "filename": "timid-0.1.1.tar.gz", "has_sig": false, "md5_digest": "5b110a32881c3ca68f72fb46fd08dfa2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 57773, "upload_time": "2015-11-17T19:31:10", "url": "https://files.pythonhosted.org/packages/55/b8/efaa89e694086d4e6d9b85b99bc3497c0ebba756c6323afb13a9ded231ff/timid-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "904a22d8925874994e8adb18098b9e64", "sha256": "9eb44c20abc9ec5859555c2eb4ee07fb2f2d0514c3b0c7c936f8c2c9a3b8c68d" }, "downloads": -1, "filename": "timid-0.1.2.tar.gz", "has_sig": false, "md5_digest": "904a22d8925874994e8adb18098b9e64", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 58187, "upload_time": "2016-01-21T22:26:00", "url": "https://files.pythonhosted.org/packages/e9/7e/aa4826867fef8b02e647c6bf4142d590a942b52946680d4713b7ffd903f4/timid-0.1.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "904a22d8925874994e8adb18098b9e64", "sha256": "9eb44c20abc9ec5859555c2eb4ee07fb2f2d0514c3b0c7c936f8c2c9a3b8c68d" }, "downloads": -1, "filename": "timid-0.1.2.tar.gz", "has_sig": false, "md5_digest": "904a22d8925874994e8adb18098b9e64", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 58187, "upload_time": "2016-01-21T22:26:00", "url": "https://files.pythonhosted.org/packages/e9/7e/aa4826867fef8b02e647c6bf4142d590a942b52946680d4713b7ffd903f4/timid-0.1.2.tar.gz" } ] }