{ "info": { "author": "Uwe Schmitt", "author_email": "uwe.schmitt@id.ethz.ch", "bugtrack_url": null, "classifiers": [], "description": "pytest-regtest\n==============\n\npytest-regtest is a *pytest*-plugin for implementing regression tests.\nCompared to functional testing a regression test does not test if\nsoftware produces correct results, instead a regression test checks if\nsoftware behaves the same way as it did before introduced changes.\n\nMore about regression testing at\n. Regression testing\nis a common technique to get started when refactoring legacy code\nlacking a test suite.\n\n*pytest-regtest* allows capturing selected output which then can be\ncompared to the captured output from former runs.\n\nTo install and activate this plugin execute:\n\n $ pip install pytest-regtest\n\n*pytest-regtest* plugin provides a fixture named *regtest* which can be\nused as a file handle for recording data:\n\n def test_squares_up_to_ten(regtest):\n\n result = [i*i for i in range(10)]\n\n # one way to record output:\n print(result, file=regtest)\n\n # alternative method to record output:\n regtest.write(\"done\")\n\n # or using a context manager:\n with regtest:\n print(\"this will be recorded\")\n\nIf you run this test script with *pytest* the first time there is no\nrecorded output for this test function so far and thus the test will\nfail with a message including a diff:\n\n $ py.test\n ...\n\n regression test output differences for test_demo.py::test_squares_up_to_ten:\n\n > --- current\n > +++ tobe\n > @@ -1,2 +1 @@\n > -[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n > -done\n > +\n\nThe output tells us what the current output is, and that the \"tobe\" output\nis still empty.\n\nFor accepting this output, we run *pytest* with the *--reset-regtest*\nflag:\n\n $ py.test --regtest-reset\n\nNow the next execution of *py.test* will succeed:\n\n $ py.test\n\nNow we break the test by modifying the code under test to compute the first\neleven square numbers:\n\n from __future__ import print_function\n\n def test_squares_up_to_ten(regtest):\n\n result = [i*i for i in range(11)] # changed !\n\n # one way to record output:\n print(result, file=regtest)\n\n # alternative method to record output:\n regtest.write(\"done\")\n\nThe next run of pytest delivers a nice diff of the current and expected output\nfrom this test function:\n\n $ py.test\n\n ...\n > --- current\n > +++ tobe\n > @@ -1,2 +1,2 @@\n > -[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]\n > +[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n > done\n\n\nThe recorded output was written to files in the subfolder\n`_regtest_outputs` next to your test script(s). You might keep this\nfolder under version control.\n\n\nOther features\n--------------\n\nAnother way to record output is to capture all output to `sys.stdout`:\n\n def test_squares_up_to_ten(regtest):\n\n result = [i*i for i in range(10)]\n\n with regtest():\n print(result)\n\nYou can reset recorded output of files and functions individually as:\n\n $ py.test --regtest-reset tests/test_00.py\n $ py.test --regtest-reset tests/test_00.py::test_squares_up_to_ten\n\nTo supress the diff and only see the stats use:\n\n $ py.test --regtest-nodiff\n\nTo see recorded output during test execution run:\n\n $ py.test --regtest-tee -s\n\nIf you develop on mixed platforms it might be usefull to ignore white\nspaces at the end of the lines when comparing output. This can be\nachieved by specifying:\n\n $ py.test --regtest-ignore-line-endings\n\n\nFixing unavoidable changes in recorded output\n----------------------------------------------\n\nThe recorded output can contain data which is changing from test run to test\nrun, e.g. pathes created with the `tmpdir` fixture or hexadecimal object ids,\nwhen objects are printed.\n\nThe plugin already replaces such changing data in the recorded output,\nand one can register own converters in `conftest.py` in the tests\nfolder. For example:\n\n import pytest_regtest\n\n @pytest_regtest.register_converter_pre\n def fix_before(txt):\n \"\"\"modify recorded output before the default fixes\n like temp folders or hex object ids are applied\"\"\"\n\n # remove lines with passwords:\n lines = txt.split('\\n')\n lines = [l for l in lines if \"password is\" not in l]\n return '\\n'.join(lines)\n\n @pytest_regtest.register_converter_post\n def after(txt):\n \"\"\"modify recorded output after the default fixes\n like temp folders or hex object ids are applied\"\"\"\n\n # for demo only\n return txt.upper()\n\nThis can be used to fix substrings like \"computation need 1.23 seconds\"\nto \"computation needed