{
    "info": {
        "author": "JL Peyret",
        "author_email": "jpeyret@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 2 - Pre-Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6"
        ],
        "description": "# \ud83c\udf70 Cake first! \ud83e\udd52 Veggies later.\n\n#### Let's run a failing test\n\n`pytest test_urls_security_psroledefn.py::Test_List::test_it`\n\n````\n... snipped out...\nE       <title>\nE   -    Search Rolez  \ud83d\udc48 \u274c TYPO \u274c\nE   ?               ^\nE\nE   +    Search Roles   \u2705 what we actually want\nE   ?               ^\n... snipped out...\nFAILED test_urls_security_psroledefn.py::Test_List::test_it - AssertionError: '<!DO[395 chars] Rolez\\n  \ud83d\ude27\ud83d\ude27\ud83d\ude27\n````\n\nNote:  old-school `unittests` are supported:  `python test_urls_security_psroledefn.py Test_List.test_it`\n\n#### Now run a diff command to see what went wrong.\n\n\n`ksdiff exp/test_urls_security_psroledefn.Test_List.test_it.html got/test_urls_security_psroledefn.Test_List.test_it.html` \n\nIn this case, the left hand file is what this particular test was **expecting** and the right hand side  is what it **got**.  Notice how the file names match the python test file, test class and method.\n\n\n\n![](docs/screenshots/01.different.png)\n\nLooks like somebody fixed a typo and that's why the test is failing.  \n\nNote:  this was using **Kaleidoscope** on macOS, but you could use gnu `diff` just as well.\n\n\n#### \ud83c\udf70 How to reset expectations.\n\nWe can use Kaleidcscope to tell Lazy to expect `Roles` rather than `Rolez`.  Save the `exp` file.\n\n![](docs/screenshots/02.reset.png)\n\n#### \ud83c\udf70 and rerun the test, which now works.\n\n````\npytest test_urls_security_psroledefn.py::Test_List::test_it\n============================================================= test session starts =================\nplatform darwin -- Python 3.6.8, pytest-5.0.1, py-1.8.0, pluggy-0.12.0\nrootdir: /Users/myuser/beemee\nplugins: celery-4.3.0, cov-2.7.1\ncollected 1 item\n\ntest_urls_security_psroledefn.py .                                                          [100%]\n\n========================================================== 1 passed in 4.73 seconds ===============\n\n````\n\n(btw, the 4.73 secondes isn't really Lazy's fault, this is my live test suite)\n\n\n### The directory structure.\n\nLazy's required configuration includes 2 environment variables, `$lzrt_template_dirname_exp`, the **expectations** directory and `$lzrt_template_dirname_got`, the **received** / got directory.  `exp` and `got`, for short.\n\nEach time a test is run, outputs are saved in `got` and compared to what stored in `exp`.  (No, those directories are not really side by side).  You don't really care what's in `got`, it gets rewritten each time.  The contents of `exp` are essentially a test spec however.\n\n\n````\n\u251c\u2500\u2500 exp\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 test_urls_security_psroledefn \n\u2502\u00a0\u00a0     \u2514\u2500\u2500 test_urls_security_psroledefn.Test_List.test_it.html\n\u2514\u2500\u2500 got\n    \u2514\u2500\u2500 test_urls_security_psroledefn\n        \u2514\u2500\u2500 test_urls_security_psroledefn.Test_List.test_it.html\n````\n\n#### \ud83c\udf70 The test method\n\n**test_urls_security_psroledefn.py**:\n\n````\nclass Test_List(Base):\n\t....\n\n    def test_it(self):\n    \t\"\"\" get data from an url configured elsewhere \"\"\"\n\n\t\t# this is test suite code and has nothing to do with Lazy\n        response = self.get(self.T_URL_BASE)\t\n        self.assertEqual(response.status_code, 200)  #\ud83d\udc48 you still need non-content tests!\n\n\t\t# \ud83d\udc47 That's pretty much it, as far as Lazy goes \ud83c\udf70\n\t\t#    this is what formats the data, runs the assertion and saves outputs to the file system.\n        self.lazychecks_html(response)  \n\t\t\n\n````\n\n#### \ud83c\udf70 But... what's that Base class?\n\n````\n\nfrom <path>.customlazy import CustomGenericLazyMixin\n\nclass LazyMixin(CustomGenericLazyMixin):\n\t\"\"\" \n\tThis class needs to be copy-pasted into EACH `test_xxx.py` script\n\tas it tracks file system and module information from Python built-in variables\n\tthis is what positions output in `exp` and `got` directories\n\t\"\"\"\n\t\n\t# \ud83c\udf70 - always the same!\n    lazy_filename = GenericLazyMixin.get_basename(__name__, __file__, __module__)\n\n\n\nclass Base(LazyMixin, unittest.TestCase):\n\n\t\"\"\"\ud83c\udf70 Lazy's lazycheck_xxx methods are directly available here, nothing to do.\"\"\"\n\n\tdef get(self, T_URL):\n\t\t.... whole buncha stuff relating to the test suite, like that `self.get(<url>)`\n\n````\n\n# \ud83e\udd52\ud83e\udd52\ud83e\udd52 : Veggie time:  let's be honest, how much work is this?\n\n\nThere's quite a bit of upfront work you need to do on your base Custom class.  Some of it can be improved in future versions of Lazy.  But the filter functions will remain complex and you have to write them.  The good news is that you only need to do it once.\n\nBasic **customlazy.py** example:\n\n````\n\nfrom lazy_regression_tests.core import (\n    LazyMixin, RemoveTextFilter, CSSRemoveFilter \n)\n\n\nclass CustomGenericLazyMixin(LazyMixin):\n\t\"\"\" \n\t\ud83e\udd52\ud83e\udd52\ud83e\udd52\ud83e\udd52 :-( There's work to do here... \n\t\"\"\"\n\n\t#each extension expects to find a matching lazy_filter_xxxx\n\tlazy_filter_html = build_html_filter()\n\n    def lazychecks_html(\n        self, response, suffix=\"\"\n    ):\n    \t\"\"\"\t\n    \tThis is where you tell Lazy how to check html.\n    \t\ud83e\udd52\ud83e\udd52 This could be put in the core class, but it hasn't been done yet.\n    \t\"\"\"\n        response = getattr(response, \"content\", None) or response\n        res_html = self.assertLazy(\n            response, \"html\", suffix=suffix\n        )\n        return res_html\n\n\ndef build_html_filter(onlyonce=False):\n\n    \"\"\"\n    \ud83e\udd52\ud83e\udd52\ud83e\udd52\ud83e\udd52\ud83e\udd52\n    unfortunately, diff-based regression tests requires you to strip out\n    things that vary frequently.\n    in Django that will be CSRF Token, even/odd <tr> CSS classes, timestamps....\n\n    _You_ need to do this, using regex and Lazy's utility functions\n\n    Here's a (simplified) example:\n    \"\"\"\n\n\n    li_remove = [\n        # the csrf token is by nature always changing.\n        # security nonces, if used, will also need scrubbing\n        re.compile(\"var\\scsrfmiddlewaretoken\\s=\\s\"),\n\n        # This a Vue/Webpack production time bundling artefact...\n        re.compile('<link type=\"text/css\" href=\"/static/webpack/styles/'),\n\n        # in my case, what I call usergroups need separate processing because they change as well\n        # the CSSRemove filter will save what it finds in lazycheck_html's results, under `found.<hitname>`\n        CSSRemoveFilter(\"#usergroup_table\", hitname=\"usergroup_table\"),\n    ]\n\n    res = RemoveTextFilter(li_remove)\n    return res\n\n\n````\n\n\n# \ud83e\udd52\ud83e\udd52\ud83e\udd52 DISCLAIMERS (more Veggies) \ud83e\udd52\ud83e\udd52\ud83e\udd52\n\n### The priority is code that works, **for me**.\n\nI actively use Lazy in development and testing.  I've tried to keep full test coverage for what's uploaded to pypi.  And it really works.  But, at the same time, whenever I need something new I usually just dump into into my app's custom CustomGenericLazyMixin and get it working there.\n\nIf it looks as if it can be useful, I'll put in `lazy-regression-tests`, but might not write tests for it.  Example: `CSSRemoveFilter`.\n\nSome of my custom functions really need to go back into the core, but they're only in the examples directory.  With things like `lazychecks_json` and `lazycheck_yaml`, they're often only in `lazy_regression_tests.examples.customlazy.CustomGenericLazyMixin`.\n\nThis is also why this write up and doc are ... light. \ud83d\udc47\n\n#### \ud83e\udd52\ud83e\udd52\ud83e\udd52TODO: \n- documenting the core classes\n- type-hinting\n- bit of refactoring \n- add Python 2.7 support \n    - 2.7-capable test code is especially important now\n- better support for unittest *and* pytest command lines\n\n\n\n### \ud83e\udd52\ud83e\udd52 You need to manage `diff` launches yourself.\n\nThe sample contains a template bash shell to launch the appropriate diff but getting something like that working is very much customization territory.  `find -cmin -5` or the like, in the `got` directory , can help you, but the general idea is you want to manage one error at a time with `pytest -x` or `unitest -f` switches.\n\n\n### \ud83e\udd52\ud83e\udd52\ud83e\udd52\ud83e\udd52 Stripping out transient and variable output is hard!\n\n- I've used diffing for a looong time.  The biggest barrier is avoiding constant comparison exceptions from\ndata that is expected to change.  That's what the filter utilities are for, but you still need to tweak\n**your** outputs.  Some classic gotchas:\n\n\t- timestamps\n\t- CSRF protection tokens\n\t- ORM auto-generated `id` keys\n\t- Webpack resource hashes\n\t- randomnly-ordered data\n\n- Related to that is the notion of formatters.  I run all my html through `BeautifulSoup.prettify()`.  Big\nhuge chunks of text with haphazardly located newlines **will** bite you.\n\n- **You** need to sort data, even if your application doesn't care.  Get into the habit of adding `ORDER BY ` to your \nqueries.\n\n\n### Some features work, but with messy code that I haven't adjusted yet.\n\nFor example, Lazy has the notion of directives and is supposed to get them from the command line and environment variables.\nIn practice, I now only use the environment variables so the command line handling code is crufty.  I'm still using both regular `unittest` and `pytest` so command line switches are an extra-sore point.\n\nThe core structure was written up very quickly, over about 2 days.  Some of the design choices are quite crufty in hindsight.  \n\n### You still need to write validation checks\n\nThat `self.assertEqual(response.status_code, 200)` was necessary because, **if your code breaks and starts returning 404s the last you want to do is telling Lazy that the warning page presented to the end user is now the expected behavior.**\n\n\n\n# \ud83c\udf70\ud83c\udf70\ud83c\udf70 Extra features\n\n\n### Directives\n\nenvironment variable `$lzrt_directive` can be used to manipulate lazy's behavior.  For example, if you've modified your templating system\nand all output is expected to change, then set `$lzrt_directive=baseline`.  Lazy will report errors, but continue without failing the tests \n**and** it will copy all received data to their match `exp` files.  Use it when you know it's appropriate and don't forget to reset it **immediately** afterwards.\n\n### SQL?  JSON?\n\nYup.  `self.lazychecks_sql(got)`  Watch your ORM code, for example.  \n\na formatter for sql can be as simple as:\n\n````\ndef format_sql(sql: str, *args, **kwds) -> str:\n    \"\"\"linefeed on commas and strip out empty lines\"\"\"\n\n    sql = sql.replace(\",\", \"\\n,\")\n    li = [line.lstrip() for line in sql.split(\"\\n\") if line.strip()]\n    return \"\\n\".join(li)\n\n````\n\ngiving:\n\n````\ninsert into bme_c_pspnlgrpdefn ( rdbname \n, market \n, actions \n, descr \n````\n\nJSON?: `self.lazychecks_json({\"some\" : \"data\"}})`\n\n\n### Complex composable objects?\n\nI've had *some* success taking arbitrary objects or dictionaries, pushing them through a `yaml.dump` and comparing them.  \n\nLet's say you a OrderProcessor object that gets composed from a reportStrategy object and a saveStrategy object.\n\nJust `self.lazycheck_yaml(order_processor_instance)`.\n\n#### Gotchas? \n\n- handling un-pickable and custom objects and attributes\n\t-Yaml dump is better at custom objects.\n- any variable attribute like a `OrderProcessor.todaysDate` variable.\n\n\n## P.S.  Not really a big \ud83c\udf70 lover and I am OK with \ud83e\udd52 ;-)\n\n=======\nHistory\n=======\n\n0.1.0 (2018-07-09)\n------------------\n\n* package created\n\n\n0.2.0 (2019-08-14)\n------------------\n\n* First release on PyPI.\n  \n\n0.2.1 (2019-08-15)\n------------------\n\n*  fixed the bad urls from lazy_regression_tests to lazy-regression-tests.  Github link should work now\n   \n0.2.3 (2019-08-18)\n------------------\n\n*  updated README.md",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/jpeyret/lazy-regression-tests",
        "keywords": "lazy_regression_tests",
        "license": "MIT license",
        "maintainer": "",
        "maintainer_email": "",
        "name": "lazy-regression-tests",
        "package_url": "https://pypi.org/project/lazy-regression-tests/",
        "platform": "",
        "project_url": "https://pypi.org/project/lazy-regression-tests/",
        "project_urls": {
            "Homepage": "https://github.com/jpeyret/lazy-regression-tests"
        },
        "release_url": "https://pypi.org/project/lazy-regression-tests/0.2.3/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "a very lazy way to regression test almost anything that has consistent structured text output",
        "version": "0.2.3"
    },
    "last_serial": 5695614,
    "releases": {
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6ef1874f71e5ce0326f8eec107b7ab17",
                    "sha256": "b058e43b53e5cf7385173cba817d92637a0690e3efa513f02702088fce8fa311"
                },
                "downloads": -1,
                "filename": "lazy_regression_tests-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "6ef1874f71e5ce0326f8eec107b7ab17",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 177153,
                "upload_time": "2019-08-14T21:14:35",
                "url": "https://files.pythonhosted.org/packages/8b/b9/13e17049068c18309964c24b6bc6a91a09709d06d18aa2606ad7e0d1735c/lazy_regression_tests-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "247ff586b05104bba4e45c5a418b5a45",
                    "sha256": "fd29d5fca479b36a824dffa751f57e900eeba64e716ae033a75ee1116e53524c"
                },
                "downloads": -1,
                "filename": "lazy_regression_tests-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "247ff586b05104bba4e45c5a418b5a45",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 177159,
                "upload_time": "2019-08-15T18:14:37",
                "url": "https://files.pythonhosted.org/packages/ba/1b/58c322b66c16e9eb00d120975f17c2fe92d4d60f45efecab195210ac1388/lazy_regression_tests-0.2.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b623c36565423eefb6ee6f0de6db0758",
                    "sha256": "89681bc8ae594bf74b46eb87e2b6371afac0c1c8ea55e4575b612d86a4a4ccb7"
                },
                "downloads": -1,
                "filename": "lazy_regression_tests-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "b623c36565423eefb6ee6f0de6db0758",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 125807,
                "upload_time": "2019-08-18T19:31:22",
                "url": "https://files.pythonhosted.org/packages/5c/dc/5eac477cda599e0e1bdc8edc7bf937aa3e41799fc2788f05b58040ff80f4/lazy_regression_tests-0.2.2.tar.gz"
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c17c40e73d4a1c3f50938e0fbbb37add",
                    "sha256": "ce73662a90977071ef2e92f29b0acef8f70d0f78e5a269eb1f862c77b91ca0c7"
                },
                "downloads": -1,
                "filename": "lazy_regression_tests-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "c17c40e73d4a1c3f50938e0fbbb37add",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 125855,
                "upload_time": "2019-08-18T19:50:42",
                "url": "https://files.pythonhosted.org/packages/fe/3d/98a390d6018231e68abfd6bff1bda042cfea1beac25499eacdbebb9b65ce/lazy_regression_tests-0.2.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "c17c40e73d4a1c3f50938e0fbbb37add",
                "sha256": "ce73662a90977071ef2e92f29b0acef8f70d0f78e5a269eb1f862c77b91ca0c7"
            },
            "downloads": -1,
            "filename": "lazy_regression_tests-0.2.3.tar.gz",
            "has_sig": false,
            "md5_digest": "c17c40e73d4a1c3f50938e0fbbb37add",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 125855,
            "upload_time": "2019-08-18T19:50:42",
            "url": "https://files.pythonhosted.org/packages/fe/3d/98a390d6018231e68abfd6bff1bda042cfea1beac25499eacdbebb9b65ce/lazy_regression_tests-0.2.3.tar.gz"
        }
    ]
}