{ "info": { "author": "Ruediger Schmidt", "author_email": "ruediger.schmidt@gmx.net", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Software Development :: Build Tools" ], "description": "\n.. image:: https://travis-ci.org/rrschmidt/scrapy_tdd.svg?branch=master\n :target: https://travis-ci.org/rrschmidt/scrapy_tdd\n\n.. image:: https://codeclimate.com/github/codeclimate/codeclimate/badges/coverage.svg\n :target: https://codeclimate.com/github/rrschmidt/scrapy_tdd/coverage\n :alt: Test Coverage\n\nscrapy_tdd\n==========\n\nHelpers and examples to build Scrapy Crawlers in a test driven way.\n\nMotivation / Why should I develop Scrapy Crawlers using TDD?\n------------------------------------------------------------\n\n#. The develop - test cycle goes down to a few seconds and so it allows you to get a properly\n working scraper up much faster\n#. When bugs are discovered in \"the wild\" with real data, new example files, a test and a fix can be created and tested\n much faster\n#. It allows for fast refactoring without breaking anything - which results in much cleaner scraper code\n#. It just feels right when you are used to be doing TDD\n\nWhat's the difference to Scrapy's Spiders Contracts?\n----------------------------------------------------\n\nScrapy has its own builtin testing feature named `Spiders Contracts `_\n\nI tried to use them for some time, but decided to build real unit tests in a unit test framework like py.test because\nof these shortcomings:\n\n- its philosophy is geared towards testing against contracts (thus the name) that by nature are more broad and less\n specific concepts. Testing for exact field contents in items can be done, but is difficult and fragile\n- its documentation and basic set of features is a bit thin\n- it mixes implementation code with contract descriptions which is only usable when there are few and simple contracts\n\n\nInstallation\n============\n\n``pip install scrapy_tdd``\n\nQuick Start Examples\n====================\n\n def describe_fancy_spider():\n to_test = MySpider().from_crawler(get_crawler())\n\n def describe_parse_suggested_terms():\n resp = response_from(\"Result_JSON_Widget.txt\")\n results = to_test.parse(resp)\n\n def should_get_item():\n item = results\n assert item[0][\"lorem\"] == 'ipsum'\n assert item[0][\"iterem\"] == \"ipsem\"\n\n\nFull Documentation\n==================\n\n... coming soon ...\n\nMissing / next steps\n====================\n\n* Mocking Request-Response pairs\n\nHow to contribute\n=================\n\n... coming soon ...\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/rrschmidt/ScrapyTDD", "keywords": "scrapy development tdd test", "license": "GPLv3", "maintainer": "", "maintainer_email": "", "name": "scrapy-tdd", "package_url": "https://pypi.org/project/scrapy-tdd/", "platform": "", "project_url": "https://pypi.org/project/scrapy-tdd/", "project_urls": { "Homepage": "https://github.com/rrschmidt/ScrapyTDD" }, "release_url": "https://pypi.org/project/scrapy-tdd/0.1.3/", "requires_dist": null, "requires_python": "", "summary": "Helpers and examples to build Scrapy Crawlers in a test driven way.", "version": "0.1.3" }, "last_serial": 3309297, "releases": { "0.1.2rc1": [ { "comment_text": "", "digests": { "md5": "a54ca610a0ee17ee506616b526a63bea", "sha256": "cd7285a540ce019885282c467e93fff932a6655540e53a294cb07ab0da885dc3" }, "downloads": -1, "filename": "scrapy_tdd-0.1.2rc1.tar.gz", "has_sig": false, "md5_digest": "a54ca610a0ee17ee506616b526a63bea", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3802, "upload_time": "2017-05-02T19:55:40", "url": "https://files.pythonhosted.org/packages/43/58/0f5be4719cf6122e19461361bbbc72e418471b90ceedd59fd3127ed757f4/scrapy_tdd-0.1.2rc1.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "e131efed9659d2930fb7ab811a72cf16", "sha256": "f6db44a93c5e1291fb3445521e6bb79d0e025a161bd10104a0e2ea393d808cee" }, "downloads": -1, "filename": "scrapy_tdd-0.1.3.tar.gz", "has_sig": false, "md5_digest": "e131efed9659d2930fb7ab811a72cf16", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4933, "upload_time": "2017-11-06T12:48:22", "url": "https://files.pythonhosted.org/packages/a7/98/37c4a10599e46a6dd9c6a60ed0517c745a44995ac64f5ac89da4d1184d79/scrapy_tdd-0.1.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e131efed9659d2930fb7ab811a72cf16", "sha256": "f6db44a93c5e1291fb3445521e6bb79d0e025a161bd10104a0e2ea393d808cee" }, "downloads": -1, "filename": "scrapy_tdd-0.1.3.tar.gz", "has_sig": false, "md5_digest": "e131efed9659d2930fb7ab811a72cf16", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4933, "upload_time": "2017-11-06T12:48:22", "url": "https://files.pythonhosted.org/packages/a7/98/37c4a10599e46a6dd9c6a60ed0517c745a44995ac64f5ac89da4d1184d79/scrapy_tdd-0.1.3.tar.gz" } ] }