{
"info": {
"author": "SpazioDati",
"author_email": "hello@spaziodati.eu",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 3 - Alpha",
"Environment :: Console",
"Intended Audience :: Developers",
"License :: OSI Approved :: BSD License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Topic :: Utilities"
],
"description": "scrapyrwiki\n===========\n\nA collection of helpers for running scrapers built with\n`Scrapy `_ in `ScraperWiki `_\n\n\nLaunch scraper without scrapy CLI\n---------------------------------\n\nExample:\n\n.. code:: python\n\n from scrapy.conf import settings\n from scrapyrwiki import run_spider\n\n def main():\n run_spider(MySpider(), settings)\n\n if __name__ == '__main__':\n main()\n\n\nSave produced data to ScraperWiki\n---------------------------------\n\nJust add \"scrapyrwiki.pipelines.ScraperWikiPipeline\" to ITEM_PIPELINES\n\nExample:\n\n.. code:: python\n\n from scrapy.conf import settings\n from scrapyrwiki import run_spider\n\n def scraperwiki():\n options = {\n 'SW_SAVE_BUFFER': 5,\n 'SW_UNIQUE_KEYS': {\"MyItem\": ['url']},\n 'ITEM_PIPELINES': ['scrapyrwiki.pipelines.ScraperWikiPipeline'],\n }\n settings.overrides.update(options)\n run_spider(MySpider(), settings)\n\n\n if __name__ == 'scraper':\n scraperwiki()\n\n\nCheck spider contracts in CI\n----------------------------\n\nJust launch spider with run_tests\n\nExample:\n\n.. code:: python\n\n from scrapyrwiki import run_tests\n from scrapy.conf import settings\n\n run_tests(MySpider(), \"output.xml\", settings)\n\nNote: For testing the HTTP cache is used. In the directory where the script is\nlaunched there must be a scrapy.cfg (needed by Scrapy to identify that's a scraper\ndirectory) and a .scrapy directory with the HTTP cache db.\n\nThe output is in XUnit format, tested on `Jenkins `_\n\n\nLog scraper errors to Sentry\n----------------------------\n\nInstall `scrapy-sentry `_ and set the\nenvironment variable SENTRY_DSN with the Sentry key. Scrapyrwiki will handle\neverything for you.",
"description_content_type": null,
"docs_url": null,
"download_url": "UNKNOWN",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "http://github.com/SpazioDati/scrapyrwiki",
"keywords": null,
"license": "BSD",
"maintainer": null,
"maintainer_email": null,
"name": "scrapyrwiki",
"package_url": "https://pypi.org/project/scrapyrwiki/",
"platform": "UNKNOWN",
"project_url": "https://pypi.org/project/scrapyrwiki/",
"project_urls": {
"Download": "UNKNOWN",
"Homepage": "http://github.com/SpazioDati/scrapyrwiki"
},
"release_url": "https://pypi.org/project/scrapyrwiki/0.2/",
"requires_dist": null,
"requires_python": null,
"summary": "A collection of helpers for running Scrapy in ScraperWiki",
"version": "0.2"
},
"last_serial": 799339,
"releases": {
"0.1": [
{
"comment_text": "",
"digests": {
"md5": "7791e8c63c2e1bccd3add0dfa0610d23",
"sha256": "9b81189ad4bbc56061e4c211adb9963669a6ca7780f6ecef35ead176881d05d6"
},
"downloads": -1,
"filename": "scrapyrwiki-0.1.tar.gz",
"has_sig": false,
"md5_digest": "7791e8c63c2e1bccd3add0dfa0610d23",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3353,
"upload_time": "2013-01-28T14:39:56",
"url": "https://files.pythonhosted.org/packages/d5/b9/8c7de81a92f45267638c5f402e25a34e907f833238c1a2fde00d26887103/scrapyrwiki-0.1.tar.gz"
}
],
"0.2": [
{
"comment_text": "",
"digests": {
"md5": "edcec4d73d677c3f89507aebde4d5edd",
"sha256": "0afe100bdbc403955228309d9942066a13278d99843630a89a0e25ec72f4dcd8"
},
"downloads": -1,
"filename": "scrapyrwiki-0.2.tar.gz",
"has_sig": false,
"md5_digest": "edcec4d73d677c3f89507aebde4d5edd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3525,
"upload_time": "2013-02-27T09:38:19",
"url": "https://files.pythonhosted.org/packages/56/69/f6486c5083066040f0461e24ae39cc371c43728230cfb9fd91618207b9b7/scrapyrwiki-0.2.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "edcec4d73d677c3f89507aebde4d5edd",
"sha256": "0afe100bdbc403955228309d9942066a13278d99843630a89a0e25ec72f4dcd8"
},
"downloads": -1,
"filename": "scrapyrwiki-0.2.tar.gz",
"has_sig": false,
"md5_digest": "edcec4d73d677c3f89507aebde4d5edd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3525,
"upload_time": "2013-02-27T09:38:19",
"url": "https://files.pythonhosted.org/packages/56/69/f6486c5083066040f0461e24ae39cc371c43728230cfb9fd91618207b9b7/scrapyrwiki-0.2.tar.gz"
}
]
}