{
    "info": {
        "author": "Rolando Espinoza La fuente",
        "author_email": "rndmax84@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "Programming Language :: Python"
        ],
        "description": "ScrapyDo\n========\n\nCrochet_-based blocking API for Scrapy_.\n\nThis module provides function helpers to run Scrapy_ in a blocking fashion. See\nthe `scrapydo-overview.ipynb <http://nbviewer.ipython.org/github/darkrho/scrapydo/blob/master/notebooks/scrapydo-overview.ipynb>`_\nnotebook for a quick overview of this module.\n\n\nInstallation\n============\n\nUsing ``pip``::\n\n  pip install scrapydo\n\n\nUsage\n=====\n\nThe function ``scrapydo.setup`` must be called once to initialize the reactor.\n\nExample:\n\n.. code:: python\n\n    import scrapydo\n    scrapydo.setup()\n\n    scrapydo.default_settings.update({\n        'LOG_LEVEL': 'DEBUG',\n        'CLOSESPIDER_PAGECOUNT': 10,\n    })\n\n    # Enable logging display\n    import logging\n    logging.basicConfig(level=logging.DEBUG)\n\n    # Fetch a single URL.\n    response = scrapydo.fetch(\"http://example.com\")\n\n    # Crawl an URL with given callback.\n    def parse_page(response):\n        yield {\n            'title': response.css('title').extract(),\n            'url': response.url,\n        }\n        for href in response.css('a::attr(href)'):\n            url = response.urljoin(href)\n            yield Request(url, callback=parse_page)\n\n    items = scrapydo.crawl('http://example.com', callback)\n\n    # Run an existing spider class.\n    spider_args = {'foo': 'bar'}\n    items = scrapydo.run_spider(MySpider, **spider_args)\n\n\nAvailable Functions\n===================\n\n``scrapydo.setup()``\n    Initialize reactor.\n\n``scrapydo.fetch(url, spider_cls=DefaultSpider, capture_items=True, return_crawler=False, settings=None, timeout=DEFAULT_TIMEOUT)``\n    Fetches an URL and returns the response.\n\n``scrapydo.crawl(url, callback, spider_cls=DefaultSpider, capture_items=True, return_crawler=False, settings=None, timeout=DEFAULT_TIMEOUT)``\n    Crawls an URL with given callback and returns the scraped items.\n\n``scrapydo.run_spider(spider_cls, capture_items=True, return_crawler=False, settings=None, timeout=DEFAULT_TIMEOUT, **kwargs)``\n    Runs a spider and returns the scraped items.\n\n``highlight(code, lexer='html', formatter='html', output_wrapper=None)``\n    Highlights given code using pygments. This function is suitable for use in a IPython notebook.\n\n\n.. _Scrapy: http://scrapy.org\n.. _Crochet: https://github.com/itamarst/crochet",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/rolando/scrapydo",
        "keywords": "",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "scrapydo",
        "package_url": "https://pypi.org/project/scrapydo/",
        "platform": "",
        "project_url": "https://pypi.org/project/scrapydo/",
        "project_urls": {
            "Homepage": "https://github.com/rolando/scrapydo"
        },
        "release_url": "https://pypi.org/project/scrapydo/0.2.2/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Crochet-based blocking API for Scrapy.",
        "version": "0.2.2"
    },
    "last_serial": 2664826,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "246a3ad9fdf91723c34e2009241c14d1",
                    "sha256": "f2b84a32d56bdb260b227ab304ef2add2847f00f9c4e0f470d6f4f96d0d30345"
                },
                "downloads": -1,
                "filename": "scrapydo-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "246a3ad9fdf91723c34e2009241c14d1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4525,
                "upload_time": "2015-07-27T04:51:07",
                "url": "https://files.pythonhosted.org/packages/b6/dc/7746d8b1677db3e025ee08b54e88247b90ef2564840a6d922284595e4720/scrapydo-0.1.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "144c0ca9e7aae39304cdc1ea335c934e",
                    "sha256": "7c302fe4261f89aaf9e9e358d9eb8b0875d5e39dab566eca6c3e7d046b26a558"
                },
                "downloads": -1,
                "filename": "scrapydo-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "144c0ca9e7aae39304cdc1ea335c934e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4915,
                "upload_time": "2017-02-24T02:47:46",
                "url": "https://files.pythonhosted.org/packages/5a/4c/3ee820d839f8a8339cb7e0dd3d2196b0c058bba92b97cb3517285eee7b0a/scrapydo-0.2.1.tar.gz"
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "02850c8f449b5b7f51816dad1db2d7e4",
                    "sha256": "376ce6fd133dfcefcc1ac5e2d61e768ea38866467c3ba36caa73b0e5b66876b9"
                },
                "downloads": -1,
                "filename": "scrapydo-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "02850c8f449b5b7f51816dad1db2d7e4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4888,
                "upload_time": "2017-02-24T03:16:42",
                "url": "https://files.pythonhosted.org/packages/0e/7b/99c06256a12e7c3e084cd559fee33a76599778e0fa04c3f10ae48d1afcbc/scrapydo-0.2.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "02850c8f449b5b7f51816dad1db2d7e4",
                "sha256": "376ce6fd133dfcefcc1ac5e2d61e768ea38866467c3ba36caa73b0e5b66876b9"
            },
            "downloads": -1,
            "filename": "scrapydo-0.2.2.tar.gz",
            "has_sig": false,
            "md5_digest": "02850c8f449b5b7f51816dad1db2d7e4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 4888,
            "upload_time": "2017-02-24T03:16:42",
            "url": "https://files.pythonhosted.org/packages/0e/7b/99c06256a12e7c3e084cd559fee33a76599778e0fa04c3f10ae48d1afcbc/scrapydo-0.2.2.tar.gz"
        }
    ]
}