{ "info": { "author": "Rolando Espinoza", "author_email": "rolando at rmax.io", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5" ], "description": "======================\nScrapy Inline Requests\n======================\n\n.. image:: https://img.shields.io/pypi/v/scrapy-inline-requests.svg\n :target: https://pypi.python.org/pypi/scrapy-inline-requests\n\n.. image:: https://img.shields.io/pypi/pyversions/scrapy-inline-requests.svg\n :target: https://pypi.python.org/pypi/scrapy-inline-requests\n\n.. image:: https://readthedocs.org/projects/scrapy-inline-requests/badge/?version=latest\n :target: https://readthedocs.org/projects/scrapy-inline-requests/?badge=latest\n :alt: Documentation Status\n\n.. image:: https://img.shields.io/travis/rolando/scrapy-inline-requests.svg\n :target: https://travis-ci.org/rolando/scrapy-inline-requests\n\n.. image:: https://codecov.io/github/rolando/scrapy-inline-requests/coverage.svg?branch=master\n :alt: Coverage Status\n :target: https://codecov.io/github/rolando/scrapy-inline-requests\n\n.. image:: https://landscape.io/github/rolando/scrapy-inline-requests/master/landscape.svg?style=flat\n :target: https://landscape.io/github/rolando/scrapy-inline-requests/master\n :alt: Code Quality Status\n\n.. image:: https://requires.io/github/rolando/scrapy-inline-requests/requirements.svg?branch=master\n :alt: Requirements Status\n :target: https://requires.io/github/rolando/scrapy-inline-requests/requirements/?branch=master\n\nA decorator for writing coroutine-like spider callbacks.\n\n* Free software: MIT license\n* Documentation: https://scrapy-inline-requests.readthedocs.org.\n* Python versions: 2.7, 3.4+\n\nQuickstart\n----------\n\nThe spider below shows a simple use case of scraping a page and following a few links:\n\n.. code:: python\n\n from inline_requests import inline_requests\n from scrapy import Spider, Request\n\n class MySpider(Spider):\n name = 'myspider'\n start_urls = ['http://httpbin.org/html']\n\n @inline_requests\n def parse(self, response):\n urls = [response.url]\n for i in range(10):\n next_url = response.urljoin('?page=%d' % i)\n try:\n next_resp = yield Request(next_url)\n urls.append(next_resp.url)\n except Exception:\n self.logger.info(\"Failed request %s\", i, exc_info=True)\n\n yield {'urls': urls}\n\n\nSee the ``examples/`` directory for a more complex spider.\n\n.. warning::\n\n The generator resumes its execution when a request's response is processed,\n this means the generator won't be resume after yielding an item or a request\n with it's own callback.\n\n\nKnown Issues\n------------\n\n* Middlewares can drop or ignore non-200 status responses causing the callback\n to not continue its execution. This can be overcome by using the flag\n ``handle_httpstatus_all``. See the `httperror middleware`_ documentation.\n* High concurrency and large responses can cause higher memory usage.\n* This decorator assumes your method have the following signature\n ``(self, response)``.\n* Wrapped requests may not be able to be serialized by persistent backends.\n* Unless you know what you are doing, the decorated method must be a spider\n method and return a **generator** instance.\n\n.. _`httperror middleware`: http://doc.scrapy.org/en/latest/topics/spider-middleware.html#scrapy.spidermiddlewares.httperror.HttpErrorMiddleware\n\n=======\nHistory\n=======\n\n\n0.3.1 (2016-07-04)\n------------------\n\n* Added deprecation about decorating non-spider functions.\n* Warn if the callback returns requests with callback or errback set. This\n reverts the compability with requests with callbacks.\n\n0.3.0 (2016-06-24)\n------------------\n* ~~Backward incompatible change: Added more restrictions to the request object (no callback/errback).~~\n* Cleanup callback/errback attributes before sending back the request to the\n generator. This fixes an edge case when using ``request.replace()``.\n* Simplified example spider.\n\n0.2.0 (2016-06-23)\n------------------\n\n* Python 3 support.\n\n\n0.1.2 (2016-05-22)\n------------------\n\n* Scrapy API and documentation updates.\n\n0.1.1 (2013-02-03)\n------------------\n\n* Minor tweaks and fixes.\n\n0.1.0 (2012-02-03)\n------------------\n\n* First release on PyPI.", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/rolando/scrapy-inline-requests", "keywords": "scrapy-inline-requests", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "scrapy-inline-requests", "package_url": "https://pypi.org/project/scrapy-inline-requests/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/scrapy-inline-requests/", "project_urls": { "Homepage": "https://github.com/rolando/scrapy-inline-requests" }, "release_url": "https://pypi.org/project/scrapy-inline-requests/0.3.1/", "requires_dist": [ "Scrapy (>=1.0)", "six (>=1.5)" ], "requires_python": "", "summary": "A decorator for writing coroutine-like spider callbacks.", "version": "0.3.1" }, "last_serial": 2202705, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "eda5e4517d878d7de9caf4feaa5f8e48", "sha256": "fa3b2085239893337df2f8b2f433c968a067003481f8a41c96014d91c7c2573d" }, "downloads": -1, "filename": "scrapy-inline-requests-0.1.tar.gz", "has_sig": false, "md5_digest": "eda5e4517d878d7de9caf4feaa5f8e48", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1484, "upload_time": "2012-02-03T22:13:13", "url": "https://files.pythonhosted.org/packages/4a/c3/cda8069772e70ac6fd1b302967f7f6984dd63706ca6aaee10cbd311adf22/scrapy-inline-requests-0.1.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "95ae110521a3493a99b23f3e954b9007", "sha256": "2b3e991b5b1644bc025b2938f5ae726a152cbdb8bd29fe540de6423aadceb978" }, "downloads": -1, "filename": "scrapy-inline-requests-0.1.1.tar.gz", "has_sig": false, "md5_digest": "95ae110521a3493a99b23f3e954b9007", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2556, "upload_time": "2013-02-04T00:33:49", "url": "https://files.pythonhosted.org/packages/bc/cf/c11635cd2a707f0947fe071336cc039b43139d64487be3950de99430817d/scrapy-inline-requests-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "2eaa7871d5c8c9932fd3f66df154c036", "sha256": "8c28fc4489b8f111e7f323c19ca2fdc733f7d51bdf618b9bafc1e849a622076e" }, "downloads": -1, "filename": "scrapy-inline-requests-0.1.2.tar.gz", "has_sig": false, "md5_digest": "2eaa7871d5c8c9932fd3f66df154c036", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2561, "upload_time": "2013-02-04T00:35:23", "url": "https://files.pythonhosted.org/packages/9d/13/562638eb418d50b23a843af1c14412d1a84aa20aaac0be4d2b10226a837b/scrapy-inline-requests-0.1.2.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "1b0d9c624bc2cc9139d029f556af815c", "sha256": "5a827bb1e8f287e80be29b003edb7957a07fb11278c44ea2e74c12f708b56fdd" }, "downloads": -1, "filename": "scrapy_inline_requests-0.2.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "1b0d9c624bc2cc9139d029f556af815c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4777, "upload_time": "2016-06-23T20:05:35", "url": "https://files.pythonhosted.org/packages/40/4b/fc449f58847a67f140d0c48c014138f863f5c4d36672a3cb7bdf3757d9b2/scrapy_inline_requests-0.2.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "70bc4c8061c5bbdd6dcc25b658cfbebe", "sha256": "b8995ab28eab9aaa5324f0a11bc2197ad39e952413d038f2b4e4051a4855bd5a" }, "downloads": -1, "filename": "scrapy-inline-requests-0.2.0.tar.gz", "has_sig": false, "md5_digest": "70bc4c8061c5bbdd6dcc25b658cfbebe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2949, "upload_time": "2016-06-23T20:05:39", "url": "https://files.pythonhosted.org/packages/8e/b4/9691dde4dc2092d6211b128caf23069d6e36815ac839eed70ff28b46d151/scrapy-inline-requests-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "7807004e623f6e42b8a9adec7f0526d8", "sha256": "3d74e5d89697b205dcb7958df3ed5567302c4da6508d9f65f1cd62d613d9f341" }, "downloads": -1, "filename": "scrapy_inline_requests-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7807004e623f6e42b8a9adec7f0526d8", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9554, "upload_time": "2016-06-24T03:22:49", "url": "https://files.pythonhosted.org/packages/8e/ba/35fc41a996b2c5ec6e87575c7369b8798dd30577bb8e21f6b601b103ae9e/scrapy_inline_requests-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "fd9e5db8d0a45f8f81cf51f077e69419", "sha256": "77ea3e973224368e39180e55c729d16e53b12b9960ae813cccc275919a03c5a4" }, "downloads": -1, "filename": "scrapy-inline-requests-0.3.0.tar.gz", "has_sig": false, "md5_digest": "fd9e5db8d0a45f8f81cf51f077e69419", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 223157, "upload_time": "2016-06-24T03:22:55", "url": "https://files.pythonhosted.org/packages/cb/30/9f568511b446f8bba790ac2f8a3a643f924da544d2e37fa3e63975580050/scrapy-inline-requests-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "e08db935ce4e4cfecec426b2c480d427", "sha256": "d5b5443e37aba5c3d0acf739f3b02354f24e705256a713c5069b0dbafd685f2e" }, "downloads": -1, "filename": "scrapy_inline_requests-0.3.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e08db935ce4e4cfecec426b2c480d427", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8157, "upload_time": "2016-07-05T01:14:25", "url": "https://files.pythonhosted.org/packages/49/a7/f5093677d9cdff3d6fffb1e2324e66c5c90719cde432027c8902004ed4cb/scrapy_inline_requests-0.3.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f68704db4c6b16244f5a1cd3730eb8d5", "sha256": "06e884dee63d8293180ed622a3a8c00125248144f94213ac81277ebd84224a4d" }, "downloads": -1, "filename": "scrapy-inline-requests-0.3.1.tar.gz", "has_sig": false, "md5_digest": "f68704db4c6b16244f5a1cd3730eb8d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19130, "upload_time": "2016-07-05T01:14:30", "url": "https://files.pythonhosted.org/packages/e5/5e/47c1266b9be69f23249e808c97649052bf7e2aeed30e5874fad9762d6a4b/scrapy-inline-requests-0.3.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e08db935ce4e4cfecec426b2c480d427", "sha256": "d5b5443e37aba5c3d0acf739f3b02354f24e705256a713c5069b0dbafd685f2e" }, "downloads": -1, "filename": "scrapy_inline_requests-0.3.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e08db935ce4e4cfecec426b2c480d427", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8157, "upload_time": "2016-07-05T01:14:25", "url": "https://files.pythonhosted.org/packages/49/a7/f5093677d9cdff3d6fffb1e2324e66c5c90719cde432027c8902004ed4cb/scrapy_inline_requests-0.3.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f68704db4c6b16244f5a1cd3730eb8d5", "sha256": "06e884dee63d8293180ed622a3a8c00125248144f94213ac81277ebd84224a4d" }, "downloads": -1, "filename": "scrapy-inline-requests-0.3.1.tar.gz", "has_sig": false, "md5_digest": "f68704db4c6b16244f5a1cd3730eb8d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19130, "upload_time": "2016-07-05T01:14:30", "url": "https://files.pythonhosted.org/packages/e5/5e/47c1266b9be69f23249e808c97649052bf7e2aeed30e5874fad9762d6a4b/scrapy-inline-requests-0.3.1.tar.gz" } ] }