{ "info": { "author": "nekidaem", "author_email": "order@nekidaem.ru", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# proxydata-scrapy\n\n**\u0420\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u0435 \u0434\u043b\u044f \u043f\u0440\u043e\u043a\u0441\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u0432 \u043f\u0430\u0443\u043a\u043e\u0432 scrapy \u0447\u0435\u0440\u0435\u0437 \u0441\u0435\u0440\u0432\u0438\u0441 [proxyfordevelopers.com][site]**\n\n## \u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430\n\n\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430 \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e `pip`\n```\npip install proxydata-scrapy\n```\n\n\u041b\u0438\u0431\u043e \u0441\u043a\u043e\u043f\u0438\u0440\u0443\u0439\u0442\u0435 middleware \u043a\u043b\u0430\u0441\u0441 `PFDDownloaderMiddleware` \u0432 \u0444\u0430\u0439\u043b `middlewares.py` \u0441\u0432\u043e\u0435\u0433\u043e \u043f\u0440\u043e\u0435\u043a\u0442\u0430\n\n\u0414\u043e\u0431\u0430\u0432\u044c\u0442\u0435 \u0440\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u0435 \u0432 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u0443\u044e `DOWNLOADER_MIDDLEWARES` \u0432 `settings.py` \u0432\u0430\u0448\u0435\u0433\u043e \u043f\u0440\u043e\u0435\u043a\u0442\u0430.\n\u0422\u0430\u043a-\u0436\u0435 \u0434\u043b\u044f \u043a\u043e\u0440\u0440\u0435\u0442\u043d\u043e\u0439 \u0440\u0430\u0431\u043e\u0442\u044b \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u043e\u0442\u043a\u043b\u044e\u0447\u0438\u0442\u044c middlewares `RobotsTxtMiddleware` \u0438 `HttpCompressionMiddleware` \n\n\u0414\u043e\u0431\u0430\u0432\u044c\u0442\u0435 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u0443\u044e `PFD_TOKEN` \u0432 `settings.py` \u0432\u0430\u0448\u0435\u0433\u043e \u043f\u0440\u043e\u0435\u043a\u0442\u0430.\n\u0414\u043b\u044f \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u044f \u0442\u043e\u043a\u0435\u043d\u0430 \u0432\u0430\u043c \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u0437\u0430\u0440\u0435\u0433\u0438\u0441\u0442\u0440\u0438\u0440\u043e\u0432\u0430\u0442\u044c\u0441\u044f \u0432 \u0441\u0435\u0440\u0432\u0438\u0441\u0435 [proxyfordevelopers][site]\n\n```\nDOWNLOADER_MIDDLEWARES = {\n 'scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware': None,\n 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': None,\n 'proxydata_scrapy.middlewares.PFDDownloaderMiddleware': 800,\n}\n\nPFD_TOKEN = 'xyz'\n```\n\n\n## \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\n\n\u041f\u0440\u0438 \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e\u0441\u0442\u0438 \u043f\u0435\u0440\u0435\u0434\u0430\u0447\u0438 \u0434\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u043f\u0430\u0440\u0430\u043c\u0435\u0442\u0440\u043e\u0432 \u0432 API **proxyfordevelopers.com** \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0439\u0442\u0435 \u043f\u0435\u0440\u0435\u043c\u0435\u043d\u043d\u0443\u044e `cb_kwargs` \u043a\u043b\u0430\u0441\u0441\u0430 `scrapy.http.Request`\n\u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440:\n```\nclass QuotesSpider(scrapy.Spider):\n name = \"quotes\"\n start_urls = [\n 'http://quotes.toscrape.com/page/1/',\n ]\n\n def start_requests(self):\n cb_kwargs = {\n 'proxy_type': 1,\n 'js_enabled': True,\n 'country': 'RU',\n 'protocol': 1\n }\n for url in self.start_urls:\n yield Request(url, dont_filter=True, cb_kwargs=cb_kwargs)\n```\n\n\n[site]: https://proxyfordevelopers.com\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "proxydata-scrapy", "package_url": "https://pypi.org/project/proxydata-scrapy/", "platform": "", "project_url": "https://pypi.org/project/proxydata-scrapy/", "project_urls": null, "release_url": "https://pypi.org/project/proxydata-scrapy/0.0.2b0/", "requires_dist": null, "requires_python": ">=3.6", "summary": "Middleware for proxying scrapy requests through the service proxyfordevelopers.com", "version": "0.0.2b0" }, "last_serial": 5959865, "releases": { "0.0.1b0": [ { "comment_text": "", "digests": { "md5": "e82de714490fff991ad80c0a18c724e8", "sha256": "4b399a0656ff7ea9cd5320163db3281315408f77397640fe3672fc15cb247e82" }, "downloads": -1, "filename": "proxydata_scrapy-0.0.1b0-py3-none-any.whl", "has_sig": false, "md5_digest": "e82de714490fff991ad80c0a18c724e8", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6", "size": 5220, "upload_time": "2019-10-11T11:22:20", "url": "https://files.pythonhosted.org/packages/57/88/ffa6613fdae83dbb784761cc0e00c4abbe15ca120462f1f9f646b085d3bd/proxydata_scrapy-0.0.1b0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3fb297cd07b348622f8eccd21e655907", "sha256": "50e53473c9d116964dec3beb5d48216696aa03f22cbd8e883759613ac0891f30" }, "downloads": -1, "filename": "proxydata-scrapy-0.0.1b0.tar.gz", "has_sig": false, "md5_digest": "3fb297cd07b348622f8eccd21e655907", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 3816, "upload_time": "2019-10-11T11:22:22", "url": "https://files.pythonhosted.org/packages/1c/8b/fb2bc4198906667f77fd22dd8ee0bc44d432d0992cb3e25cf652dda40f57/proxydata-scrapy-0.0.1b0.tar.gz" } ], "0.0.2b0": [ { "comment_text": "", "digests": { "md5": "a73b58dad122013de0e7fb1bc72f7928", "sha256": "98b4219f1b9f7a450f59965d3ceaa869ad467a09ba338cf39c8f807d9dffa6ec" }, "downloads": -1, "filename": "proxydata_scrapy-0.0.2b0-py3-none-any.whl", "has_sig": false, "md5_digest": "a73b58dad122013de0e7fb1bc72f7928", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6", "size": 5219, "upload_time": "2019-10-11T11:25:32", "url": "https://files.pythonhosted.org/packages/07/ac/41734dc9d254e56ae070e472e0fc494b1d3253b86308b27f04886937a483/proxydata_scrapy-0.0.2b0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "bfa15846caf8558cb3a3e95e0d38502e", "sha256": "c9d5cf3d92cf05dfc098f99f4df6c0d7b46fd9b38eabe233ca8c8d467fc0ccfe" }, "downloads": -1, "filename": "proxydata-scrapy-0.0.2b0.tar.gz", "has_sig": false, "md5_digest": "bfa15846caf8558cb3a3e95e0d38502e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 3815, "upload_time": "2019-10-11T11:25:34", "url": "https://files.pythonhosted.org/packages/19/27/a9ab3c0caa22b84a03f2268c39aea208f9ddaf5cc3deff4ade07b4104143/proxydata-scrapy-0.0.2b0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "a73b58dad122013de0e7fb1bc72f7928", "sha256": "98b4219f1b9f7a450f59965d3ceaa869ad467a09ba338cf39c8f807d9dffa6ec" }, "downloads": -1, "filename": "proxydata_scrapy-0.0.2b0-py3-none-any.whl", "has_sig": false, "md5_digest": "a73b58dad122013de0e7fb1bc72f7928", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6", "size": 5219, "upload_time": "2019-10-11T11:25:32", "url": "https://files.pythonhosted.org/packages/07/ac/41734dc9d254e56ae070e472e0fc494b1d3253b86308b27f04886937a483/proxydata_scrapy-0.0.2b0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "bfa15846caf8558cb3a3e95e0d38502e", "sha256": "c9d5cf3d92cf05dfc098f99f4df6c0d7b46fd9b38eabe233ca8c8d467fc0ccfe" }, "downloads": -1, "filename": "proxydata-scrapy-0.0.2b0.tar.gz", "has_sig": false, "md5_digest": "bfa15846caf8558cb3a3e95e0d38502e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 3815, "upload_time": "2019-10-11T11:25:34", "url": "https://files.pythonhosted.org/packages/19/27/a9ab3c0caa22b84a03f2268c39aea208f9ddaf5cc3deff4ade07b4104143/proxydata-scrapy-0.0.2b0.tar.gz" } ] }