{ "info": { "author": "Alexander Afanasyev", "author_email": "afanasieffav@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Framework :: Scrapy", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Programming Language :: Python", "Topic :: Internet :: WWW/HTTP" ], "description": ".. image:: https://badge.fury.io/py/scrapy-fake-useragent.svg\r\n :target: http://badge.fury.io/py/scrapy-fake-useragent\r\n :alt: PyPI version\r\n\r\n.. image:: https://requires.io/github/alecxe/scrapy-fake-useragent/requirements.svg?branch=master\r\n :target: https://requires.io/github/alecxe/scrapy-fake-useragent/requirements/?branch=master\r\n :alt: Requirements Status\r\n\r\nscrapy-fake-useragent\r\n=====================\r\n\r\nRandom User-Agent middleware based on\r\n`fake-useragent `__. It\r\npicks up ``User-Agent`` strings based on `usage\r\nstatistics `__\r\nfrom a `real world database `__.\r\n\r\nInstallation\r\n-------------\r\n\r\nThe simplest way is to install it via `pip`:\r\n\r\n pip install scrapy-fake-useragent\r\n\r\nConfiguration\r\n-------------\r\n\r\nTurn off the built-in ``UserAgentMiddleware`` and add\r\n``RandomUserAgentMiddleware``.\r\n\r\nIn Scrapy >=1.0:\r\n\r\n.. code:: python\r\n\r\n DOWNLOADER_MIDDLEWARES = {\r\n 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None,\r\n 'scrapy_fake_useragent.middleware.RandomUserAgentMiddleware': 400,\r\n }\r\n\r\nIn Scrapy <1.0:\r\n\r\n.. code:: python\r\n\r\n DOWNLOADER_MIDDLEWARES = {\r\n 'scrapy.contrib.downloadermiddleware.useragent.UserAgentMiddleware': None,\r\n 'scrapy_fake_useragent.middleware.RandomUserAgentMiddleware': 400,\r\n }\r\n\r\nConfiguring User-Agent type\r\n---------------------------\r\n\r\nThere's a configuration parameter ``RANDOM_UA_TYPE`` defaulting to ``random`` which is passed verbatim to the fake-user-agent. Therefore you can set it to say ``firefox`` to mimic only firefox browsers. Most useful though would be to use ``desktop`` or ``mobile`` values to send desktop or mobile strings respectively.\r\n\r\nUsage with `scrapy-proxies`\r\n---------------------------\r\n\r\nTo use with middlewares of random proxy such as `scrapy-proxies `_, you need:\r\n\r\n1. set ``RANDOM_UA_PER_PROXY`` to True to allow switch per proxy\r\n\r\n2. set priority of ``RandomUserAgentMiddleware`` to be greater than ``scrapy-proxies``, so that proxy is set before handle UA\r\n\r\n\r\n.. |GitHub version| image:: https://badge.fury.io/gh/alecxe%2Fscrapy-fake-useragent.svg\r\n :target: http://badge.fury.io/gh/alecxe%2Fscrapy-fake-useragent\r\n.. |Requirements Status| image:: https://requires.io/github/alecxe/scrapy-fake-useragent/requirements.svg?branch=master\r\n :target: https://requires.io/github/alecxe/scrapy-fake-useragent/requirements/?branch=master\r\n\r\nConfiguring Fake-UserAgent fallback\r\n-----------------------------------\r\n\r\nThere's a configuration parameter ``FAKEUSERAGENT_FALLBACK`` defaulting to\r\n``None``. You can set it to a string value, for example ``Mozilla`` or\r\n``Your favorite browser``, this configuration can completely disable any\r\nannoying exception.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/alecxe/scrapy-fake-useragent", "keywords": "scrapy proxy user-agent web-scraping", "license": "New BSD License", "maintainer": "", "maintainer_email": "", "name": "scrapy-fake-useragent", "package_url": "https://pypi.org/project/scrapy-fake-useragent/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/scrapy-fake-useragent/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/alecxe/scrapy-fake-useragent" }, "release_url": "https://pypi.org/project/scrapy-fake-useragent/1.1.0/", "requires_dist": null, "requires_python": null, "summary": "Use a random User-Agent provided by fake-useragent for every request", "version": "1.1.0" }, "last_serial": 2955031, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "e35b567fa3532fa0eb1e2706754852ba", "sha256": "697281fc0954fd29d7fc9e5ead74a6288d86e062db6fa419e980707d4da96eb5" }, "downloads": -1, "filename": "scrapy_fake_useragent-0.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e35b567fa3532fa0eb1e2706754852ba", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 3797, "upload_time": "2015-06-21T04:19:55", "url": "https://files.pythonhosted.org/packages/98/82/8158eed69fb3cca8bf5f960aa9156ae8b95b4af00e26d1deb7b8ff9e0964/scrapy_fake_useragent-0.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "304005fd14a21856b48c3cf13b2a82a7", "sha256": "88e557917164a22a4b85db4554f9016fbe0c245bea3d7a57a4adcd6d992213c4" }, "downloads": -1, "filename": "scrapy-fake-useragent-0.0.1.tar.gz", "has_sig": false, "md5_digest": "304005fd14a21856b48c3cf13b2a82a7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1832, "upload_time": "2015-06-21T04:12:35", "url": "https://files.pythonhosted.org/packages/93/ed/5579d4e2cfffdc82503bd76b5b1ec60a73f85de612a85a8550eb6020cd20/scrapy-fake-useragent-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "7f127d2e18ade59b0151529a59650a98", "sha256": "cec0677675761d4677b018161926a353a52185752690181d100c12dc5a015e88" }, "downloads": -1, "filename": "scrapy_fake_useragent-0.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7f127d2e18ade59b0151529a59650a98", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 4384, "upload_time": "2016-09-23T14:14:35", "url": "https://files.pythonhosted.org/packages/5b/b2/e658fc92548883709990b72acb6115ad0305881ca6b513382e516ceb1a01/scrapy_fake_useragent-0.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4fef19a59493d9194223804635ad5b5f", "sha256": "e8cd95c6274256faeda2fb48511e2f4f1eaf9f7dd6c48992d2cfe2a780e2d5ff" }, "downloads": -1, "filename": "scrapy-fake-useragent-0.0.2.tar.gz", "has_sig": false, "md5_digest": "4fef19a59493d9194223804635ad5b5f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2245, "upload_time": "2016-09-23T14:15:25", "url": "https://files.pythonhosted.org/packages/73/72/389d98e183f238b845ecf1f0865bf42f2055ad1481be58adbef1a93660a6/scrapy-fake-useragent-0.0.2.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "065eed0a6df8106fcd029f1717270c66", "sha256": "f1a9b4cb51eb3337672b364fd5022ba0116d9f1e8631fa0935e69d9f70e86bc7" }, "downloads": -1, "filename": "scrapy_fake_useragent-1.0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "065eed0a6df8106fcd029f1717270c66", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 4446, "upload_time": "2017-05-27T15:58:36", "url": "https://files.pythonhosted.org/packages/47/61/9018a06f5e65019f473c2f9c07b98c516c849913253e9b5cc7549b8b3075/scrapy_fake_useragent-1.0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "11c1e4d83f08bad13d189a62b2b1971c", "sha256": "a8f6c6feeba04b4ea9c67509c8cdab836707b37dec7c90dfa30822e5a97a2318" }, "downloads": -1, "filename": "scrapy-fake-useragent-1.0.2.1.tar.gz", "has_sig": false, "md5_digest": "11c1e4d83f08bad13d189a62b2b1971c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2309, "upload_time": "2017-05-27T15:58:49", "url": "https://files.pythonhosted.org/packages/7e/08/e87cbaa473ba2c9d13c1555d181f64657778a8aacd5036401b8ba1514614/scrapy-fake-useragent-1.0.2.1.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "51e477d4d6980305f7796af98637be93", "sha256": "7b7ab0854dfc443b974ff19816784b571a54170c396ed9f27e5180edce4a8492" }, "downloads": -1, "filename": "scrapy_fake_useragent-1.0.4.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "51e477d4d6980305f7796af98637be93", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 4895, "upload_time": "2017-05-27T16:03:35", "url": "https://files.pythonhosted.org/packages/72/68/a888e1150da52672b95983df33a7cbac04085b0bac39f92722663f420adc/scrapy_fake_useragent-1.0.4.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "944fb1424262e8a58fdc63a7e64bf5a1", "sha256": "c618285777b605b4fe2ad4157e214233c9e1be91d7f73712fa3546f09b4bc818" }, "downloads": -1, "filename": "scrapy-fake-useragent-1.0.4.1.tar.gz", "has_sig": false, "md5_digest": "944fb1424262e8a58fdc63a7e64bf5a1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2621, "upload_time": "2017-05-27T16:03:25", "url": "https://files.pythonhosted.org/packages/b8/d3/8987cfabc2b72b68f732c973c4f7389514160b524af7b7d2ec89a8e1ccfd/scrapy-fake-useragent-1.0.4.1.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "61d223b4a2d67ba4450bae720cf6b4f1", "sha256": "4898772a7c5e6f7c58269c895ade39aab87038d2efea73fbe15a4aab1335f0b6" }, "downloads": -1, "filename": "scrapy_fake_useragent-1.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "61d223b4a2d67ba4450bae720cf6b4f1", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 5282, "upload_time": "2017-06-16T16:39:49", "url": "https://files.pythonhosted.org/packages/0d/1b/c9e5f1917c87a09787271933dcd4bface93ee60248a6a85cc98f2fd58e2c/scrapy_fake_useragent-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8200cad1a93e9b68311d3f1252b2c168", "sha256": "cf004a651545585f2f05f0bd66cfe8c9df35778babf53bdfd856092d8b3b4068" }, "downloads": -1, "filename": "scrapy-fake-useragent-1.1.0.tar.gz", "has_sig": false, "md5_digest": "8200cad1a93e9b68311d3f1252b2c168", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2853, "upload_time": "2017-06-16T16:37:26", "url": "https://files.pythonhosted.org/packages/bd/66/2daceedbf66a879e799a7405988842f917d4f8655475b707b9944304e103/scrapy-fake-useragent-1.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "61d223b4a2d67ba4450bae720cf6b4f1", "sha256": "4898772a7c5e6f7c58269c895ade39aab87038d2efea73fbe15a4aab1335f0b6" }, "downloads": -1, "filename": "scrapy_fake_useragent-1.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "61d223b4a2d67ba4450bae720cf6b4f1", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 5282, "upload_time": "2017-06-16T16:39:49", "url": "https://files.pythonhosted.org/packages/0d/1b/c9e5f1917c87a09787271933dcd4bface93ee60248a6a85cc98f2fd58e2c/scrapy_fake_useragent-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8200cad1a93e9b68311d3f1252b2c168", "sha256": "cf004a651545585f2f05f0bd66cfe8c9df35778babf53bdfd856092d8b3b4068" }, "downloads": -1, "filename": "scrapy-fake-useragent-1.1.0.tar.gz", "has_sig": false, "md5_digest": "8200cad1a93e9b68311d3f1252b2c168", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2853, "upload_time": "2017-06-16T16:37:26", "url": "https://files.pythonhosted.org/packages/bd/66/2daceedbf66a879e799a7405988842f917d4f8655475b707b9944304e103/scrapy-fake-useragent-1.1.0.tar.gz" } ] }