{ "info": { "author": "luoc", "author_email": "luoc@posteo.org", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6" ], "description": "# mosquito\n**a request obfuscator and web scraping toolkit**\n\n_mosquito_ gives you an API similar to [requests](https://3.python-requests.org/) and in fact uses \nit internally. Each HTTP request exposes a number of information such as **user agent** or \n**IP address** that allows a server to identify you or your application. _mosquito_ let's you set \nup multiple identities and schedules your requests to them. Each identity may consist of a whole \nbunch of attributes that are supported by _requests_'s \n[session object](https://3.python-requests.org/user/advanced/#session-objects) e.g.: **headers**, \n**proxies** or **cookies**. To list all attributes available execute \n`mosquito.available_attributes()`.\n\n## Installation\n**from PyPI**\n```bash\npip install mosquito\n```\n\n## Usage\n_demo/demo.py_ \n```python\n#!/usr/bin/env python3\n# Standard library modules.\n\n# Third party modules.\n\n# Local modules\nimport mosquito\nfrom mosquito_test import httpbin\n\n# Globals and constants variables.\n\n\n# Register attribute callback using a decorator ...\n@mosquito.attribute('headers')\ndef headers():\n for name in ('linux', 'mac', 'windows'):\n yield {'user-agent': name}\n\n\n# ... or register attributes by hand.\nmosquito.register_attributes(delay=.0, params=[{'foo': 42}, {'bar': 13, 'baz': 37}])\n\n\n# Let's list all available attributes.\nprint('available:', mosquito.available_attributes())\n\n\nwith mosquito.swarm(repeat_on=(503,), max_attempts=3) as scheduler:\n # Note that the swarm uses 2 sessions only, determined by the minimum length of passed\n # attributes which is `params` in our case.\n print(f'swarm uses {len(scheduler.swarm)} sessions')\n\n for i in range(5):\n # `swarm wraps` requests' api and therefore supports get, post, put etc.\n # parameters passed directly to request method will overwrite such registered before\n result = scheduler.get(httpbin('/user-agent'), params=dict(bar=0))\n print(i, result.url, result.json())\n\n # Let's provoke an error ...\n try:\n scheduler.get(httpbin('/status/404'))\n\n except mosquito.MosquitoError as mre:\n print(mre)\n\n # ... and another one, being more obstinate this time\n try:\n scheduler.get(httpbin('/status/503'))\n\n except mosquito.MosquitoError as mre:\n print(mre)\n```\n\n\n## Testing\nSome unit tests require a httpbin instance which is [httpbin.org](https://httpbin.org) by default. \nFor sake of speed and reliability it's recommended to run your own instance using the docker \nimage. Check [hub.docker.com/r/kennethreitz/httpbin](https://hub.docker.com/r/kennethreitz/httpbin/) \nfor more information.\n\n```bash\n# run httpbin server using docker\ndocker run -p 8080:80 kennethreitz/httpbin\n\n# let mosquito know its location by setting an environment variable\nexport HTTPBIN_BASE_URL=http://localhost:8080 \n```\n\nThe actual test is ran by:\n\n```bash\npython -m mosquito_test\n```\n\n## Feedback\nFor feedback of any kind write an issue at \n[gitlab.com](https://gitlab.com/luoc0815/mosquito/issues). Thank you for using _mosquito_.\n```\nmosquito \\ /\n \\ | /\n / \\ | / \\\n \\ \\|/ /\n \\, o^o ,/\n \\,/\"\\,/\n ,,,,----,{/X\\},----,,,,\n ,,---'''' _-'{\\X/}'-_ ''''---,,\n /' ,-'/ \\V/ \\'-, '\\\n( ,--''/ | (_) | \\''--, )\n '--,,-'' | | /_\\ | | ''-,,--'\n /' | (_-_) | '\\\n / /' \\_/ '\\ \\\n / / (_) \\ \\\n / V \\\n / \\\n / \\ \n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://gitlab.com/luoc0815/mosquito", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "mosquito", "package_url": "https://pypi.org/project/mosquito/", "platform": "", "project_url": "https://pypi.org/project/mosquito/", "project_urls": { "Homepage": "https://gitlab.com/luoc0815/mosquito" }, "release_url": "https://pypi.org/project/mosquito/0.2.0/", "requires_dist": [ "requests", "tqdm", "nose ; extra == 'develop'" ], "requires_python": "", "summary": "a request obfuscator and web scraping toolkit", "version": "0.2.0" }, "last_serial": 5478740, "releases": { "0.0.0": [ { "comment_text": "", "digests": { "md5": "0bf069c0a0d1541d80b16acb43c7b885", "sha256": "8228bc46230aa53d79ef38f93a84d71826f56734a61cce780fc81f7f8ed4615c" }, "downloads": -1, "filename": "mosquito-0.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "0bf069c0a0d1541d80b16acb43c7b885", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6332, "upload_time": "2019-05-04T17:12:01", "url": "https://files.pythonhosted.org/packages/74/7b/9204ec485d192f69aea32c4b966010810e61ef1cddf362eaf16c8e2be1d9/mosquito-0.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d3dd6dea38a3dd1648f6f5756078fe2a", "sha256": "9706f66eb2795f4f9f013067ae04a88d764e82d156c440527720818633b7fcd5" }, "downloads": -1, "filename": "mosquito-0.0.0.tar.gz", "has_sig": false, "md5_digest": "d3dd6dea38a3dd1648f6f5756078fe2a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4337, "upload_time": "2019-05-04T17:12:04", "url": "https://files.pythonhosted.org/packages/3b/05/ed4fe389df82337775fe673bcb317165ae9f7632c9c51254d62a0b0cd248/mosquito-0.0.0.tar.gz" } ], "0.1.0": [ { "comment_text": "", "digests": { "md5": "bfabf04a3e9719c250aa7999754f42c5", "sha256": "e9c0c0c32ff14e44fd6a5dcbdeb9c8955de2452068606736fef4fe866f70c4c4" }, "downloads": -1, "filename": "mosquito-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "bfabf04a3e9719c250aa7999754f42c5", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 19426, "upload_time": "2019-05-30T15:25:54", "url": "https://files.pythonhosted.org/packages/00/5a/fddf25b66ad5435a30811a02b7eebe2d93750511c1a0904b687920f65546/mosquito-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2b299fad3f33c650bbe195ee9ebca9f2", "sha256": "d7a9b4e628088dd0ddcbaea4137d1b56f95ac9e2f2cc7d4ef2b89fc954640744" }, "downloads": -1, "filename": "mosquito-0.1.0.tar.gz", "has_sig": false, "md5_digest": "2b299fad3f33c650bbe195ee9ebca9f2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16253, "upload_time": "2019-05-30T15:25:56", "url": "https://files.pythonhosted.org/packages/1f/4c/bde79b352fabb9ec6bd426c5e1844a378163582e6ff78d089b5a63f80b96/mosquito-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "084567ae95feff6d06257db9519deb35", "sha256": "6d510fd56f5c9f811b1816efac15694ec48e4d15546dbba9ec06a2a6ee2f7b60" }, "downloads": -1, "filename": "mosquito-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "084567ae95feff6d06257db9519deb35", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20356, "upload_time": "2019-07-02T23:11:20", "url": "https://files.pythonhosted.org/packages/cf/b8/47395b35ff150baf27f2f4d7fe47da770b659daea75fe406e132cffa7a67/mosquito-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "47630cb7c1b34423fd8b7079e66304b8", "sha256": "5826a22ce69877267121ab910b1edb1694c97cf31a71dbdac86da81936d19626" }, "downloads": -1, "filename": "mosquito-0.2.0.tar.gz", "has_sig": false, "md5_digest": "47630cb7c1b34423fd8b7079e66304b8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16690, "upload_time": "2019-07-02T23:11:22", "url": "https://files.pythonhosted.org/packages/ad/14/bd2742826adb226f6276a7a996baf484dea7d6072a1cf8cf0c9861569120/mosquito-0.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "084567ae95feff6d06257db9519deb35", "sha256": "6d510fd56f5c9f811b1816efac15694ec48e4d15546dbba9ec06a2a6ee2f7b60" }, "downloads": -1, "filename": "mosquito-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "084567ae95feff6d06257db9519deb35", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 20356, "upload_time": "2019-07-02T23:11:20", "url": "https://files.pythonhosted.org/packages/cf/b8/47395b35ff150baf27f2f4d7fe47da770b659daea75fe406e132cffa7a67/mosquito-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "47630cb7c1b34423fd8b7079e66304b8", "sha256": "5826a22ce69877267121ab910b1edb1694c97cf31a71dbdac86da81936d19626" }, "downloads": -1, "filename": "mosquito-0.2.0.tar.gz", "has_sig": false, "md5_digest": "47630cb7c1b34423fd8b7079e66304b8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16690, "upload_time": "2019-07-02T23:11:22", "url": "https://files.pythonhosted.org/packages/ad/14/bd2742826adb226f6276a7a996baf484dea7d6072a1cf8cf0c9861569120/mosquito-0.2.0.tar.gz" } ] }