{ "info": { "author": "Constverum", "author_email": "constverum@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Internet :: Proxy Servers", "Topic :: Internet :: WWW/HTTP :: Indexing/Search" ], "description": "ProxyBroker\n===========\n\n.. image:: https://img.shields.io/pypi/v/proxybroker.svg?style=flat-square\n :target: https://pypi.python.org/pypi/proxybroker/\n.. image:: https://img.shields.io/travis/constverum/ProxyBroker.svg?style=flat-square\n :target: https://travis-ci.org/constverum/ProxyBroker\n.. image:: https://img.shields.io/pypi/wheel/proxybroker.svg?style=flat-square\n :target: https://pypi.python.org/pypi/proxybroker/\n.. image:: https://img.shields.io/pypi/pyversions/proxybroker.svg?style=flat-square\n :target: https://pypi.python.org/pypi/proxybroker/\n.. image:: https://img.shields.io/pypi/l/proxybroker.svg?style=flat-square\n :target: https://pypi.python.org/pypi/proxybroker/\n\nProxyBroker is an open source tool that asynchronously finds public proxies from multiple sources and concurrently checks them.\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/index_find_example.gif\n\n\nFeatures\n--------\n\n* Finds more than 7000 working proxies from ~50 sources.\n* Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).\n* Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.\n* Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.\n* All proxies are checked to support Cookies and Referer (and POST requests if required).\n.. * Save found proxies to a file in custom format.\n* Automatically removes duplicate proxies.\n* Is asynchronous.\n\n\nRequirements\n------------\n\n* Python **3.5** or higher\n* `aiohttp `_\n* `aiodns `_\n* `maxminddb `_\n\n\nInstallation\n------------\n\nTo install last stable release from pypi:\n\n.. code-block:: bash\n\n $ pip install proxybroker\n\nThe latest development version can be installed directly from GitHub:\n\n.. code-block:: bash\n\n $ pip install -U git+https://github.com/constverum/ProxyBroker.git\n\n\nUsage\n-----\n\n\nCLI Examples\n~~~~~~~~~~~~\n\n\nFind\n\"\"\"\"\n\nFind and show 10 HTTP(S) proxies from United States with the high level of anonymity:\n\n.. code-block:: bash\n\n $ proxybroker find --types HTTP HTTPS --lvl High --countries US --strict -l 10\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_find_example.gif\n\n\nGrab\n\"\"\"\"\n\nFind and save to a file 10 US proxies (without a check): \n\n.. code-block:: bash\n\n $ proxybroker grab --countries US --limit 10 --outfile ./proxies.txt\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_grab_example.gif\n\n\nServe\n\"\"\"\"\"\n\nRun a local proxy server that distributes incoming requests to a pool\nof found HTTP(S) proxies with the high level of anonymity:\n\n.. code-block:: bash\n\n $ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High\n\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_serve_example.gif\n\n| Run ``proxybroker --help`` for more information on the options available.\n| Run ``proxybroker --help`` for more information on a command.\n\n\nBasic code example\n~~~~~~~~~~~~~~~~~~\n\nFind and show 10 working HTTP(S) proxies:\n\n.. code-block:: python\n\n import asyncio\n from proxybroker import Broker\n\n async def show(proxies):\n while True:\n proxy = await proxies.get()\n if proxy is None: break\n print('Found proxy: %s' % proxy)\n\n proxies = asyncio.Queue()\n broker = Broker(proxies)\n tasks = asyncio.gather(\n broker.find(types=['HTTP', 'HTTPS'], limit=10),\n show(proxies))\n\n loop = asyncio.get_event_loop()\n loop.run_until_complete(tasks)\n\n`More examples `_.\n\n\nDocumentation\n-------------\n\nhttps://proxybroker.readthedocs.io/\n\n\nTODO\n----\n\n* Check the ping, response time and speed of data transfer\n* Check site access (Google, Twitter, etc) and even your own custom URL's\n* Information about uptime\n* Checksum of data returned\n* Support for proxy authentication\n* Finding outgoing IP for cascading proxy\n* The ability to specify the address of the proxy without port (try to connect on defaulted ports)\n\n\nContributing\n------------\n\n* Fork it: https://github.com/constverum/ProxyBroker/fork\n* Create your feature branch: ``git checkout -b my-new-feature``\n* Commit your changes: ``git commit -am 'Add some feature'``\n* Push to the branch: ``git push origin my-new-feature``\n* Submit a pull request!\n\n\nLicense\n-------\n\nLicensed under the Apache License, Version 2.0\n\n*This product includes GeoLite2 data created by MaxMind, available from* `http://www.maxmind.com `_.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/constverum/ProxyBroker", "keywords": "proxy finder grabber scraper parser graber scrapper checker broker async asynchronous http https connect socks socks4 socks5", "license": "Apache License, Version 2.0", "maintainer": "", "maintainer_email": "", "name": "proxybroker", "package_url": "https://pypi.org/project/proxybroker/", "platform": "any", "project_url": "https://pypi.org/project/proxybroker/", "project_urls": { "Homepage": "https://github.com/constverum/ProxyBroker" }, "release_url": "https://pypi.org/project/proxybroker/0.3.2/", "requires_dist": [ "aiohttp (>=3.5.4)", "aiodns (>=2.0.0)", "attrs (==19.1.0)", "maxminddb (>=1.4.1)" ], "requires_python": ">=3.5.3", "summary": "[Finder/Checker/Server] Finds public proxies from multiple sources and concurrently checks them. Supports HTTP(S) and SOCKS4/5.", "version": "0.3.2" }, "last_serial": 4931681, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "2b40d2c8c45af103ea227fcf2134f077", "sha256": "85ac1c613349149a7b03fdde1b2ce9c62e95e920c1e4a1441796096ddd10eb17" }, "downloads": -1, "filename": "proxybroker-0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "2b40d2c8c45af103ea227fcf2134f077", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1103456, "upload_time": "2016-02-23T15:29:47", "url": "https://files.pythonhosted.org/packages/e3/48/d2255df6141fba1b645f5d20ff004641cc6bfc55af1173ca13be41140f67/proxybroker-0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "035d470617e39814236e948a2778d9b5", "sha256": "520e49028371db7e518f49bcd93e52d3317805b3395aa5f571703619fd524db0" }, "downloads": -1, "filename": "proxybroker-0.1.tar.gz", "has_sig": false, "md5_digest": "035d470617e39814236e948a2778d9b5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1097344, "upload_time": "2016-02-23T15:29:53", "url": "https://files.pythonhosted.org/packages/af/05/8a5e592e777b82a0d921245a0ec77dc3717f83db1eb6dd8b560c3865bc4c/proxybroker-0.1.tar.gz" } ], "0.1.1": [], "0.1.2": [ { "comment_text": "", "digests": { "md5": "632c6894e2611427f6752f48b57c0549", "sha256": "8cfab1e5d8a81fc659bcdfe39668e4c241f04f0e9167c5a039dfef577d968292" }, "downloads": -1, "filename": "proxybroker-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "632c6894e2611427f6752f48b57c0549", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1103648, "upload_time": "2016-02-27T11:11:00", "url": "https://files.pythonhosted.org/packages/53/74/5b8b37f9025180a128d3eab4816ce1bf8f4f3e724ec8995aa281100760e9/proxybroker-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4bb418800afb8a2432b8027a40c9a522", "sha256": "ef8afbad9360ff7b95da337d290480c90f3dac5a6358b0a6e620adc2a776bd09" }, "downloads": -1, "filename": "proxybroker-0.1.2.tar.gz", "has_sig": false, "md5_digest": "4bb418800afb8a2432b8027a40c9a522", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1439726, "upload_time": "2016-02-27T11:11:07", "url": "https://files.pythonhosted.org/packages/07/45/db9024eef3227d0b73ff129942cc6fb9e42e9efeb3e933ae27e4c9669d13/proxybroker-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "a1b1df7a04a482dc1256b070f7536fac", "sha256": "db857215acae6cc0879c30281b2ec397c62c359d529dc5d6e5bfa2eec5e01c0c" }, "downloads": -1, "filename": "proxybroker-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "a1b1df7a04a482dc1256b070f7536fac", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1447370, "upload_time": "2016-03-25T23:27:37", "url": "https://files.pythonhosted.org/packages/01/a1/91b7e2d60d2b664dfb9b13ab7cab892ed893ed0cf30ca7cb16acb52ea62b/proxybroker-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f09bdbd2ab72f5963e09b116411db9f5", "sha256": "85d0394c1e6a72a6ea7cb4996e94b0f88e0615c327f4d8a3519a8bc541147339" }, "downloads": -1, "filename": "proxybroker-0.1.3.tar.gz", "has_sig": false, "md5_digest": "f09bdbd2ab72f5963e09b116411db9f5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1442435, "upload_time": "2016-03-25T23:27:47", "url": "https://files.pythonhosted.org/packages/a9/af/5f1b2c3c371fbd3734dc5939554c01c6751bc14e51ce8d3224c448a2b941/proxybroker-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "b02b81023263dfa1d8a800ffce7f4737", "sha256": "171e15639574d58ef5f748b11aea010b8b900a6ce2863991ec38e93ae9fb138f" }, "downloads": -1, "filename": "proxybroker-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "b02b81023263dfa1d8a800ffce7f4737", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1447398, "upload_time": "2016-04-07T12:09:43", "url": "https://files.pythonhosted.org/packages/1c/d6/fb732c78529edef2ae4b0ccb4f4ca44b6654137aeb6e4d0ca7562ca28428/proxybroker-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d70e41ea923f5d737cb0739473542525", "sha256": "9fac03e7f1e2b8592c08dd830a6346ac79febbe7c1fc59e555f32149d30497bd" }, "downloads": -1, "filename": "proxybroker-0.1.4.tar.gz", "has_sig": false, "md5_digest": "d70e41ea923f5d737cb0739473542525", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1442523, "upload_time": "2016-04-07T12:09:49", "url": "https://files.pythonhosted.org/packages/e9/9c/67092acf7851c6f22e78e50b9b05c3475a4c4ea9e84aa6d50fc5cdf9d18b/proxybroker-0.1.4.tar.gz" } ], "0.1b4": [ { "comment_text": "", "digests": { "md5": "3888fb2099e69daf4023c374a6d34b98", "sha256": "efa6dc5814fef3f7a3558681c0074894054690e6211a9f803f66fba9a5508f23" }, "downloads": -1, "filename": "proxybroker-0.1b4-py3-none-any.whl", "has_sig": false, "md5_digest": "3888fb2099e69daf4023c374a6d34b98", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1078370, "upload_time": "2016-01-20T22:41:25", "url": "https://files.pythonhosted.org/packages/59/c2/bac91c19deaf7598914a59b708ee65cb0b6224f3eac5c832921a923d4a50/proxybroker-0.1b4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a5ab8f1c5be5250a26a64f9d432b802e", "sha256": "a8eb8592a5d73808568133af9af25a83ca9e04c4eb3c533d5007bed1123a9e92" }, "downloads": -1, "filename": "proxybroker-0.1b4.tar.gz", "has_sig": false, "md5_digest": "a5ab8f1c5be5250a26a64f9d432b802e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1072007, "upload_time": "2016-01-20T22:41:32", "url": "https://files.pythonhosted.org/packages/cb/69/1715c27865a940619ea2cc198e20bb57eedbec92e5a7be030ea46903fd16/proxybroker-0.1b4.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "b327e89649491923afe5928fd0356075", "sha256": "3eba7ee8f90693f1655a83a5d95ed3e6ab0daae94cfa40f28505719903dc9708" }, "downloads": -1, "filename": "proxybroker-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "b327e89649491923afe5928fd0356075", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1566829, "upload_time": "2017-09-17T16:41:22", "url": "https://files.pythonhosted.org/packages/67/b0/a058e13271318a8148d88e577b56bc2015ad2e8d848f868464594555ff1c/proxybroker-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "28d88321695f8e422da464af1a66eef6", "sha256": "95c96b6a8190f1d43256ad92f83402771bd4c81bf9667d50b0707c3ed12945e3" }, "downloads": -1, "filename": "proxybroker-0.2.0.tar.gz", "has_sig": false, "md5_digest": "28d88321695f8e422da464af1a66eef6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1555307, "upload_time": "2017-09-17T16:41:26", "url": "https://files.pythonhosted.org/packages/b9/e0/715affc21c919fe237460ab471832e97f0a10272145eefcd2fdf931624f7/proxybroker-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "f5ea379667a79b3c3abb7db1fabd4c9f", "sha256": "eb05fcea4015501fdd5b4f8009fbcc8a471ff957255e2d499f1d02b2b0bd4c6b" }, "downloads": -1, "filename": "proxybroker-0.3.0-py3-none-any.whl", "has_sig": false, "md5_digest": "f5ea379667a79b3c3abb7db1fabd4c9f", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1568476, "upload_time": "2017-10-31T09:55:37", "url": "https://files.pythonhosted.org/packages/5d/d0/b32482ab3d460ded77af48982fe79d4effaffccc02dbc502ee7047170ed8/proxybroker-0.3.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "57e1f64ff46795835f9abb71d53d1c9e", "sha256": "ce9a7352e1b92a4704f6dffe241e5b4a37b09aae86f1d69237a3f4bfd1bde394" }, "downloads": -1, "filename": "proxybroker-0.3.0.tar.gz", "has_sig": false, "md5_digest": "57e1f64ff46795835f9abb71d53d1c9e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1557527, "upload_time": "2017-10-31T09:55:39", "url": "https://files.pythonhosted.org/packages/8f/8b/e9297f60ad9866769465422042347794adedb988b0ef943439a635aa0530/proxybroker-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "b03ad42c494398a277a881835aad0edd", "sha256": "5c3fa8c0b6f3eb66fe0aa6c032671fa7f7bbed4947fad9c755535ea7f225402e" }, "downloads": -1, "filename": "proxybroker-0.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "b03ad42c494398a277a881835aad0edd", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 1568554, "upload_time": "2018-02-22T13:22:03", "url": "https://files.pythonhosted.org/packages/b8/46/e6f85d49cd9982cde7722aabc760d2a540fd2f1141ca3d3e967012306a1b/proxybroker-0.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f83b515c3ba83072ccbb958a0b0c96fa", "sha256": "e969740e7177d2c97f802094968cffc21f476c68e705dac2489166dbda6ed67b" }, "downloads": -1, "filename": "proxybroker-0.3.1.tar.gz", "has_sig": false, "md5_digest": "f83b515c3ba83072ccbb958a0b0c96fa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1557811, "upload_time": "2018-02-22T13:22:06", "url": "https://files.pythonhosted.org/packages/22/01/19acc197b3f772619a56d18c1be6ff0f8b0fa1698941a8ad9870dcbe8283/proxybroker-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "851a324425748289bac89ddd79f39497", "sha256": "2b02fdf903de4b0875c7b811bd95a7e76cb313e3a96b9069a4f37b9979f07674" }, "downloads": -1, "filename": "proxybroker-0.3.2-py3-none-any.whl", "has_sig": false, "md5_digest": "851a324425748289bac89ddd79f39497", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5.3", "size": 1570141, "upload_time": "2019-03-12T19:47:46", "url": "https://files.pythonhosted.org/packages/27/bd/7ffee9acbf97ab702f373678a9ac61aefbd59710b1544cf6c6a31d114203/proxybroker-0.3.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b27f377e978ba270fdd088ff3397e85b", "sha256": "a358533b8dc56c4500b54d8786c998e75687ff21843e6672e6b1409c867a9350" }, "downloads": -1, "filename": "proxybroker-0.3.2.tar.gz", "has_sig": false, "md5_digest": "b27f377e978ba270fdd088ff3397e85b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5.3", "size": 1560641, "upload_time": "2019-03-12T19:47:49", "url": "https://files.pythonhosted.org/packages/6b/b6/e44546093c4b0d9fd6e87e1d5d3bc0d24576bb71b4d7a48a95cb3933a019/proxybroker-0.3.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "851a324425748289bac89ddd79f39497", "sha256": "2b02fdf903de4b0875c7b811bd95a7e76cb313e3a96b9069a4f37b9979f07674" }, "downloads": -1, "filename": "proxybroker-0.3.2-py3-none-any.whl", "has_sig": false, "md5_digest": "851a324425748289bac89ddd79f39497", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.5.3", "size": 1570141, "upload_time": "2019-03-12T19:47:46", "url": "https://files.pythonhosted.org/packages/27/bd/7ffee9acbf97ab702f373678a9ac61aefbd59710b1544cf6c6a31d114203/proxybroker-0.3.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b27f377e978ba270fdd088ff3397e85b", "sha256": "a358533b8dc56c4500b54d8786c998e75687ff21843e6672e6b1409c867a9350" }, "downloads": -1, "filename": "proxybroker-0.3.2.tar.gz", "has_sig": false, "md5_digest": "b27f377e978ba270fdd088ff3397e85b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5.3", "size": 1560641, "upload_time": "2019-03-12T19:47:49", "url": "https://files.pythonhosted.org/packages/6b/b6/e44546093c4b0d9fd6e87e1d5d3bc0d24576bb71b4d7a48a95cb3933a019/proxybroker-0.3.2.tar.gz" } ] }