{ "info": { "author": "Gabriel \"s0lst1c3\" Ryan", "author_email": "gabriel@solstice.me", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "Intended Audience :: System Administrators", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: MacOS :: MacOS X", "Operating System :: POSIX", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 2.7", "Topic :: Internet :: Proxy Servers", "Topic :: Internet :: WWW/HTTP", "Topic :: Security", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: System :: Systems Administration", "Topic :: Utilities" ], "description": "grey_harvest\n============\n\nScrapes the web for reliable http/https proxies and tests them for\nspeed and reliability. Can be used as both a python module and a \ncommand line utility. When run as a command line utility, proxies\nare sent to stdout. When run as a module, it returns a generator.\n\nCheck out the project on PyPi at https://pypi.python.org/pypi/grey_harvest/0.1.3.5\n\nKey Features\n------------\n- Quickly and easily generate a list of reliable http/https proxies\n- Usable as a comamnd line utility or a python module\n- Can filter for proxies that support SSL\n- Can filter for proxies locationed within specific countries\n- Can exclude proxies located within specific countries\n\nInstallation\n------------\n\nFirst, install the following dependencies::\n\n\t# On Centos/RHEL/Fedora:\n\tsudo yum install python-devel libxlt-devel libxml-devel\n\n\t# On Debian/Ubuntu:\n\tsudo apt-get install python-dev libxml2-dev libxslt1-dev\n\nThen install grey_harvest using pip as follows::\n\t\n\tpip install grey_harvest\n\nUsage\n-----\n\nWe can generate a list of 10 viable proxies with the following command::\n\n\t# use the -n flag to specify number of proxies to generate\n\tgrey_harvest -n 10\n\t\t\nTo select only proxies with SSL enabled, we do this::\n\n\t# use the -H flag to select only https proxies\n\tgrey_harvest -n 10 -H\n\nWe can use the -a flag to filter for proxies located within a\nlist of specific countries. For example, to choose proxies located\nwithin Ukraine, Hong Kong, and the United States, we'd use this::\n\n\t# use the -a flag to filter by country\n\tgrey_harvest -a \"United States\" \"Hong Kong\" Ukraine -n 10\n\n\nWe can use the -p flag to filter for ports running on specific ports::\n\n\t# the -p flag to only use proxies that run on port 80\n\tgrey_harvest -p 80 -n 10\n\n\nWe can deny proxies located within specific countries by using\nthe -d flag. Proxies located within China are blocked by default\nas they are often located behind the Great Firewall, and as such\ntend to be unreliable. This can be changed within grey_harvest.py's\ninternal configs.::\n\n\t# use the -d flag to deny proxies located within France and\n\t# Germany\n\tgrey_harvest -d France Germany -n 10 -H\n\ngrey_harvest library - basic example\n------------------------------------\n\nBefore diving into the documentation for the grey_harvest library,\ncheck out how easily we can generate a list of 20 proxies::\n\n\timport gray_harvest\n\n\t''' spawn a harvester '''\n\tharvester = grey_harvest.GreyHarvester()\n\n\t''' harvest some proxies from teh interwebz '''\n\tcount = 0\n\tfor proxy in harvester.run():\n\t\tprint proxy\n\t\tcount += 1\n\t\tif count >= 20:\n\t\t\tbreak\n\nThat's it. We now have 20 http/https proxies ready to go.\n\t\t\n\n\n\n\n\n\n.. :changelog:\n\nHistory\n=======\n\n0.1.5 (2016-04-20)\n------------------\n- Fixed connections errors that occur when specifying custom test domain\n\n0.1.4 (2016-04-19)\n------------------\n- Users can now filter for proxies running on specific ports\n\n0.1.3 (2015-05-26)\n------------------\n- Added documentation\n\n0.1.2 (2015-05-26)\n------------------\n\n- Corrected some build issues\n\n0.1.0 (2015-05-26)\n------------------\n\n- Initial release\n\n\nCredits\n=======\n\n\"grey_harvest\" is written and maintained by Gabriel 's0lst1c3' Ryan.\n\nContributors\n------------\n\nPlease add yourself here alphabetically when you submit your first pull request.", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/s0lst1c3/grey_harvest/tarball/0.1.2", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/s0lst1c3/grey_harvest", "keywords": null, "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "grey_harvest", "package_url": "https://pypi.org/project/grey_harvest/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/grey_harvest/", "project_urls": { "Download": "https://github.com/s0lst1c3/grey_harvest/tarball/0.1.2", "Homepage": "https://github.com/s0lst1c3/grey_harvest" }, "release_url": "https://pypi.org/project/grey_harvest/0.1.5/", "requires_dist": null, "requires_python": null, "summary": "Generate lists of free, reliable http(s) proxies.", "version": "0.1.5" }, "last_serial": 2074776, "releases": { "0.1.2": [ { "comment_text": "", "digests": { "md5": "b9c39d51e0cfa5d4b69bc7b0073c3588", "sha256": "c94ac315dce9a81604438ebbd4c8aafc97bee5a9914f20a5399ecbcb4bfbc39a" }, "downloads": -1, "filename": "grey_harvest-0.1.2-py2-none-any.whl", "has_sig": false, "md5_digest": "b9c39d51e0cfa5d4b69bc7b0073c3588", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 6098, "upload_time": "2015-05-26T09:38:23", "url": "https://files.pythonhosted.org/packages/a8/87/c69ce39e9aaab459c70690b56299f07af25b009b4a2874c5813e8d50454f/grey_harvest-0.1.2-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "a7721a8986a8fe95453f34bc93a53189", "sha256": "4683297415bcf58c32b4520b2c93f543d069a2b91359f2a1807e959164eb0ba9" }, "downloads": -1, "filename": "grey_harvest-0.1.2.tar.gz", "has_sig": false, "md5_digest": "a7721a8986a8fe95453f34bc93a53189", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5205, "upload_time": "2015-05-26T09:38:07", "url": "https://files.pythonhosted.org/packages/16/73/6535d6473b24c4f6288bb3f717d785dc50e289a80a1735685236d5bc73d9/grey_harvest-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "d5c96f98ff10560f853ee245c8a29112", "sha256": "0b4754970d761fef3b1af1d93a29c26638157930e8e9f3bae6b0a3605d83e054" }, "downloads": -1, "filename": "grey_harvest-0.1.3-py2-none-any.whl", "has_sig": false, "md5_digest": "d5c96f98ff10560f853ee245c8a29112", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 8026, "upload_time": "2015-05-26T10:51:24", "url": "https://files.pythonhosted.org/packages/d0/30/0a7a48e54ba23d918faee58661ba2185a691d25e648d22f7b1b9fc0cf4ce/grey_harvest-0.1.3-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "23772151b174cecc7dad5c64414b1e98", "sha256": "b2bc8ddc9b78bd7fd94ef82f9106e1354811c735821fb4ed6f9506a5e8214938" }, "downloads": -1, "filename": "grey_harvest-0.1.3.tar.gz", "has_sig": false, "md5_digest": "23772151b174cecc7dad5c64414b1e98", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6342, "upload_time": "2015-05-26T10:51:11", "url": "https://files.pythonhosted.org/packages/f3/fa/bb489ceecf7df03a184dd41a56a92b5e09def404c8125498bb932660b555/grey_harvest-0.1.3.tar.gz" } ], "0.1.3.5": [ { "comment_text": "", "digests": { "md5": "c4ceb7ae230e7543b357493ad515a859", "sha256": "a75207e06fc5e024da9de104b6f17512ff3c718d1b7f458242a471a0a331446c" }, "downloads": -1, "filename": "grey_harvest-0.1.3.5-py2-none-any.whl", "has_sig": false, "md5_digest": "c4ceb7ae230e7543b357493ad515a859", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 8089, "upload_time": "2015-05-26T10:58:45", "url": "https://files.pythonhosted.org/packages/e1/2d/c75816f6c030d76a3af3537eee3af9d1e0beb1e57ed1a8eb20af08e0c09e/grey_harvest-0.1.3.5-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "48a6698db3e8006a8c7cb06b39a20598", "sha256": "d4381e99ee5a8170ab80a51930901ea825616aa74330dfaccf95672aa033400c" }, "downloads": -1, "filename": "grey_harvest-0.1.3.5.tar.gz", "has_sig": false, "md5_digest": "48a6698db3e8006a8c7cb06b39a20598", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6358, "upload_time": "2015-05-26T10:58:05", "url": "https://files.pythonhosted.org/packages/31/13/b1c7aee01d361edf72f6683120cc4006a3c26d271ba3e6defe8583067c44/grey_harvest-0.1.3.5.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "e0d0e3932f7507d29a2444b1746bc14a", "sha256": "4def5e6b9b18c7ccff93041e7f36f9ed8f6454a210857f400073be4135a312d0" }, "downloads": -1, "filename": "grey_harvest-0.1.4.tar.gz", "has_sig": false, "md5_digest": "e0d0e3932f7507d29a2444b1746bc14a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6657, "upload_time": "2016-04-20T00:23:21", "url": "https://files.pythonhosted.org/packages/ad/7c/71a6f3aa51a043edbf80454fe37bf39f4060f693b458791ce6a8e6e0f08d/grey_harvest-0.1.4.tar.gz" } ], "0.1.5": [ { "comment_text": "", "digests": { "md5": "e575d9b53c959d2761ad7b97cd301f9a", "sha256": "3702a80d9742f8c507813da5a0498487ba4486df3ee81f7894e45ce627b17b79" }, "downloads": -1, "filename": "grey_harvest-0.1.5.tar.gz", "has_sig": false, "md5_digest": "e575d9b53c959d2761ad7b97cd301f9a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6708, "upload_time": "2016-04-20T19:33:06", "url": "https://files.pythonhosted.org/packages/4f/0e/16074fbb2a87d56290c711ed98e19d19974bef120258e2b8fee18c3ecfc6/grey_harvest-0.1.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e575d9b53c959d2761ad7b97cd301f9a", "sha256": "3702a80d9742f8c507813da5a0498487ba4486df3ee81f7894e45ce627b17b79" }, "downloads": -1, "filename": "grey_harvest-0.1.5.tar.gz", "has_sig": false, "md5_digest": "e575d9b53c959d2761ad7b97cd301f9a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6708, "upload_time": "2016-04-20T19:33:06", "url": "https://files.pythonhosted.org/packages/4f/0e/16074fbb2a87d56290c711ed98e19d19974bef120258e2b8fee18c3ecfc6/grey_harvest-0.1.5.tar.gz" } ] }