{
    "info": {
        "author": "DevAlone",
        "author_email": "dev@d3d.info",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Topic :: Internet"
        ],
        "description": "# proxy_py\n\nproxy_py is a program which collects proxies, saves them in a database\nand makes periodically checks.\nIt has a server for getting proxies with nice API(see below).\n\n## Where is the documentation?\n\nIt's [here](http://proxy-py.readthedocs.io)\n\n## How to build?\n\n1 Clone this repository\n\n`git clone https://github.com/DevAlone/proxy_py.git`\n\n2 Install requirements\n\n```bash\ncd proxy_py\npip3 install -r requirements.txt\n```\n\n3 Create settings file\n\n`cp config_examples/settings.py proxy_py/settings.py`\n\n4 Install postgresql and change database configuration in settings.py file\n\n5 (Optional) Configure alembic\n\n6 Run your application\n\n`python3 main.py`\n\n7 Enjoy!\n\n## I'm too lazy. Can I just use it?\n\n`TODO: update, old version!`\n\nYes, you can download virtualbox image\n[here](https://drive.google.com/file/d/1oPf6xwOADRH95oZW0vkPr1Uu_iLDe9jc/view?usp=sharing).\nAfter downloading check that port forwarding is still working,\nyou need forwarding of 55555 host port to 55555 guest.\n\n## How to get proxies?\n\nproxy_py has a server, based on aiohttp, which is listening 127.0.0.1:55555\n(you can change it in the settings file) and provides proxies.\nTo get proxies you should send the following json request\non address `http://127.0.0.1:55555/api/v1/`\n(or other domain if behind reverse proxy):\n\n```json\n{\n\t\"model\": \"proxy\",\n\t\"method\": \"get\",\n\t\"order_by\": \"response_time, uptime\"\n}\n```\n\nNote: order_by makes the result sorted\nby one or more fields(separated by comma).\nYou can skip it. The required fields are `model` and `method`.\n\nIt's gonna return you the json response like this:\n\n```json\n{\n\t\"count\": 1,\n\t\"data\": [{\n\t\t\t\"address\": \"http://127.0.0.1:8080\",\n\t\t\t\"auth_data\": \"\",\n\t\t\t\"bad_proxy\": false,\n\t\t\t\"domain\": \"127.0.0.1\",\n\t\t\t\"last_check_time\": 1509466165,\n\t\t\t\"number_of_bad_checks\": 0,\n\t\t\t\"port\": 8080,\n\t\t\t\"protocol\": \"http\",\n\t\t\t\"response_time\": 461691,\n\t\t\t\"uptime\": 1509460949\n\t\t}\n\t],\n\t\"has_more\": false,\n\t\"status\": \"ok\",\n\t\"status_code\": 200\n}\n```\n\nNote: All fields except *protocol*, *domain*, *port*, *auth_data*,\n*checking_period* and *address* CAN be null\n\nOr error if something went wrong:\n\n```json\n{\n    \"error_message\": \"You should specify \\\"model\\\"\",\n    \"status\": \"error\",\n    \"status_code\": 400\n}\n```\n\nNote: status_code is also duplicated in HTTP status code\n\nExample using curl:\n\n`curl -X POST http://127.0.0.1:55555/api/v1/ -H \"Content-Type: application/json\" --data '{\"model\": \"proxy\", \"method\": \"get\"}'`\n\nExample using httpie:\n\n`http POST http://127.0.0.1:55555/api/v1/ model=proxy method=get`\n\nExample using python's `requests` library:\n\n```python\nimport requests\nimport json\n\n\ndef get_proxies():\n    result = []\n    json_data = {\n        \"model\": \"proxy\",\n        \"method\": \"get\",\n    }\n\n    response = requests.post('http://127.0.0.1:55555/api/v1/', json=json_data)\n    if response.status_code == 200:\n        response = json.loads(response.text)\n        for proxy in response['data']:\n            result.append(proxy['address'])\n    else:\n        # check error here\n        pass\n\n    return result\n```\nExample using aiohttp library:\n\n```python\nimport aiohttp\n\n\nasync def get_proxies():\n    result = []\n    json_data = {\n        \"model\": \"proxy\",\n        \"method\": \"get\",\n    }\n\n    async with aiohttp.ClientSession() as session:\n        async with session.post('http://127.0.0.1:55555/api/v1/', json=json_data) as response:\n            if response.status == 200:\n                response = json.loads(await response.text())\n                for proxy in response['data']:\n                    result.append(proxy['address'])\n            else:\n                # check error here\n                pass\n\n    return result\n```\n\n## How to interact with API?\n\nRead more about API  [here](https://github.com/DevAlone/proxy_py/tree/master/docs/API.md)\n\n## How to contribute?\n\n`TODO: write guide about it`\n\n## How to test it?\n\nIf you made the changes to code and want to check that you didn't break\nanything, go [here](https://github.com/DevAlone/proxy_py/tree/master/docs/tests.md)\n\n## How to deploy on production using supervisor, nginx and postgresql in 8 steps?\n\n1 Install supervisor, nginx and postgresql\n\n`root@server:~$ apt install supervisor nginx postgresql`\n\n2 Create virtual environment and install requirements on it\n\n3 Copy settings.py example:\n\n`proxy_py@server:~/proxy_py$ cp config_examples/settings.py proxy_py/`\n\n4 create unprivileged user in postgresql database\nand change database authentication data in settings.py\n\n```bash\nproxy_py@server:~/proxy_py$ vim proxy_py/settings.py\n```\n\n```bash\nDATABASE_CONNECTION_KWARGS = {\n    'database': 'YOUR_POSTGRES_DATABASE',\n    'user': 'YOUR_POSTGRES_USER',\n    'password': 'YOUR_POSTGRES_PASSWORD',\n    # number of simultaneous connections\n    # 'max_connections': 20,\n}\n```\n\n5 Copy supervisor config example and change it for your case\n\n```bash\nroot@server:~$ cp /home/proxy_py/proxy_py/config_examples/proxy_py.supervisor.conf /etc/supervisor/conf.d/proxy_py.conf\nroot@server:~$ vim /etc/supervisor/conf.d/proxy_py.conf\n```\n\n6 Copy nginx config example, enable it and change if you need\n\n```bash\nroot@server:~$ cp /home/proxy_py/proxy_py/config_examples/proxy_py.nginx.conf /etc/nginx/sites-available/proxy_py\nroot@server:~$ ln -s /etc/nginx/sites-available/proxy_py /etc/nginx/sites-enabled/\nroot@server:~$ vim /etc/nginx/sites-available/proxy_py\n```\n\n7 Restart supervisor and Nginx\n\n```bash\nroot@server:~$ supervisorctl reread\nroot@server:~$ supervisorctl update\nroot@server:~$ /etc/init.d/nginx configtest\nroot@server:~$ /etc/init.d/nginx restart\n```\n\n8 Enjoy using it on your server!\n\n## What is it depend on?\n\nSee `requirements.txt`\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/DevAlone/proxy_py",
        "keywords": "proxy proxies proxy_collector aiohttp peewee_async",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "proxypy",
        "package_url": "https://pypi.org/project/proxypy/",
        "platform": "",
        "project_url": "https://pypi.org/project/proxypy/",
        "project_urls": {
            "Homepage": "https://github.com/DevAlone/proxy_py"
        },
        "release_url": "https://pypi.org/project/proxypy/2.0/",
        "requires_dist": [
            "yarl",
            "aiohttp",
            "aiosocks",
            "lxml",
            "PySocks",
            "fake-useragent",
            "aiohttp-jinja2",
            "jinja2",
            "peewee-async",
            "aiopg",
            "check-manifest; extra == 'dev'",
            "coverage; extra == 'test'"
        ],
        "requires_python": "",
        "summary": "Proxy collector",
        "version": "2.0"
    },
    "last_serial": 3746431,
    "releases": {
        "2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ebb3e2c7567308795b1f451582a8b6ad",
                    "sha256": "0be482da4ab82f260a2857810ca3e39aeab1b9fad018dc3a3cad443255781157"
                },
                "downloads": -1,
                "filename": "proxypy-2.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ebb3e2c7567308795b1f451582a8b6ad",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 3691,
                "upload_time": "2018-04-08T18:35:19",
                "url": "https://files.pythonhosted.org/packages/be/96/16b907b1b492374d6cdac88c2a4039f2799dbb8d1a7977149b23da7b1a09/proxypy-2.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "cbf68ef812a97b293a171018d78729ae",
                    "sha256": "fcc12ca7d8c94d42ad0c8802f7882ccfe73af38aa384eb54555df969d1ee1fe3"
                },
                "downloads": -1,
                "filename": "proxypy-2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "cbf68ef812a97b293a171018d78729ae",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19033,
                "upload_time": "2018-04-08T18:35:21",
                "url": "https://files.pythonhosted.org/packages/98/0c/c3753d0fd11f9d675f3b6f5db4ed6c510d1facecffeeed2341e6f4ba9a3f/proxypy-2.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "ebb3e2c7567308795b1f451582a8b6ad",
                "sha256": "0be482da4ab82f260a2857810ca3e39aeab1b9fad018dc3a3cad443255781157"
            },
            "downloads": -1,
            "filename": "proxypy-2.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "ebb3e2c7567308795b1f451582a8b6ad",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 3691,
            "upload_time": "2018-04-08T18:35:19",
            "url": "https://files.pythonhosted.org/packages/be/96/16b907b1b492374d6cdac88c2a4039f2799dbb8d1a7977149b23da7b1a09/proxypy-2.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "cbf68ef812a97b293a171018d78729ae",
                "sha256": "fcc12ca7d8c94d42ad0c8802f7882ccfe73af38aa384eb54555df969d1ee1fe3"
            },
            "downloads": -1,
            "filename": "proxypy-2.0.tar.gz",
            "has_sig": false,
            "md5_digest": "cbf68ef812a97b293a171018d78729ae",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 19033,
            "upload_time": "2018-04-08T18:35:21",
            "url": "https://files.pythonhosted.org/packages/98/0c/c3753d0fd11f9d675f3b6f5db4ed6c510d1facecffeeed2341e6f4ba9a3f/proxypy-2.0.tar.gz"
        }
    ]
}