{
    "info": {
        "author": "Konstantin Lopuhin, Mikhail Korobov",
        "author_email": "kostia.lopuhin@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Topic :: Internet :: WWW/HTTP :: Indexing/Search"
        ],
        "description": "Autologin middleware\n====================\n\n.. image:: https://img.shields.io/pypi/v/autologin-middleware.svg\n   :target: https://pypi.python.org/pypi/autologin-middleware\n   :alt: PyPI Version\n\n.. image:: https://img.shields.io/travis/TeamHG-Memex/autologin-middleware/master.svg\n   :target: http://travis-ci.org/TeamHG-Memex/autologin-middleware\n   :alt: Build Status\n\n.. image:: https://codecov.io/github/TeamHG-Memex/autologin-middleware/coverage.svg?branch=master\n   :target: https://codecov.io/github/TeamHG-Memex/autologin-middleware?branch=master\n   :alt: Code Coverage\n\nThis is a a Scrapy middleware that uses\n`autologin <https://github.com/TeamHG-Memex/autologin>`_ http-api\nto maintain a logged-in state for a scrapy spider.\n\nAutologin middleware uses autologin to make all requests while being\nlogged in. It uses autologin to get cookies, detects logouts and tries\nto avoid them in the future. A single authorization domain for the spider\nis assumed. Autologin middleware also puts ``autologin_active`` into\n``request.meta``, which is ``True`` only if we are logged in\n(and to ``False`` if domain is skipped or login failed).\nIf requests are made via `splash <http://splash.readthedocs.org>`_\n(and ``SPLASH_URL`` is set),\nautologin middleware passes it to autologin,\nand this splash instance is also used to obtain login cookies.\n\nInstallation\n------------\n\nIt works on python 2.7 and python 3, and requires at least scrapy 1.1.\nInstall with pip::\n\n    pip install autologin-middleware\n\n\nConfiguration\n-------------\n\n1. Include the autologin middleware into the project settings\n   and specify autologin url::\n\n    AUTOLOGIN_URL = 'http://127.0.0.1:8089'\n    AUTOLOGIN_ENABLED = True\n    DOWNLOADER_MIDDLEWARES['autologin_middleware.AutologinMiddleware'] = 605\n\n2. Cookie support is also required. There are currently several options:\n\n  - scrapy cookie middleware (``COOKIES_ENABLED = True``),\n    but autologin middleware requires access to cookies, so you need to enable\n    a custom cookie middleware::\n\n        DOWNLOADER_MIDDLEWARES = {\n            'autologin_middleware.AutologinMiddleware': 605,\n            'scrapy.downloadermiddlewares.cookies.CookiesMiddleware': None,\n            'autologin_middleware.ExposeCookiesMiddleware': 700,\n        }\n\n  - `scrapy-splash <https://github.com/scrapy-plugins/scrapy-splash>`_\n    cookie middleware (``scrapy_splash.SplashCookiesMiddleware``)\n  - any other middleware that gets cookies from ``request.cookies`` and\n    sets ``response.cookiejar`` like scrapy-splash middleware,\n    or exposes them in ``response.flags`` like ``ExposeCookiesMiddleware``.\n\n3. Optional but highly recommended: avoid logouts - see optional settings and\n   \"Avoiding logouts\" section below. The reason why it's important is that\n   logout detection is less robust, on some sites the old cookies might still\n   work so you'll be using multiple sessions, etc. If you don't ever logout,\n   you avoid all this problems.\n\nThere are some optional settings:\n\n- ``AUTOLOGIN_COOKIES``: pass auth cookies after manual login\n  (format is ``name=value; name2=value2``).\n- ``AUTOLOGIN_LOGOUT_URL``: pass url substring to avoid.\n- ``AUTOLOGIN_CHECK_LOGOUT``: set to ``False`` in order to disable automatic\n  logout detection: it remembers cookies obtained during login and\n  checks them on each response to see if any disappeared. This can be\n  problematic for sites that set a lot of cookies on login,\n  so this is an option to disable it.\n  If you disable it, you must rely on avoiding logout links with\n  ``link_looks_like_logout`` (see below), or setting a custom\n  ``AUTOLOGIN_LOGOUT_URL``.\n- ``AUTOLOGIN_USERNAME``, ``AUTOLOGIN_PASSWORD``, ``AUTOLOGIN_LOGIN_URL``,\n  ``AUTOLOGIN_EXTRA_JS`` are passed to autologin and override values\n  from stored credentials.  ``AUTOLOGIN_LOGIN_URL`` is a relative url,\n  and can be omitted if it is the same as the start url.\n  ``AUTOLOGIN_EXTRA_JS`` is required only if you want to use the ``extra_js``\n  feature of the autologin.\n\nIt is also possible to override some settings per-request via corresponding\nlower-case keys in ``request.meta``: ``autologin_username``,\n``autologin_password``, ``autologin_extra_js``, ``autologin_login_url`` and\n``autologin_logout_url``.\n\nAutologin middleware passes the following settings to the autologin:\n``SPLASH_URL``, ``USER_AGENT``, ``HTTP_PROXY``, ``HTTPS_PROXY``, so they\nare used for autologin requests.\n\nAvoiding logouts\n----------------\n\nThere is also an utility ``autologin_middleware.link_looks_like_logout``\nfor checking if a links looks like a logout link: you can use it in the\nspider to avoid logout links. Logouts are handled\nby the autologin middleware by default\n(unless ``AUTOLOGIN_CHECK_LOGOUT`` is ``False``),\nbut avoiding logout links can be beneficial for two reasons:\n\n- no time is waster retrying requests that were logged out\n- in some cases, logout urls can be unique, and the spider will be logging\n  out continuously (for example, ``/logout?sid=UNIQUE_ID``).\n\n\nAPI\n---\n\nThere is no special API: autologin middleware just ensures that all requests are\nmade while being logged in. As mentioned in the \"Configuration\" section above,\nyou can override some settings on the per-request basis in ``reqeuest.meta``.\n\nAutologin response is available in ``response.meta['autologin_response']``,\nif we made requests to autologin while processing this request.\nYou might want to use the ``\"status\"`` field of the autologin response\nto do some bookkeeping.\n\nMiddleware also always puts ``\"autologin_active\"`` into ``response.meta``,\nwhich is ``True`` only if we are logged in (and ``False`` if domain is skipped\nor login failed).\n\nAdditionally, you set ``skip_autologin`` key in ``request.meta`` to ``True``\nin order to completely disable middleware for this request.\n\n\nUsage with Splash\n-----------------\n\nAutologin middleware supports splash via\n`scrapy-splash <https://github.com/scrapy-plugins/scrapy-splash>`_,\nbut correctly settings everything up can be tricky.\n\nFirst, you need to specify the following settings\n(check scrapy-splash docs for more details)::\n\n    SPLASH_URL = 'http://127.0.0.1:8050'\n    SPIDER_MIDDLEWARES = {\n        'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,\n    }\n    DUPEFILTER_CLASS = 'scrapy_splash.SplashAwareDupeFilter'\n    DOWNLOADER_MIDDLEWARES = {\n        'autologin_middleware.AutologinMiddleware': 605,\n        'scrapy_splash.SplashCookiesMiddleware': 723,\n        'scrapy_splash.SplashMiddleware': 725,\n        'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,\n    }\n\nSecond, you need to make requests to splash and pass cookies with\n``splash:init_cookies(splash.args.cookies)``, and return them in the\n``cookies`` field using ``splash:get_cookies()``. If you are already using\na splash script (``execute`` endpoint), modify your script accordingly.\nBut if you just want to crawl using splash, you can use\n``autologin_middleware.splash.splash_request`` instead of ``scrapy.Request``.\nIt has a minimal lua script that passes cookies and returns html, so you won't\nneed to change anything else in you spider.\n\n\nDevelopment\n-----------\n\nYou need to start ``autologin-http-api`` (from\n`autologin <https://github.com/TeamHG-Memex/autologin>`_),\nand `splash <http://splash.readthedocs.org>`_ (the easiest option is to run\n``docker run -p 8050:8050 scrapinghub/splash``).\n\nRun tests with tox::\n\n    tox\n\nWhen using Docker to run Splash on OS X and Windows, it will start on\na non-default address, so you need to specify it when running tests,\nfor example::\n\n    SPLASH_URL=http://192.168.99.100:8050 tox\n\n\nLicense\n-------\n\nLicense is MIT.",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/TeamHG-Memex/autologin-middleware",
        "keywords": null,
        "license": "MIT",
        "maintainer": null,
        "maintainer_email": null,
        "name": "autologin-middleware",
        "package_url": "https://pypi.org/project/autologin-middleware/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/autologin-middleware/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/TeamHG-Memex/autologin-middleware"
        },
        "release_url": "https://pypi.org/project/autologin-middleware/0.1.6/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "A Scrapy middleware to use with autologin",
        "version": "0.1.6"
    },
    "last_serial": 4308950,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "548b333af701863cd43db7ee253f71c6",
                    "sha256": "0a4f8cf597122a0a74a794a6129e8ba09b82c99a56fcc798ed55823b09f7b149"
                },
                "downloads": -1,
                "filename": "autologin_middleware-0.1.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "548b333af701863cd43db7ee253f71c6",
                "packagetype": "bdist_wheel",
                "python_version": "3.4",
                "requires_python": null,
                "size": 9299,
                "upload_time": "2016-05-20T14:50:28",
                "url": "https://files.pythonhosted.org/packages/80/7e/e9f46d1a7c4c22e69696f9b0c8dc63c579a99d71b066002766dfdc849ffe/autologin_middleware-0.1.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "87429e8a39fd25d5a7bce7a436b27f45",
                    "sha256": "ab59389ce526551226926f067534a84f6479af4957c6cd23d7666e81f93c1dce"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "87429e8a39fd25d5a7bce7a436b27f45",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6005,
                "upload_time": "2016-05-20T14:50:17",
                "url": "https://files.pythonhosted.org/packages/12/d0/efac16f705c72fffd812e50697f3f6511e0146782bd1d441f851b94c2b71/autologin-middleware-0.1.0.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "05a19aa0d81731809ed2e676336dcae7",
                    "sha256": "9267fc01f979e200e4c326dad047041015a8d63c73f9355cdb53b80851fd75d9"
                },
                "downloads": -1,
                "filename": "autologin_middleware-0.1.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "05a19aa0d81731809ed2e676336dcae7",
                "packagetype": "bdist_wheel",
                "python_version": "3.4",
                "requires_python": null,
                "size": 9312,
                "upload_time": "2016-05-20T19:42:43",
                "url": "https://files.pythonhosted.org/packages/dc/61/160386653373ab21b13e6a222ec3a7b5e6e6faafd84c8a9cc101f04e9195/autologin_middleware-0.1.1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f5ab553cb0478a99dc253e9d72f4d675",
                    "sha256": "385908050830c9e43b0ef27889b654d46649e72e73fc2e73153df766b0ec0501"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "f5ab553cb0478a99dc253e9d72f4d675",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6007,
                "upload_time": "2016-05-20T19:42:37",
                "url": "https://files.pythonhosted.org/packages/e8/a8/2cd9032026326e92a2e2bb4add0181a78923dd011d296d6a680c271b4b6a/autologin-middleware-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6f18309efa2a30dd9080e53792846bfe",
                    "sha256": "ddcfba906cbef4f97c78a1fabba81277a0b8dc4e85656c7e2cd2123086e58967"
                },
                "downloads": -1,
                "filename": "autologin_middleware-0.1.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6f18309efa2a30dd9080e53792846bfe",
                "packagetype": "bdist_wheel",
                "python_version": "3.4",
                "requires_python": null,
                "size": 9871,
                "upload_time": "2016-05-24T21:20:55",
                "url": "https://files.pythonhosted.org/packages/bd/61/2eb2df13239a61fa7189c65f4e4b6dc711b66a064fedfbd38634ce3144f9/autologin_middleware-0.1.2-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "652840f8ca280b81f4640f95855f1122",
                    "sha256": "58b935c0a7368a576ab5ee4f9c78520b142220db944ef3e93e37fe202bdcf951"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "652840f8ca280b81f4640f95855f1122",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6467,
                "upload_time": "2016-05-24T21:20:47",
                "url": "https://files.pythonhosted.org/packages/04/3a/81bdef8327cb286c1240113b0af06b56c39782960af2eb295a69e08d57a0/autologin-middleware-0.1.2.tar.gz"
            }
        ],
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f3f590e453fe1506a8a91313d2ac99c8",
                    "sha256": "daa8da97d76cad888f687fd6d7bab62da147b18158fb7dea8cb106b5292cc338"
                },
                "downloads": -1,
                "filename": "autologin_middleware-0.1.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f3f590e453fe1506a8a91313d2ac99c8",
                "packagetype": "bdist_wheel",
                "python_version": "3.4",
                "requires_python": null,
                "size": 11997,
                "upload_time": "2016-06-06T15:13:44",
                "url": "https://files.pythonhosted.org/packages/83/c2/383e2b012f7f3ab7083adc03c0b2dfea5669edafeeb56acc3707ef11ceec/autologin_middleware-0.1.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "cd47b18f62578e0c46c157f83eb932c7",
                    "sha256": "3a07516cd7ffcaabc6191fcb4bcc7b50cb46fac0a280b80569149b48cf7d6b88"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "cd47b18f62578e0c46c157f83eb932c7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7757,
                "upload_time": "2016-06-06T15:13:40",
                "url": "https://files.pythonhosted.org/packages/d2/b9/0a439fc9be7cf164c284146f3dcf46670d0425b5efeb0c5b59596711e07c/autologin-middleware-0.1.3.tar.gz"
            }
        ],
        "0.1.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "91b108e60df723e673d5a7a76f8ba584",
                    "sha256": "03af79e710830b6e7ddc9e55860e1e1d673eb3d431a0710e5b2171528f0ea263"
                },
                "downloads": -1,
                "filename": "autologin_middleware-0.1.4-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "91b108e60df723e673d5a7a76f8ba584",
                "packagetype": "bdist_wheel",
                "python_version": "3.4",
                "requires_python": null,
                "size": 12468,
                "upload_time": "2016-08-31T10:30:08",
                "url": "https://files.pythonhosted.org/packages/59/f6/2e9230e28a15ee39d84cd3239d9584739d19ab60cc3cc582cb5dac330dc2/autologin_middleware-0.1.4-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "76fc90f9d4702135cd9e9e51541e6d26",
                    "sha256": "ae1b7a2243e8ffd2cad8e0a74b473f50ceec4537228c914ac4d266c415b58fe0"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.4.tar.gz",
                "has_sig": false,
                "md5_digest": "76fc90f9d4702135cd9e9e51541e6d26",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8114,
                "upload_time": "2016-08-31T10:30:05",
                "url": "https://files.pythonhosted.org/packages/8f/f4/750ee5d677edd4c03ddad35d70e4a17eb94643d834ae0f62c17058320c24/autologin-middleware-0.1.4.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fd6e0b82331cd4f6ab3298814ebf22a9",
                    "sha256": "1e2a334dd90b613c2f0384d8b5c64ff219a8cd238dbf0957da76579995880214"
                },
                "downloads": -1,
                "filename": "autologin_middleware-0.1.5-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "fd6e0b82331cd4f6ab3298814ebf22a9",
                "packagetype": "bdist_wheel",
                "python_version": "3.4",
                "requires_python": null,
                "size": 12494,
                "upload_time": "2016-09-06T11:32:01",
                "url": "https://files.pythonhosted.org/packages/be/48/96b50c59b59292a07582ff5eb44118a2d78a7bd051cb53b889f5332bb448/autologin_middleware-0.1.5-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "21c741dcfd498da480d2a0d1f9386875",
                    "sha256": "9d85bb55a588be22252eafe6798285621ba21145f620bef4a2194798c35044e8"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "21c741dcfd498da480d2a0d1f9386875",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8142,
                "upload_time": "2016-09-06T11:31:58",
                "url": "https://files.pythonhosted.org/packages/9d/eb/d787ca92dd6c6970c15770df3bcbb37bc62c2a5d046d543c9b26a5a37e61/autologin-middleware-0.1.5.tar.gz"
            }
        ],
        "0.1.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f59fc5b320346e8e18fc20ee794c87ec",
                    "sha256": "c54ca6ea07fc1ed25ffd73010c52f19fece0c6823e0a55ae5fd4e9f76ff6cb09"
                },
                "downloads": -1,
                "filename": "autologin-middleware-0.1.6.tar.gz",
                "has_sig": false,
                "md5_digest": "f59fc5b320346e8e18fc20ee794c87ec",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8382,
                "upload_time": "2017-04-05T12:46:13",
                "url": "https://files.pythonhosted.org/packages/55/02/8c35b5b79debdd42fb0e711d59392bab75a80fc71c595aaca2a4969ee2e6/autologin-middleware-0.1.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "f59fc5b320346e8e18fc20ee794c87ec",
                "sha256": "c54ca6ea07fc1ed25ffd73010c52f19fece0c6823e0a55ae5fd4e9f76ff6cb09"
            },
            "downloads": -1,
            "filename": "autologin-middleware-0.1.6.tar.gz",
            "has_sig": false,
            "md5_digest": "f59fc5b320346e8e18fc20ee794c87ec",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8382,
            "upload_time": "2017-04-05T12:46:13",
            "url": "https://files.pythonhosted.org/packages/55/02/8c35b5b79debdd42fb0e711d59392bab75a80fc71c595aaca2a4969ee2e6/autologin-middleware-0.1.6.tar.gz"
        }
    ]
}