{
    "info": {
        "author": "Jay Zeng, Michael Malocha, Julien Duponchelle",
        "author_email": "michael@knockrentals.com, julien@duponchelle.info",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Environment :: No Input/Output (Daemon)",
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: OS Independent",
            "Programming Language :: Python"
        ],
        "description": "Description\n===========\nScrapy pipeline which allows you to store scrapy items in Elastic Search.\n\nInstall\n=======\n::\n\n   pip install ScrapyElasticSearch\n\n   If you need support for ntlm:\n   pip install \"ScrapyElasticSearch[extras]\"\n\nUsage (Configure settings.py:)\n----------------------\n::\n\n   ITEM_PIPELINES = {\n       'scrapyelasticsearch.scrapyelasticsearch.ElasticSearchPipeline': 500\n   }\n\n   ELASTICSEARCH_SERVERS = ['localhost']\n   ELASTICSEARCH_INDEX = 'scrapy'\n   ELASTICSEARCH_INDEX_DATE_FORMAT = '%Y-%m'\n   ELASTICSEARCH_TYPE = 'items'\n   ELASTICSEARCH_UNIQ_KEY = 'url'  # Custom uniqe key\n\n   # can also accept a list of fields if need a composite key\n   ELASTICSEARCH_UNIQ_KEY = ['url', 'id']\n\nELASTICSEARCH_SERVERS - list of hosts or string (single host). Host format: protocl://username:password@host:port.\nExamples:\n    - ['http://username:password@elasticsearch.example.com:9200']\n    - ['http://elasticsearch.example.com:9200']\n    - 'https://elasticsearch.example.com:9200'\n\nAvailable parameters (in settings.py:)\n----------------------\n::\n\n   ELASTICSEARCH_INDEX - elastic search index\n   ELASTICSEARCH_INDEX_DATE_FORMAT - the format for date suffix for the index, see python datetime.strftime for format. Default is no date suffix.\n   ELASTICSEARCH_TYPE - elastic search type\n   ELASTICSEARCH_UNIQ_KEY - optional field, unique key in string (must be a field or a list declared in model, see items.py)\n   ELASTICSEARCH_BUFFER_LENGTH - optional field, number of items to be processed during each bulk insertion to Elasticsearch. Default size is 500.\n   ELASTICSEARCH_AUTH  - optional field, set to 'NTLM' to use NTLM authentification\n   ELASTICSEARCH_USERNAME - optional field, set to 'DOMAIN\\username', only used with NLTM authentification\n   ELASTICSEARCH_PASSWORD - optional field, set to your 'password', only used with NLTM authentification\n\n   ELASTICSEARCH_CA - optional settings to if es servers require custom CA files.\n   Example:\n   ELASTICSEARCH_CA = {\n        'CA_CERT': '/path/to/cacert.pem',\n        'CLIENT_CERT': '/path/to/client_cert.pem',\n        'CLIENT_KEY': '/path/to/client_key.pem'\n  }\n\n\nHere is an example app (dirbot https://github.com/jayzeng/dirbot) in case you are still confused.\n\nDependencies\n=========\nSee requirements.txt\n\nChangelog\n=========\n* 0.9: Accept custom CA cert to connect to es clusters\n* 0.8: Added support for NTLM authentification\n* 0.7.1: Added date format to the index name and a small bug fix\n    - ELASTICSEARCH_BUFFER_LENGTH default was 9999, this has been changed to reflect documentation.\n\n* 0.7: A number of backwards incompatibility changes are introduced:\n    - Changed ELASTICSEARCH_SERVER to ELASTICSEARCH_SERVERS\n    - ELASTICSEARCH_SERVERS accepts string or list\n    - ELASTICSEARCH_PORT is removed, you can specify it in the url\n    - ELASTICSEARCH_USERNAME and ELASTICSEARCH_PASSWORD are removed. You can use this format ELASTICSEARCH_SERVERS=['http://username:password@host:port']\n    - Changed scrapy.log to logging as scrapy now uses the logging module\n\n* 0.6.1: Able to pull configs from spiders (in addition to reading from config file)\n* 0.6: Bug fix\n* 0.5: Abilit to persist object; Option to specify logging level\n* 0.4: Remove debug\n* 0.3: Auth support\n* 0.2: Scrapy 0.18 support\n* 0.1: Initial release\n\nIssues\n=============\nIf you find any bugs or have any questions, please report them to \"issues\" (https://github.com/knockrentals/scrapy-elasticsearch/issues)\n\nContributors\n=============\n* Jay Zeng (Maintainer) (https://github.com/jayzeng)\n* Michael Malocha (https://github.com/mjm159)\n* Ignacio Vazquez (https://github.com/ignaciovazquez)\n* Julien Duponchelle (https://github.com/noplay)\n* Jay Stewart (https://github.com/solidground)\n* Alessio Cimarelli (https://github.com/jenkin)\n* Doug Parker (https://github.com/dougiep16)\n* Jean-Sebastien Gervais (https://github.com/jsgervais)\n\n\nLicence\n=======\nCopyright 2014 Michael Malocha\n\nExpanded on the work by Julien Duponchelle\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "https://github.com/knockrentals/scrapy-elasticsearch/tarball/master",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/knockrentals/scrapy-elasticsearch.git",
        "keywords": "scrapy elastic search",
        "license": "Apache License, Version 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "ScrapyElasticSearch",
        "package_url": "https://pypi.org/project/ScrapyElasticSearch/",
        "platform": "Any",
        "project_url": "https://pypi.org/project/ScrapyElasticSearch/",
        "project_urls": {
            "Download": "https://github.com/knockrentals/scrapy-elasticsearch/tarball/master",
            "Homepage": "https://github.com/knockrentals/scrapy-elasticsearch.git"
        },
        "release_url": "https://pypi.org/project/ScrapyElasticSearch/0.9.1/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Scrapy pipeline which allow you to store multiple scrapy items in Elastic Search.",
        "version": "0.9.1"
    },
    "last_serial": 3264651,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "622d60493fa63172374ba59ac68fa1b0",
                    "sha256": "04f92014b5709ce2e9eb4ee9b3ca1fbe5c630c5209d26eb5bcd9a9edc072afd9"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "622d60493fa63172374ba59ac68fa1b0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1495,
                "upload_time": "2011-02-12T20:58:33",
                "url": "https://files.pythonhosted.org/packages/b1/b6/97b0e302aacccdd0e7a43fb0b5cb0c167427f6efd624fe3c2192c523c345/ScrapyElasticSearch-0.1.tar.gz"
            }
        ],
        "0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0022cec0daeb0b28c352807adad54bfe",
                    "sha256": "0f5b40f4478bca728cea295646d18d829c609d1559542d4cc528099d2121175f"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "0022cec0daeb0b28c352807adad54bfe",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1531,
                "upload_time": "2013-10-09T19:21:08",
                "url": "https://files.pythonhosted.org/packages/60/a7/0483522aa912b59646bc5abe3f06fae3bacf24da808140fc773d08a3c63e/ScrapyElasticSearch-0.2.tar.gz"
            }
        ],
        "0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "35b56a26d0584416b1966a9c99f5a1ac",
                    "sha256": "8a17588110afb2994e20c50230791d831fc595709c7d754604dea5ed57bea888"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "35b56a26d0584416b1966a9c99f5a1ac",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1644,
                "upload_time": "2013-10-12T17:42:48",
                "url": "https://files.pythonhosted.org/packages/a9/d8/b14043f763596947da6670d75b85456a364ac0dc28e6173ea022203ca516/ScrapyElasticSearch-0.3.tar.gz"
            }
        ],
        "0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e26440ea2d79e625a15bda64d343fddd",
                    "sha256": "b05899c0ea18752bf0060122a21c2da06220326f340d93b1c9f3141e3a380edd"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.4.tar.gz",
                "has_sig": false,
                "md5_digest": "e26440ea2d79e625a15bda64d343fddd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1634,
                "upload_time": "2013-12-01T21:30:21",
                "url": "https://files.pythonhosted.org/packages/d1/d8/cf293863a37fe54122e45d4a02e15bfcc6c6b14e4d65b0322993e5556e1d/ScrapyElasticSearch-0.4.tar.gz"
            }
        ],
        "0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cd7200df896c577b28dc1c4ad990512f",
                    "sha256": "140c24ad67114f42b2038b95892662152900d266517722d96b1701961ab5bbb0"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.5.tar.gz",
                "has_sig": false,
                "md5_digest": "cd7200df896c577b28dc1c4ad990512f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2184,
                "upload_time": "2014-10-18T06:08:49",
                "url": "https://files.pythonhosted.org/packages/bc/7a/5ed00b3e1f3523593d59313af54aacdaa917d559b1e4d01227a21c1e7eaa/ScrapyElasticSearch-0.5.tar.gz"
            }
        ],
        "0.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ca5d76df922c3ceffb7ab711f7d89013",
                    "sha256": "5157f7cca4d3ab819e3f75ac4f89a7d22a452daad283ef477f74ea4d9bce2b0a"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.6.tar.gz",
                "has_sig": false,
                "md5_digest": "ca5d76df922c3ceffb7ab711f7d89013",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2205,
                "upload_time": "2014-12-03T01:50:35",
                "url": "https://files.pythonhosted.org/packages/95/3c/6c52377c7fbfd0852a72ed72156a4b415a4970fba25ec9c26d97f78bdf25/ScrapyElasticSearch-0.6.tar.gz"
            }
        ],
        "0.6.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "331581111fe99f2456ecc1dc0e8e5ae1",
                    "sha256": "ee0c6d2fa862bb24072ea9d5be5ca18f2745f3150d5422b818603072001e0d66"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.6.1.tar.gz",
                "has_sig": false,
                "md5_digest": "331581111fe99f2456ecc1dc0e8e5ae1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2253,
                "upload_time": "2014-12-07T07:27:36",
                "url": "https://files.pythonhosted.org/packages/4b/1d/e42fe85dd2ee9c103502866e8c25444de7a184103a37ce9017d2e277ae37/ScrapyElasticSearch-0.6.1.tar.gz"
            }
        ],
        "0.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ca0410dd040c8b89a9e071482309c026",
                    "sha256": "11b26dbe7938b3280be8b57ca968bc067147534e675422d30fbadade87963c2b"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.7.tar.gz",
                "has_sig": false,
                "md5_digest": "ca0410dd040c8b89a9e071482309c026",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3940,
                "upload_time": "2016-04-30T06:40:41",
                "url": "https://files.pythonhosted.org/packages/4b/9d/5e3f1e5e4c5d19faf2f6be4cdc05d25763014d6627ea734a1f678da5e388/ScrapyElasticSearch-0.7.tar.gz"
            }
        ],
        "0.7.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6a73937e9f4f1d0edf4bccb7b38404c4",
                    "sha256": "ccfa8c4a4dcbceee2f00ef7c477cdde1ab52430829d11fcf8a6d7c5cf1a3efda"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.7.1.tar.gz",
                "has_sig": false,
                "md5_digest": "6a73937e9f4f1d0edf4bccb7b38404c4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4191,
                "upload_time": "2016-06-15T00:46:11",
                "url": "https://files.pythonhosted.org/packages/2f/0b/3e54d779a3d53b07be3428ae8f1844ff45351b8a0b3745bfc61af9a75275/ScrapyElasticSearch-0.7.1.tar.gz"
            }
        ],
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9d927e3e91cb370e430da3257f88e83f",
                    "sha256": "874b6894552620ffa2a465fc91f0a3d40e86d0c418cf42870ad11d8796b8aba8"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9d927e3e91cb370e430da3257f88e83f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9613,
                "upload_time": "2016-06-15T00:48:20",
                "url": "https://files.pythonhosted.org/packages/a6/36/38428128a3b04463cbc47eab105cdc133e5dc48b2a40327c22f2f45d040e/ScrapyElasticSearch-0.8.0.tar.gz"
            }
        ],
        "0.8.1": [],
        "0.8.2": [],
        "0.8.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7f6a1732d1f4e367d1dfe5e91bbb3ebb",
                    "sha256": "4ace3ced44b3913539a4d764de04d6a1603e12b9e86841dd38b4c834d7bf0f99"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.3.tar.gz",
                "has_sig": false,
                "md5_digest": "7f6a1732d1f4e367d1dfe5e91bbb3ebb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9639,
                "upload_time": "2016-09-01T04:52:42",
                "url": "https://files.pythonhosted.org/packages/d5/25/5ab42e511da1c2bfa19cad252d0d22b0f58c3d2fd89f6e3f578ceb50ba96/ScrapyElasticSearch-0.8.3.tar.gz"
            }
        ],
        "0.8.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "52edd4d0e79f1c83c1f640725deb5ce8",
                    "sha256": "c78e7c5c51b33e593f86fec9ad7161e830975787ea45227e1fe1024d900dc42b"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.4.tar.gz",
                "has_sig": false,
                "md5_digest": "52edd4d0e79f1c83c1f640725deb5ce8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9637,
                "upload_time": "2016-09-06T21:11:45",
                "url": "https://files.pythonhosted.org/packages/77/7e/d5a257e29fed4b8d36bdab1c4a51ac4273d12d5faba69292d735e87385e6/ScrapyElasticSearch-0.8.4.tar.gz"
            }
        ],
        "0.8.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6dbe45e1db414e48dd77d30c64328e4f",
                    "sha256": "897038a90fcec60bc7da73c1bd8b5e3987b1555234e27a6cb1539b6f056122b5"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.5.tar.gz",
                "has_sig": false,
                "md5_digest": "6dbe45e1db414e48dd77d30c64328e4f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9641,
                "upload_time": "2016-09-11T19:18:57",
                "url": "https://files.pythonhosted.org/packages/19/5e/ed74d85d07f670fc75875406be42e63fe5a0abd6d5c78c746883253f5633/ScrapyElasticSearch-0.8.5.tar.gz"
            }
        ],
        "0.8.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "65be1a6061b68bcc3552ea3e936d69f3",
                    "sha256": "1e598faf2130188aaec97f0949485be6b43daf42c9bac5e2b224b4b797240989"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.6.tar.gz",
                "has_sig": false,
                "md5_digest": "65be1a6061b68bcc3552ea3e936d69f3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9655,
                "upload_time": "2016-10-19T22:30:33",
                "url": "https://files.pythonhosted.org/packages/df/bd/3493ff0f2dc0e55ded06f8e4adaaf15744489999622873725bee49ca71bc/ScrapyElasticSearch-0.8.6.tar.gz"
            }
        ],
        "0.8.7": [],
        "0.8.8": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ffb415cc4ba7601e35f89406502cb446",
                    "sha256": "f080e3be08797f0a5f125d9d508fb5330397706201255344683a2beeb1f258d0"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.8.tar.gz",
                "has_sig": false,
                "md5_digest": "ffb415cc4ba7601e35f89406502cb446",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9772,
                "upload_time": "2016-11-05T21:54:49",
                "url": "https://files.pythonhosted.org/packages/44/10/40a4410419905155fc8e3d43040f9ec60f677cffad9eb964d0b1078db119/ScrapyElasticSearch-0.8.8.tar.gz"
            }
        ],
        "0.8.9": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "016da30cf8bbe3c840c61c321c64c7f2",
                    "sha256": "0c1bde4559a7d71ec7b4f3b293aa62b7fb3fc9281391c053b1818a70f6304b37"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.8.9.tar.gz",
                "has_sig": false,
                "md5_digest": "016da30cf8bbe3c840c61c321c64c7f2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9772,
                "upload_time": "2016-11-14T02:36:54",
                "url": "https://files.pythonhosted.org/packages/c9/8f/e9c08bd24e5ea2540d74601d5fafdba9b6dfd55404fa1ec91be84eb5c661/ScrapyElasticSearch-0.8.9.tar.gz"
            }
        ],
        "0.9.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "26bb716441fa8f50830d3a9e4ea1f309",
                    "sha256": "270ebef86a9907e8d2e2eda5b442b935fe12a1b36ce786c6e25eb3cf0db97919"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.9.0.tar.gz",
                "has_sig": false,
                "md5_digest": "26bb716441fa8f50830d3a9e4ea1f309",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10322,
                "upload_time": "2017-08-28T17:42:01",
                "url": "https://files.pythonhosted.org/packages/23/23/e0a3a6b48ccb094e26c84ad648f651f551985976617b15bf2dc4c43f929f/ScrapyElasticSearch-0.9.0.tar.gz"
            }
        ],
        "0.9.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0cea88aa5271bc975dd04d3ceca1d8e6",
                    "sha256": "5b915359db40feff12d37f57ff6d95498c57c4b6b8e7e22a4683dfce2cbb4669"
                },
                "downloads": -1,
                "filename": "ScrapyElasticSearch-0.9.1.tar.gz",
                "has_sig": false,
                "md5_digest": "0cea88aa5271bc975dd04d3ceca1d8e6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10357,
                "upload_time": "2017-10-20T05:04:46",
                "url": "https://files.pythonhosted.org/packages/23/f0/9b72896c8b9b7b93c33f76fa9e586119b2ba2966bf612a5961e1f5aad273/ScrapyElasticSearch-0.9.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "0cea88aa5271bc975dd04d3ceca1d8e6",
                "sha256": "5b915359db40feff12d37f57ff6d95498c57c4b6b8e7e22a4683dfce2cbb4669"
            },
            "downloads": -1,
            "filename": "ScrapyElasticSearch-0.9.1.tar.gz",
            "has_sig": false,
            "md5_digest": "0cea88aa5271bc975dd04d3ceca1d8e6",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10357,
            "upload_time": "2017-10-20T05:04:46",
            "url": "https://files.pythonhosted.org/packages/23/f0/9b72896c8b9b7b93c33f76fa9e586119b2ba2966bf612a5961e1f5aad273/ScrapyElasticSearch-0.9.1.tar.gz"
        }
    ]
}