{
    "info": {
        "author": "Open Knowledge Foundation",
        "author_email": "info@okfn.org",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Environment :: Web Environment",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
            "Topic :: Software Development :: Libraries :: Python Modules"
        ],
        "description": "# tableschema-elasticsearch-py\n\n[![Travis](https://img.shields.io/travis/frictionlessdata/tableschema-elasticsearch-py/master.svg)](https://travis-ci.org/frictionlessdata/tableschema-elasticsearch-py)\n[![Coveralls](http://img.shields.io/coveralls/frictionlessdata/tableschema-elasticsearch-py/master.svg)](https://coveralls.io/r/frictionlessdata/tableschema-elasticsearch-py?branch=master)\n[![PyPi](https://img.shields.io/pypi/v/tableschema-elasticsearch.svg)](https://pypi.python.org/pypi/tableschema-elasticsearch)\n[![Github](https://img.shields.io/badge/github-master-brightgreen)](https://github.com/frictionlessdata/tableschema-elasticsearch-py)\n[![Gitter](https://img.shields.io/gitter/room/frictionlessdata/chat.svg)](https://gitter.im/frictionlessdata/chat)\n\nGenerate and load ElasticSearch indexes based on [Table Schema](http://specs.frictionlessdata.io/table-schema/) descriptors.\n\n## Features\n\n- implements `tableschema.Storage` interface\n\n## Contents\n\n<!--TOC-->\n\n  - [Getting Started](#getting-started)\n    - [Installation](#installation)\n    - [Examples](#examples)\n  - [Documentation](#documentation)\n    - [Storage](#storage)\n    - [Mappings](#mappings)\n  - [Contributing](#contributing)\n  - [Changelog](#changelog)\n\n<!--TOC-->\n\n## Getting Started\n\n### Installation\n\nThe package use semantic versioning. It means that major versions  could include breaking changes. It's highly recommended to specify `package` version range in your `setup/requirements` file e.g. `package>=1.0,<2.0`.\n\n```bash\npip install tableschema-elasticsearch\n```\n\n### Examples\n\nCode examples in this readme requires Python 3.3+ interpreter. You could see even more example in [examples](https://github.com/frictionlessdata/tableschema-spss-py/tree/master/examples) directory.\n\n```python\nimport elasticsearch\nimport jsontableschema_es\n\nINDEX_NAME = 'testing_index'\n\n# Connect to Elasticsearch instance running on localhost\nes=elasticsearch.Elasticsearch()\nstorage=jsontableschema_es.Storage(es)\n\n# List all indexes\nprint(list(storage.buckets))\n\n# Create a new index\nstorage.create('test', [\n    ('numbers',\n     {\n         'fields': [\n             {\n                 'name': 'num',\n                 'type': 'number'\n             }\n         ]\n     })\n])\n\n# Write data to index\nl=list(storage.write(INDEX_NAME, 'numbers', ({'num':i} for i in range(1000)), ['num']))\nprint(len(l))\nprint(l[:10], '...')\n\nl=list(storage.write(INDEX_NAME, 'numbers', ({'num':i} for i in range(500,1500)), ['num']))\nprint(len(l))\nprint(l[:10], '...')\n\n# Read all data from index\nstorage=jsontableschema_es.Storage(es)\nprint(list(storage.buckets))\nl=list(storage.read(INDEX_NAME))\nprint(len(l))\nprint(l[:10])\n\n```\n\n## Documentation\n\nThe whole public API of this package is described here and follows semantic versioning rules. Everyting outside of this readme are private API and could be changed without any notification on any new version.\n\n### Storage\n\nPackage implements [Tabular Storage](https://github.com/frictionlessdata/tableschema-py#storage) interface (see full documentation on the link):\n\n![Storage](https://i.imgur.com/RQgrxqp.png)\n\nThis driver provides an additional API:\n\n#### `Storage(es=None)`\n\n- `es (object)` - `elasticsearch.Elastisearc` instance. If not provided new one will be created.\n\nIn this driver `elasticsearch` is used as the db wrapper. We can get storage this way:\n\n```python\nfrom elasticsearch import Elasticsearch\nfrom jsontableschema_sql import Storage\n\nengine = Elasticsearch()\nstorage = Storage(engine)\n```\n\nThen we could interact with storage ('buckets' are ElasticSearch indexes in this context):\n\n```python\nstorage.buckets # iterator over bucket names\nstorage.create('bucket', [(doc_type, descriptor)],\n               reindex=False,\n               always_recreate=False,\n               mapping_generator_cls=None)\n        # reindex will copy existing documents from an existing index with the same name (in case of a mapping conflict)\n        # always_recreate will always recreate an index, even if it already exists. default is to update mappings only.\n        # mapping_generator_cls allows customization of the generated mapping\nstorage.delete('bucket')\nstorage.describe('bucket') # return descriptor, not implemented yet\nstorage.iter('bucket', doc_type=optional) # yield rows\nstorage.read('bucket', doc_type=optional) # return rows\nstorage.write('bucket', doc_type, rows, primary_key,\n              as_generator=False)\n        # primary_key is a list of field names which will be used to generate document ids\n```\n\nWhen creating indexes, we always create an index with a semi-random name and a matching alias that points to it. This allows us to decide whether to re-index documents whenever we're re-creating an index, or to discard the existing records.\n\n### Mappings\n\nWhen creating indexes, the tableschema types are converted to ES types and a mapping is generated for the index.\n\nSome special properties in the schema provide extra information for generating the mapping:\n - `array` types need also to have the `es:itemType` property which specifies the inner data type of array items.\n - `object` types need also to have the `es:schema` property which provides a tableschema for the inner document contained in that object (or have `es:enabled=false` to disable indexing of that field).\n\nExample:\n```json\n{\n  \"fields\": [\n    {\n      \"name\": \"my-number\",\n      \"type\": \"number\"\n    },\n    {\n      \"name\": \"my-array-of-dates\",\n      \"type\": \"array\",\n      \"es:itemType\": \"date\"\n    },\n    {\n      \"name\": \"my-person-object\",\n      \"type\": \"object\",\n      \"es:schema\": {\n        \"fields\": [\n          {\"name\": \"name\", \"type\": \"string\"},\n          {\"name\": \"surname\", \"type\": \"string\"},\n          {\"name\": \"age\", \"type\": \"integer\"},\n          {\"name\": \"date-of-birth\", \"type\": \"date\", \"format\": \"%Y-%m-%d\"}\n        ]\n      }\n    },\n    {\n      \"name\": \"my-library\",\n      \"type\": \"array\",\n      \"es:itemType\": \"object\",\n      \"es:schema\": {\n        \"fields\": [\n          {\"name\": \"title\", \"type\": \"string\"},\n          {\"name\": \"isbn\", \"type\": \"string\"},\n          {\"name\": \"num-of-pages\", \"type\": \"integer\"}\n        ]\n      }\n    },\n    {\n      \"name\": \"my-user-provded-object\",\n      \"type\": \"object\",\n      \"es:enabled\": false\n    }\n  ]\n}\n```\n\n#### Custom mappings\n\nBy providing a custom mapping generator class (via `mapping_generator_cls`), inheriting from the MappingGenerator class you should be able\n\n## Contributing\n\nThe project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).\n\nRecommended way to get started is to create and activate a project virtual environment.\nTo install package and development dependencies into active environment:\n\n```\n$ make install\n```\n\nTo run tests with linting and coverage:\n\n```bash\n$ make test\n```\n\nFor linting `pylama` configured in `pylama.ini` is used. On this stage it's already\ninstalled into your environment and could be used separately with more fine-grained control\nas described in documentation - https://pylama.readthedocs.io/en/latest/.\n\nFor example to sort results by error type:\n\n```bash\n$ pylama --sort <path>\n```\n\nFor testing `tox` configured in `tox.ini` is used.\nIt's already installed into your environment and could be used separately with more fine-grained control as described in documentation - https://testrun.org/tox/latest/.\n\nFor example to check subset of tests against Python 2 environment with increased verbosity.\nAll positional arguments and options after `--` will be passed to `py.test`:\n\n```bash\ntox -e py27 -- -v tests/<path>\n```\n\nUnder the hood `tox` uses `pytest` configured in `pytest.ini`, `coverage`\nand `mock` packages. This packages are available only in tox envionments.\n\n## Changelog\n\nHere described only breaking and the most important changes. The full changelog and documentation for all released versions could be found in nicely formatted [commit history](https://github.com/frictionlessdata/tableschema-elasticsearch-py/commits/master).\n\n#### v1.0\n\n- Initial driver implementation\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/frictionlessdata/tableschema-elasticsearch-py",
        "keywords": "frictionless data",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "tableschema-elasticsearch",
        "package_url": "https://pypi.org/project/tableschema-elasticsearch/",
        "platform": "",
        "project_url": "https://pypi.org/project/tableschema-elasticsearch/",
        "project_urls": {
            "Homepage": "https://github.com/frictionlessdata/tableschema-elasticsearch-py"
        },
        "release_url": "https://pypi.org/project/tableschema-elasticsearch/1.0.0/",
        "requires_dist": [
            "six (>=1.9)",
            "elasticsearch (<6.0,>=5.0)",
            "mock ; extra == 'develop'",
            "pylama ; extra == 'develop'",
            "pytest ; extra == 'develop'",
            "pytest-cov ; extra == 'develop'",
            "tabulator ; extra == 'develop'",
            "tox ; extra == 'develop'",
            "python-dotenv ; extra == 'develop'"
        ],
        "requires_python": "",
        "summary": "Generate ES Indexes, load and extract data, based on JSON Table Schema descriptors.",
        "version": "1.0.0"
    },
    "last_serial": 5934281,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "147532078f9bff78a821ab7fd14ef0d8",
                    "sha256": "b6ff1bad0207c36a6b69253d6cb72512701db0ec9d96566c855b9c04956d7eb7"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "147532078f9bff78a821ab7fd14ef0d8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8094,
                "upload_time": "2017-08-03T09:41:22",
                "url": "https://files.pythonhosted.org/packages/0d/5d/06f3aecc11d29fbd2918a7b2635c65804679bad1895a522b1d8bc3f8b869/tableschema-elasticsearch-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "24414374acdea6602114e0ae4d490524",
                    "sha256": "da94f9696255bd7e8e3c75e5bd17da159854a8227b07b7d1819da2182bb3cd7b"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "24414374acdea6602114e0ae4d490524",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8302,
                "upload_time": "2017-08-03T20:11:29",
                "url": "https://files.pythonhosted.org/packages/f9/26/17ac26bd605d7f140c19c1bf7b86c60b610d99542c077f9f2e1e647690c9/tableschema-elasticsearch-0.0.2.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "69e617ee1c1bdb05fa9f72db866b40ea",
                    "sha256": "535fd42e3fb8404d58553f2ffc5763a055537c538f1c14fb90b62a2ad7e58c5a"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "69e617ee1c1bdb05fa9f72db866b40ea",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8303,
                "upload_time": "2017-09-10T15:57:19",
                "url": "https://files.pythonhosted.org/packages/c0/4f/099d50059e8912961e12350d49368cdb2333da11fc7cfdaa76097f20e198/tableschema-elasticsearch-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "34ec2a5d6700623fda0ec28080f6a3f9",
                    "sha256": "095c59b6d5a82910e6f48ea54a2f9666392d57f40414dea9f28bfdee15c47383"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "34ec2a5d6700623fda0ec28080f6a3f9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9864,
                "upload_time": "2017-11-24T08:36:50",
                "url": "https://files.pythonhosted.org/packages/fe/c8/2865525967342fecf60f58d74e9328b3e58cb9114f0f36278a7523181669/tableschema-elasticsearch-0.1.2.tar.gz"
            }
        ],
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ff5f4b96f53ae4a477cee1c310aaec75",
                    "sha256": "43eb9d1a3808b3456076cf61daf1b23f4261f36dbd5db0224991635c30379d1a"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "ff5f4b96f53ae4a477cee1c310aaec75",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9928,
                "upload_time": "2017-12-26T14:25:54",
                "url": "https://files.pythonhosted.org/packages/16/65/2b60c6e9c80c22140ab4cd57de729c729a49c6c7fe11af0561b3342f9ce3/tableschema-elasticsearch-0.1.3.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c4b96912873cb5bcee4e956134ebfecd",
                    "sha256": "66b508caa6e9e0ab87d44f3433de093b798078071de4f45bd75387d6bfbff865"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "c4b96912873cb5bcee4e956134ebfecd",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10241,
                "upload_time": "2018-04-02T12:05:40",
                "url": "https://files.pythonhosted.org/packages/99/e6/b2129e3289bda2ba468254a6276d8da6a5498bce5bb90a395ca68af797e1/tableschema-elasticsearch-0.2.0.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d35257800b9552a344fa0e75ade7dfbe",
                    "sha256": "a5b998916a55912f9e40d4b75dd1403bedeb9286fb1ae0a645bc32a54d481594"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d35257800b9552a344fa0e75ade7dfbe",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10275,
                "upload_time": "2018-07-22T09:14:24",
                "url": "https://files.pythonhosted.org/packages/00/e6/66d043a9ed23c8cfd7cc770263f64e6632d85c1a407639db3f99fa996492/tableschema-elasticsearch-0.3.0.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fbf6ea008a43135726796e0fc569cb5e",
                    "sha256": "fb3bb48e1c233a6d625ceee4be9d294f4705e58af717dcc4e409492a180d8c5b"
                },
                "downloads": -1,
                "filename": "tableschema_elasticsearch-0.4.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "fbf6ea008a43135726796e0fc569cb5e",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 9609,
                "upload_time": "2018-11-27T07:10:05",
                "url": "https://files.pythonhosted.org/packages/fe/6a/b94a58a18628cf02efaa14a2e25c1d3ad76681984142e34017c73a09d64a/tableschema_elasticsearch-0.4.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0d10f2122908eb5b1be06720cfc4e4b7",
                    "sha256": "20ccf91724c08cf53def015c0fbb037243a9f5d701b891db51ef2212f2ef2073"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "0d10f2122908eb5b1be06720cfc4e4b7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11074,
                "upload_time": "2018-11-27T07:10:06",
                "url": "https://files.pythonhosted.org/packages/60/a0/6d025a9f8642287ec076e01b7db38c579e9b416c3cdbc65e5e30de83edc4/tableschema-elasticsearch-0.4.0.tar.gz"
            }
        ],
        "0.5.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dfea04c9997628e10ad55218854555d4",
                    "sha256": "c109130c7025020b9c199fc4ef0e3d46b79860f1cb7f52ed26648b6e7ad0da5d"
                },
                "downloads": -1,
                "filename": "tableschema_elasticsearch-0.5.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "dfea04c9997628e10ad55218854555d4",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 9584,
                "upload_time": "2019-08-15T12:06:07",
                "url": "https://files.pythonhosted.org/packages/33/07/a1017a38060ad22f67ba8ae6e98b9dd3de89850d2543a76e10a789558149/tableschema_elasticsearch-0.5.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "891304f95eb0b5ce37fb144411dae795",
                    "sha256": "b3701ca3ea63c1c05a0954c48900f0db0dac327c379978c3c08890faf319c6e8"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-0.5.3.tar.gz",
                "has_sig": false,
                "md5_digest": "891304f95eb0b5ce37fb144411dae795",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9782,
                "upload_time": "2019-08-15T12:06:09",
                "url": "https://files.pythonhosted.org/packages/2f/4c/7858a76037309db83088b104a2ea48bb0a82b168c1cff80e62cb956a1d0f/tableschema-elasticsearch-0.5.3.tar.gz"
            }
        ],
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "24fb330ca92250aba087024bdff63f4e",
                    "sha256": "d6de8f062eba36ff8ecffbdd29a3bb9925fa85301cf0582377547c67ca877bb3"
                },
                "downloads": -1,
                "filename": "tableschema_elasticsearch-1.0.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "24fb330ca92250aba087024bdff63f4e",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 9627,
                "upload_time": "2019-10-06T09:00:41",
                "url": "https://files.pythonhosted.org/packages/5b/dc/258050ae4b79a3ae969f26a556b2fd2142a31872a6571b4e97eff5f4d1d2/tableschema_elasticsearch-1.0.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0fb7a54a2b604ba0279dce1f45b65437",
                    "sha256": "90fa91cac033d5378583eefb919e65d630195a8d4f34ce8ee4b5031d54ca06d6"
                },
                "downloads": -1,
                "filename": "tableschema-elasticsearch-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "0fb7a54a2b604ba0279dce1f45b65437",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9813,
                "upload_time": "2019-10-06T09:00:43",
                "url": "https://files.pythonhosted.org/packages/22/dc/b3eb019286c7e80ed4e0bd9457bb6f1a0797da57019b946afa0f7b29b8cb/tableschema-elasticsearch-1.0.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "24fb330ca92250aba087024bdff63f4e",
                "sha256": "d6de8f062eba36ff8ecffbdd29a3bb9925fa85301cf0582377547c67ca877bb3"
            },
            "downloads": -1,
            "filename": "tableschema_elasticsearch-1.0.0-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "24fb330ca92250aba087024bdff63f4e",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 9627,
            "upload_time": "2019-10-06T09:00:41",
            "url": "https://files.pythonhosted.org/packages/5b/dc/258050ae4b79a3ae969f26a556b2fd2142a31872a6571b4e97eff5f4d1d2/tableschema_elasticsearch-1.0.0-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "0fb7a54a2b604ba0279dce1f45b65437",
                "sha256": "90fa91cac033d5378583eefb919e65d630195a8d4f34ce8ee4b5031d54ca06d6"
            },
            "downloads": -1,
            "filename": "tableschema-elasticsearch-1.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "0fb7a54a2b604ba0279dce1f45b65437",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 9813,
            "upload_time": "2019-10-06T09:00:43",
            "url": "https://files.pythonhosted.org/packages/22/dc/b3eb019286c7e80ed4e0bd9457bb6f1a0797da57019b946afa0f7b29b8cb/tableschema-elasticsearch-1.0.0.tar.gz"
        }
    ]
}