{
"info": {
"author": "Rounak Vyas",
"author_email": "itsron143@gmail.com",
"bugtrack_url": null,
"classifiers": [],
"description": "# es-indexer\n[](https://lbesson.mit-license.org/)\n\n\n\n\n**es-indexer** (Elasticsearch Indexer) is a simple concurrent command line tool written in Python to help you quickly populate some json data into Elasticsearch.
\n\n## About\n\nUsually you'll have to use a third-party software or a client library to index data to Elasticsearch and setting that up can be really time consuming and tiresome (*cough*_logstash_*cough*). **es-indexer** helps in indexing raw contents of `*.json` documents quickly with the help of multi-processing.
\n\n**es-indexer** currently doesn't provide any syncing of the data, you'll have to reindex the data if it changes, but will always populate a new index and then create an alias, the old data will be present while re-indexing until the new index is fully populated. A future update might include syncing.
\n\nSince Elasticsearch exposes a REST-API on Port 9200, there's no need for es-indexer providing a REST-API itself.\n\n## Installation\n\n> Requires Python 3.x and is compatible with Elasticsearch 7.x.x.\n\n* es-indexer can be installed with the help of pip.\n ```\n $ pip install es-indexer\n ```\n**(OR)**\n\n* Clone the repository.\n ```\n $ git clone https://github.com/itsron717/es-indexer.git\n ```\n\n* Move inside the repo.\n ```\n $ cd es-indexer\n ```\n\n* Install the package locally.\n ```\n $ pip install .\n ```\n\n## Usage\n### Config\nYou need to create a `config.yml` before running es-indexer:\n\n host: http://127.0.0.1:9200\n index: twitter-example\n type: documents\n mapping:\n settings:\n number_of_shards: 1\n number_of_replicas: 0\n\nYou can provide a custom mapping in the config file, es-indexer will convert the yaml mapping 1:1 to json.\n```\n$ es-indexer --config path/to/config/file --source path/to/json/folder\n```\n\n## Adding more Data Sources\nMore data sources other than `json` such as `SQL`, `Filesystem`, etc are also to be added to the es-indexer tool such that it can be a one stop shop for all the indexing needs of Elasticsearch. Anybody who'd like to contribute in integrating other data sources can raise and issue and we can start working on it!. \n\n## To-DO\n- [x] Add `json` support.\n- [ ] Add `SQL` data source integration.\n- [ ] Add `FileSystem` data source integration.\n- [ ] Increase the speed of indexing.\n- [ ] Add tests.\n\n## References\n\nes-indexer was built using the insipiration of [this](https://github.com/arkste/elsi) amazing tool written in Go.\n\n## License\n\nThe MIT License (MIT)\n\nCopyright (c) 2019 [Rounak Vyas](https://www.linkedin.com/in/itsron143/)\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/itsron717/es-indexer",
"keywords": "",
"license": "MIT",
"maintainer": "",
"maintainer_email": "",
"name": "es-indexer",
"package_url": "https://pypi.org/project/es-indexer/",
"platform": "",
"project_url": "https://pypi.org/project/es-indexer/",
"project_urls": {
"Homepage": "https://github.com/itsron717/es-indexer"
},
"release_url": "https://pypi.org/project/es-indexer/0.1/",
"requires_dist": [
"Click",
"elasticsearch"
],
"requires_python": "",
"summary": "A command line tool written in Python to help quickly populate json data into Elasticsearch.",
"version": "0.1"
},
"last_serial": 5390229,
"releases": {
"0.1": [
{
"comment_text": "",
"digests": {
"md5": "cbc5f903ad1efe73b79c69d64194a575",
"sha256": "db2aa1411c83d2f097d41a56c442113e197871297455261dd96bcf7bc617b802"
},
"downloads": -1,
"filename": "es_indexer-0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cbc5f903ad1efe73b79c69d64194a575",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 3731,
"upload_time": "2019-06-12T09:22:27",
"url": "https://files.pythonhosted.org/packages/cd/17/f432ec349668e7e1f0b31feb53c9cff392dbd3593bc95be2c95a1adfe622/es_indexer-0.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "9d95cc023a98ca1e894606604a04054b",
"sha256": "ac22045fac159d71a408d790d289304dc92f3f0eabd637ad6619acd6b91b30f2"
},
"downloads": -1,
"filename": "es-indexer-0.1.tar.gz",
"has_sig": false,
"md5_digest": "9d95cc023a98ca1e894606604a04054b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3308,
"upload_time": "2019-06-12T09:22:29",
"url": "https://files.pythonhosted.org/packages/6c/08/7c66368593f6e0d02efd23879701c4fd5f749dd820731f556b653a01337c/es-indexer-0.1.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "cbc5f903ad1efe73b79c69d64194a575",
"sha256": "db2aa1411c83d2f097d41a56c442113e197871297455261dd96bcf7bc617b802"
},
"downloads": -1,
"filename": "es_indexer-0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cbc5f903ad1efe73b79c69d64194a575",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 3731,
"upload_time": "2019-06-12T09:22:27",
"url": "https://files.pythonhosted.org/packages/cd/17/f432ec349668e7e1f0b31feb53c9cff392dbd3593bc95be2c95a1adfe622/es_indexer-0.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "9d95cc023a98ca1e894606604a04054b",
"sha256": "ac22045fac159d71a408d790d289304dc92f3f0eabd637ad6619acd6b91b30f2"
},
"downloads": -1,
"filename": "es-indexer-0.1.tar.gz",
"has_sig": false,
"md5_digest": "9d95cc023a98ca1e894606604a04054b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3308,
"upload_time": "2019-06-12T09:22:29",
"url": "https://files.pythonhosted.org/packages/6c/08/7c66368593f6e0d02efd23879701c4fd5f749dd820731f556b653a01337c/es-indexer-0.1.tar.gz"
}
]
}