{
    "info": {
        "author": "la9ran9e",
        "author_email": "tvauritimur@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# Pyshard\n\n[![Build Status](https://travis-ci.org/la9ran9e/pyshard.svg?branch=master)](https://travis-ci.org/la9ran9e/pyshard)\n[![Code Coverage Status](https://codecov.io/gh/la9ran9e/pyshard/branch/master/graph/badge.svg)](https://codecov.io/gh/la9ran9e/pyshard)\n[![PyPI](https://img.shields.io/pypi/v/pyshard.svg)](https://pypi.org/project/pyshard/)\n\nPyshard is a complete distributed key-value data storage \nwritten in Python using only standard library tools.\nPyshard's using hash based sharding method. It means \nthat shard of value you write will be selected in accordance to\nkey hash (regards to [lgiordani/pyshard](https://github.com/lgiordani/pyshard)). \nThis project is experimental and should be used in another \nproject [pdx](https://github.com/la9ran9e/pdx) - distributed web indexing service.\n\n## Installation\n\n```bash\npip install pyshard\n```\n\n## Quick start\n### Bootstrap\nTo run 'hello world' service you need started up shard servers. For example:\n\n```python\n# test_server.py\n\nimport sys\nimport asyncio\nfrom pyshard import ShardServer\n\nif __name__ == '__main__':\n    loop = asyncio.get_event_loop()\n\n    server = ShardServer(host=sys.argv[1], port=int(sys.argv[2]), start=.0, end=1.0)\n    try:\n        loop.run_until_complete(server._do_run())\n    finally:\n        loop.close()\n```\n```bash\npython test_server.py localhost 5050 & \\\npython test_server.py localhost 5051\n```\n\nAfter servers started up you should start bootstrap server to map shards. \nNow bootstrap server needs config file with shard's markers:\n\n```json\n{\n  \"shards\": [\n    {\n      \"name\": \"shard0-0.5\",\n      \"start\": 0.0,\n      \"end\": 0.5,\n      \"size\": 1024,\n      \"host\": \"127.0.0.1\",\n      \"port\": 5050\n    },\n    {\n      \"name\": \"shard0.5-1\",\n      \"start\": 0.5,\n      \"end\": 1.0,\n      \"size\": 1024,\n      \"host\": \"127.0.0.1\",\n      \"port\": 5051\n    }\n  ]\n}\n```\nEvery shard has next parameters:\n`name` - unique string name of shard,\n`start` and `end` - numeric limits of key hash,\n`size` - memory limit for this shard,\n`host` and `port` - shard address.\n`start` and `end` limit means that this shard will store values with key hash in range `[start, end]`.\n\n```python\n# test_bootstrap_server.py\n\nimport asyncio\n\nfrom pyshard import BootstrapServer\n\nfrom pyshard.settings import settings\n\n\nif __name__ == '__main__':\n    loop = asyncio.get_event_loop()\n    server = BootstrapServer(*settings.BOOTSTRAP_SERVER, config_path='config_example.json',\n                             buffer_size=1024, loop=loop)\n    try:\n        loop.run_until_complete(server._do_run())\n    finally:\n        loop.close()\n\n```\n\n```bash\npython test_bootstrap_server.py\n```\n\nNow shards have got configurations from bootstrap service and ready.\n\n### App\n\n\n```python\n>>> from pyshard import Pyshard\n>>> from pyshard.settings import settings\n>>> \n>>> app = Pyshard(bootstrap_server=settings.BOOTSTRAP_SERVER)\n>>> app.create_index('test_index')\n>>> app.write(index='test_index', key='test', doc='hello world')\n60\n>>> app.read(index='test_index', key='test')\n{'hash_': 0.1671936, 'record': 'hello world'}\n>>> app.write('test_index', 'test1', {'hello': 'world'})\n54\n>>> app.read('test_index', 'test')\n{'hash_': 0.8204544, 'record': {'hello': 'world'}}\n>>> app.pop('test_index', 'test1')\n{'hash_': 0.8204544, 'record': {'hello': 'world'}}\n```\n\n### Utilities\n\nSince version 0.2.0 Pyshard has several console utilities. They are made to simplify some operations like `cat` or massive write.\n\n\nLet's make file with data. Row format: `{key}|{value}`:\n\n```bash\nprintf '1|test\\n2|{\"test\": \"test\"}\\n3|42\\n4|0.9\\n' > test_write.txt\n```\nWe can add this rows to storage using `pyshard write` command.\n```bash\n\ncat test_write.txt | pyshard write test_index --force\n\n```\n`--force` oprion for creating index `test_index` if it does not exist\n\n\nSo let's `cat` storage with index `test_index`:\n```bash\npyshard cat test_index\n```\nCommand will log results to stdout:\n```\n2|{\"hash_\": 0.2258304, \"record\": {\"test\": \"test\"}}\n3|{\"hash_\": 0.1904896, \"record\": 42}\n1|{\"hash_\": 0.8102784, \"record\": \"test\"}\n4|{\"hash_\": 0.7252864, \"record\": 0.9}\n```\n\n\n## TODO\n* Index (data tables equivalent)\n* Connection id for shard servers (now it is an address)\n* App utils (`pyshard read`, `pyshard write`)\n* Nice run methods for services\n* Makefile\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/la9ran9e/pyshard",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pyshard",
        "package_url": "https://pypi.org/project/pyshard/",
        "platform": "",
        "project_url": "https://pypi.org/project/pyshard/",
        "project_urls": {
            "Homepage": "https://github.com/la9ran9e/pyshard"
        },
        "release_url": "https://pypi.org/project/pyshard/0.2.3/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Distributed key-value storage in Python stdlib",
        "version": "0.2.3",
        "yanked": false,
        "yanked_reason": null
    },
    "last_serial": 6047786,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3804622016fa867edfbfb6e14098d5df",
                    "sha256": "6e62423869383fd16dde9d5954e3a659f2693a6f3c039d3e9dca4ce054d5966e"
                },
                "downloads": -1,
                "filename": "pyshard-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3804622016fa867edfbfb6e14098d5df",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 17429,
                "upload_time": "2019-07-18T09:54:00",
                "upload_time_iso_8601": "2019-07-18T09:54:00.746896Z",
                "url": "https://files.pythonhosted.org/packages/f7/63/87384651d2c2e795289cb6db4004f06049a2b0de7ca6050f46d842bef7b1/pyshard-0.0.1-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "4944b186a6d6ec5683527b7468b742fb",
                    "sha256": "90ad85eaa4d2e06c6ef91ce9935a5c42bcdb0ba1a2a460fd313d65f32ca15069"
                },
                "downloads": -1,
                "filename": "pyshard-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "4944b186a6d6ec5683527b7468b742fb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12198,
                "upload_time": "2019-07-18T09:54:02",
                "upload_time_iso_8601": "2019-07-18T09:54:02.638956Z",
                "url": "https://files.pythonhosted.org/packages/e0/e0/5a7b6c964f5a273c3a6dcd2bc7affb08ddac8f479510904a9d8a373c13b2/pyshard-0.0.1.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4544a604022e330b5aace6352a17d78e",
                    "sha256": "24ed767d2d8a4ecd58de7c976f0bf41d3522bfb815962075fe4475a6c650a14a"
                },
                "downloads": -1,
                "filename": "pyshard-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "4544a604022e330b5aace6352a17d78e",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 19122,
                "upload_time": "2019-07-18T22:54:44",
                "upload_time_iso_8601": "2019-07-18T22:54:44.509367Z",
                "url": "https://files.pythonhosted.org/packages/97/2e/cbb8c0392df218df64462b21e46e0cd7e38182eb234d9eac605d03089ac3/pyshard-0.1.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "5b6f5114b68a854e0be1f8a78f6cca26",
                    "sha256": "5b02dbd7f5751ee0761ef3e1655b5d2fadb417aca5d2ac80ecc95c4403e2c33f"
                },
                "downloads": -1,
                "filename": "pyshard-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "5b6f5114b68a854e0be1f8a78f6cca26",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 14796,
                "upload_time": "2019-07-18T22:54:46",
                "upload_time_iso_8601": "2019-07-18T22:54:46.313940Z",
                "url": "https://files.pythonhosted.org/packages/73/c2/bdf3e1133a10feaba6a8d28f85cb781d84593e7f174b62870376e277e6d6/pyshard-0.1.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bf0989cc009a28014bc4a11cf90ab5d8",
                    "sha256": "96290a6853830679ceb90e6661b4b444166990d9d7395478c53c674665f1a192"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "bf0989cc009a28014bc4a11cf90ab5d8",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 20995,
                "upload_time": "2019-07-20T01:57:15",
                "upload_time_iso_8601": "2019-07-20T01:57:15.994601Z",
                "url": "https://files.pythonhosted.org/packages/85/98/aa43a19f337927eb3fa68e7aa3c2c1d221a81f676d919333febe59fa2d07/pyshard-0.2.0-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1f171fb7ad1288cab83680a593ba9d37",
                    "sha256": "b4afe5502203d5c392ad14453aea40f1d1ed1f76c92d4b7a38f90a3ecbe5a43c"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1f171fb7ad1288cab83680a593ba9d37",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17033,
                "upload_time": "2019-07-20T01:57:17",
                "upload_time_iso_8601": "2019-07-20T01:57:17.918303Z",
                "url": "https://files.pythonhosted.org/packages/27/1b/547a59a59036c6e9d24b7b48d797be1ed5a24ba14526a5713c015d8a5464/pyshard-0.2.0.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a9e63562446058473a4d4317e65a683f",
                    "sha256": "94b878f17cb5d1128200c1b143d7b9d9ed0ed5bb928d132d6206b60240e35ad2"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a9e63562446058473a4d4317e65a683f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 21509,
                "upload_time": "2019-07-24T17:10:23",
                "upload_time_iso_8601": "2019-07-24T17:10:23.151631Z",
                "url": "https://files.pythonhosted.org/packages/5b/31/62d98574d08abdd365df385c0827574071297fc0a8d340a183b546775dc6/pyshard-0.2.1-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "69b1c9e890f811c45bf39fff24448406",
                    "sha256": "af294c9cc68404b916988dfadd7702426bcb347018f19a64d8729df8f9540307"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "69b1c9e890f811c45bf39fff24448406",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 15732,
                "upload_time": "2019-07-24T17:10:25",
                "upload_time_iso_8601": "2019-07-24T17:10:25.174785Z",
                "url": "https://files.pythonhosted.org/packages/49/8a/f8b3fe38fc50e1045a439fe36ae73c3a6ba9b5d9d4bdc6c0f7cdba6a8276/pyshard-0.2.1.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "53fae5648970698d656cde25a55c1f08",
                    "sha256": "67976e6b7641aa8126477233d460dea46d5e9003ce7e13ff213a999d5f23a3f1"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "53fae5648970698d656cde25a55c1f08",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 22080,
                "upload_time": "2019-10-29T00:20:10",
                "upload_time_iso_8601": "2019-10-29T00:20:10.889759Z",
                "url": "https://files.pythonhosted.org/packages/01/04/3f9971a5337aaa6b968f5c9ed579f1c5aba79ec3f9841a6139d0bfdd9cb1/pyshard-0.2.2-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "63d714b6348c14f9315030cd247ef881",
                    "sha256": "125d5bdd1055ee113f35cc220fce909d31edf57b14999fdbdfce51365a2b8881"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "63d714b6348c14f9315030cd247ef881",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18110,
                "upload_time": "2019-10-29T00:20:12",
                "upload_time_iso_8601": "2019-10-29T00:20:12.512259Z",
                "url": "https://files.pythonhosted.org/packages/33/e0/0153155b433b55dd1661972e376e1d6d2fcebf8a5ba6679d487d6f6e2075/pyshard-0.2.2.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ],
        "0.2.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "775d8c289f82f520623a5132ba9d8218",
                    "sha256": "e4f356e8d9c2a4e5ffc3c79c31bbd72ca8a33bc0219f3b8bf9bf9ceeab088597"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "775d8c289f82f520623a5132ba9d8218",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 22319,
                "upload_time": "2019-10-29T15:28:15",
                "upload_time_iso_8601": "2019-10-29T15:28:15.718429Z",
                "url": "https://files.pythonhosted.org/packages/61/2f/7a146f6323d0211377f81a93d464612447545528ba190c62c1995b98441c/pyshard-0.2.3-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "82a44da1afbe4c5cf0611379f1a9067b",
                    "sha256": "8742d5d32440f1ed2c40d0e16919c9c8d043f1ca4df3e9dd679d2c6424bf564a"
                },
                "downloads": -1,
                "filename": "pyshard-0.2.3.tar.gz",
                "has_sig": false,
                "md5_digest": "82a44da1afbe4c5cf0611379f1a9067b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 16572,
                "upload_time": "2019-10-29T15:28:19",
                "upload_time_iso_8601": "2019-10-29T15:28:19.203934Z",
                "url": "https://files.pythonhosted.org/packages/74/b3/f2d5d5cd9f0a92509ed0b7e3adfe0aed56e414bc9fd3631cc3adfab3c621/pyshard-0.2.3.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "775d8c289f82f520623a5132ba9d8218",
                "sha256": "e4f356e8d9c2a4e5ffc3c79c31bbd72ca8a33bc0219f3b8bf9bf9ceeab088597"
            },
            "downloads": -1,
            "filename": "pyshard-0.2.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "775d8c289f82f520623a5132ba9d8218",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 22319,
            "upload_time": "2019-10-29T15:28:15",
            "upload_time_iso_8601": "2019-10-29T15:28:15.718429Z",
            "url": "https://files.pythonhosted.org/packages/61/2f/7a146f6323d0211377f81a93d464612447545528ba190c62c1995b98441c/pyshard-0.2.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "82a44da1afbe4c5cf0611379f1a9067b",
                "sha256": "8742d5d32440f1ed2c40d0e16919c9c8d043f1ca4df3e9dd679d2c6424bf564a"
            },
            "downloads": -1,
            "filename": "pyshard-0.2.3.tar.gz",
            "has_sig": false,
            "md5_digest": "82a44da1afbe4c5cf0611379f1a9067b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 16572,
            "upload_time": "2019-10-29T15:28:19",
            "upload_time_iso_8601": "2019-10-29T15:28:19.203934Z",
            "url": "https://files.pythonhosted.org/packages/74/b3/f2d5d5cd9f0a92509ed0b7e3adfe0aed56e414bc9fd3631cc3adfab3c621/pyshard-0.2.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "vulnerabilities": []
}