{ "info": { "author": "Da_Blitz", "author_email": "code@pocketnix.org", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: ISC License (ISCL)", "Operating System :: OS Independent", "Programming Language :: Python :: 3.3", "Topic :: Software Development :: Interpreters", "Topic :: Software Development :: Libraries", "Topic :: System :: Clustering", "Topic :: System :: Distributed Computing" ], "description": "CRUSH HA\n========\nProof of concept of using CRUSH maps for HA node placement\n\nBased off of crush maps from `ceph `_ and research paper \nbehind it by (linked `here `_)\n\nA Gentle Introduction\n----------------------\nThe CRUSH Algorithm as it is used in ceph is used to determine on which nodes \ndata of interest may be stored without a central controller. This is done with \n'buckets' that define the relationship between nodes (or in the case of ceph \nOSDs) and their failure/allocation domains and a set of rules which specify the \nplacement of data on these nodes (and therefore their high availability \ncharacteristics) and is implemented as a simple VM that operates on the \nbuckets.\n\nTo put it more simply, the buckets define the 'topology' of your setup (how \nthings are linked and how they should fail) and rules define which nodes you \nselect for data and operate on these buckets. By manipulating the rules in \nrelationship to the failure domains in the buckets you can define arbitrary HA \npolices that prevent against any type of data loss in a generic fashion.\n\nWhy you may want to use it\n--------------------------\nThis library or its concepts may be of sue for the following problems\n\n * cache server selection\n * locating services in a cluster without a central controller\n * making placement decisions for data/jobs\n * reference for building basic byte code interpreters\n\nDiffrences to Ceph's CRUSH maps\n-------------------------------\n* Only linear buckets are implemented\n* No weights for buckets\n* Selection mechanism currently takes buckets in order they where added\n\nThe above differences are intended to be eliminated before a v1.0 release\n\nAvailable Commands\n------------------\nAs this implementation uses an interpreter, there are several instructions that \nyou can use. these map almost 1:1 with those from the original paper with some \nslight modification.\n\nuse\n####\nThe use command is used to select a bucket other than the default one provided \nto the interpreter, eg when you have to select a specific rack or geographical \nregion. As the buckets form a tree this updates the root to point to the \nspecified bucket. Note that you can still go back to the original root with \nanother use command.\n\nselect\n######\nSelect looks for nodes of the specified type and amount and places them on the \nstack. This is typically part of a 'wide' operation or 'left to right' \noperation where the intent is to cover as many possibilities before drilling \ndown.\n\ntake\n####\nTake is the counter part to select, unlike select this is a 'deep' or 'top to \nbottom' operation. For each item on the stack take will select count number of \nnodes\n\nemit\n####\nEmit takes the currently accumulated node choices and sends them back to the \ncaller while resetting the node list and stack list. This is useful as a \nbarrier if you have 2 types of criteria that need to be satisfied eg 2 nodes \nlocally, then a node remotely. Selection of the first 2 nodes would be done with \nthe above rules then 'emited', resetting the decision tree. The next set of \ninstructions would then satisfy the remote node selection and not be \nconstrained by earlier selections.\n\nTools\n-----\nThe following command can be used to run a demo program:\n\n $ python -m crushha.interpreter\n\nWhile the output is not that interesting the src code backing it may be of use \nif you intend to use this module. Setting up and enabling logging will also \ngive a highly detailed and annotated look into the decisions behind each nodes \nselection for auditing purposes.\n\nCurrent State\n--------------\nThis module is currently in a very alpha state and subject to change however \nthe api defined by the implementation is about 80% of the way there. This will \nbe designed as a coroutine due to the requirement to be able to generate an \narbitrary long list of nodes from a large pool and the iteration model it \nprovides has proved useful to work with\n\nThe current design makes use of 'yield from' from python 3.3, python versions \nbefore this will not be considered as a porting target due to the complexity of \nthe required trampolines to make it work correctly. The code is simple and is \nintended to stay so.\n\nThe Future\n----------\n * Better defined op codes\n * slight optimization for shorter rules (eliminating buckets by name in \n select)\n * Finalization of the API\n * More examples\n * hash ring bucket types\n * Predictable selection mechanism\n\nRepo: `blitz.works `_\nDocs: `docs@blitz.works `_\nEmail: code at pocketnix.org\n\n\nChangelog\n---------\n\n0.3 2016-04-03\n##############\n* Bucket.typ changed to Bucket.cls\n* More testing of basic garentees and properties of api\n* refrences to 'cmds' replaced with 'rules' and 'rulesets'\n* Bucket renamed 'LinerBucket' to bring it inline with CRUSH map paper\n* Buckets moved into seperate module\n\n0.2 2016-04-03\n##############\n\n* Added implicit 'emit' rule to all chains passed to the intepreter\n* Basic test suite added\n* Interpreter now checks there are rules to avoid infinite loops (this does not eliminate all infinite loops)\n\n0.1 2016-04-02\n##############\n\n* Inital Release\n* Working interpreter\n* Example code in interpreter.py", "description_content_type": null, "docs_url": "https://pythonhosted.org/crushha/", "download_url": "http://blitz.works/crushha/archive/tip.zip", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://blitz.works/crushha", "keywords": "crush CRUSH ha ceph containers", "license": "ISC", "maintainer": null, "maintainer_email": null, "name": "crushha", "package_url": "https://pypi.org/project/crushha/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/crushha/", "project_urls": { "Download": "http://blitz.works/crushha/archive/tip.zip", "Homepage": "http://blitz.works/crushha" }, "release_url": "https://pypi.org/project/crushha/0.2/", "requires_dist": null, "requires_python": null, "summary": "Define HA domains with CRUSH maps", "version": "0.2" }, "last_serial": 2055975, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "d8c004e2fde9dc820d28325d4d95e7fe", "sha256": "e1b48ef1ba08078cbdf439a5a281fb2ef55c1c94c2f8c60ea165ddbb018e1499" }, "downloads": -1, "filename": "crushha-0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "d8c004e2fde9dc820d28325d4d95e7fe", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 8334, "upload_time": "2016-04-02T14:08:28", "url": "https://files.pythonhosted.org/packages/3d/54/f162d2e5276909ea8c308d5d37c650ef29f3e3f06a4384f44d96de8d2202/crushha-0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "efbc18c9f29a2b87d5e706a94decdbe2", "sha256": "0f0866ce49dd1eb4990b2bf58a7db357811e2ef6b3f5e917324ef6ee94e9ac0c" }, "downloads": -1, "filename": "crushha-0.1.tar.bz2", "has_sig": false, "md5_digest": "efbc18c9f29a2b87d5e706a94decdbe2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8484, "upload_time": "2016-04-02T14:07:42", "url": "https://files.pythonhosted.org/packages/b2/2d/1d7b746f0cdaca247dccb2703ec3a3fa947d83fb02361cf6ff9bc1b488d7/crushha-0.1.tar.bz2" }, { "comment_text": "", "digests": { "md5": "07dfb919f72bf106e156471e9fd15233", "sha256": "7b78fccded7a3fa620e1526fbbada6aae262edda73cb09b554849085e07037fa" }, "downloads": -1, "filename": "crushha-0.1.zip", "has_sig": false, "md5_digest": "07dfb919f72bf106e156471e9fd15233", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14106, "upload_time": "2016-04-02T14:07:48", "url": "https://files.pythonhosted.org/packages/9a/c4/2d7f4d54c1fdc130473a91a005bc1cf8064bf10c33ec494fb9b703a2a8b1/crushha-0.1.zip" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "3a8dd01c7baae49e8e1095a3f9f04abc", "sha256": "85db72aed9a1a68a690389d17592bb26a810a6f64cd021b46aecf4c4ad0afd94" }, "downloads": -1, "filename": "crushha-0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3a8dd01c7baae49e8e1095a3f9f04abc", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 8817, "upload_time": "2016-04-03T12:11:39", "url": "https://files.pythonhosted.org/packages/c2/f0/664c71fdbe885ed215215d424586c557f106bc7b62448c1027f77646080d/crushha-0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ca2f3555ffc8a46be883ef13d55af469", "sha256": "89f65381a74eb7ce7627168fd789d4741cc7e2e99065870fd96bf718975add96" }, "downloads": -1, "filename": "crushha-0.2.tar.bz2", "has_sig": false, "md5_digest": "ca2f3555ffc8a46be883ef13d55af469", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11965, "upload_time": "2016-04-03T12:11:25", "url": "https://files.pythonhosted.org/packages/5d/4b/8fc13ae552aad5d8692b4b96a8bf02c3d5de1d3f025223213e16ca7a0b30/crushha-0.2.tar.bz2" }, { "comment_text": "", "digests": { "md5": "b074b5fdba2cd01bd43f0b07665830d3", "sha256": "df17b129d9798faec4df8e51a2c54344c57bfaeb005f0567f8821fc6338c6f72" }, "downloads": -1, "filename": "crushha-0.2.zip", "has_sig": false, "md5_digest": "b074b5fdba2cd01bd43f0b07665830d3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20569, "upload_time": "2016-04-03T12:11:31", "url": "https://files.pythonhosted.org/packages/2c/b5/de32c9f591aed29597283251916284bfeec67b1cb0f0810ab1540a9ccece/crushha-0.2.zip" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3a8dd01c7baae49e8e1095a3f9f04abc", "sha256": "85db72aed9a1a68a690389d17592bb26a810a6f64cd021b46aecf4c4ad0afd94" }, "downloads": -1, "filename": "crushha-0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "3a8dd01c7baae49e8e1095a3f9f04abc", "packagetype": "bdist_wheel", "python_version": "3.4", "requires_python": null, "size": 8817, "upload_time": "2016-04-03T12:11:39", "url": "https://files.pythonhosted.org/packages/c2/f0/664c71fdbe885ed215215d424586c557f106bc7b62448c1027f77646080d/crushha-0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ca2f3555ffc8a46be883ef13d55af469", "sha256": "89f65381a74eb7ce7627168fd789d4741cc7e2e99065870fd96bf718975add96" }, "downloads": -1, "filename": "crushha-0.2.tar.bz2", "has_sig": false, "md5_digest": "ca2f3555ffc8a46be883ef13d55af469", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11965, "upload_time": "2016-04-03T12:11:25", "url": "https://files.pythonhosted.org/packages/5d/4b/8fc13ae552aad5d8692b4b96a8bf02c3d5de1d3f025223213e16ca7a0b30/crushha-0.2.tar.bz2" }, { "comment_text": "", "digests": { "md5": "b074b5fdba2cd01bd43f0b07665830d3", "sha256": "df17b129d9798faec4df8e51a2c54344c57bfaeb005f0567f8821fc6338c6f72" }, "downloads": -1, "filename": "crushha-0.2.zip", "has_sig": false, "md5_digest": "b074b5fdba2cd01bd43f0b07665830d3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20569, "upload_time": "2016-04-03T12:11:31", "url": "https://files.pythonhosted.org/packages/2c/b5/de32c9f591aed29597283251916284bfeec67b1cb0f0810ab1540a9ccece/crushha-0.2.zip" } ] }