{ "info": { "author": "Matthew Rocklin", "author_email": "mrocklin@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "Chest\n=====\n\n|Build Status| |Coverage Status| |Version Status| |Downloads|\n\nA dictionary that spills to disk.\n\nChest acts likes a dictionary but it can write its contents to disk. This is\nuseful in the following two occasions:\n\n1. Chest can hold datasets that are larger than memory\n2. Chest persists and so can be saved and loaded for later use\n\nRelated Projects\n----------------\n\nThe standard library ``shelve`` is an alternative out-of-core dictionary.\n``Chest`` offers the following benefits over shelve_:\n\n1. Chest supports any hashable key (not just strings)\n2. Chest supports pluggable serialization and file saving schemes\n\nAlternatively one might consider a traditional key-value store database like\nRedis_.\n\nShove_ is another excellent alternative with support for a variety of stores.\n\n\nHow it works\n------------\n\nChest stores data in two locations\n\n1. An in-memory dictionary\n2. On the filesystem in a directory owned by the chest\n\nAs a user adds contents to the chest the in-memory dictionary fills up. When\na chest stores more data in memory than desired (see ``available_memory=``\nkeyword argument) it writes the larger contents of the chest to disk as pickle\nfiles (the choice of ``pickle`` is configurable). When a user asks for a value\nchest checks the in-memory store, then checks on-disk and loads the value into\nmemory if necessary, pushing other values to disk.\n\nChest is a simple project. It was intended to provide a simple interface to\nassist in the storage and retrieval of numpy arrays. However it's design and\nimplementation are agnostic to this case and so could be used in a variety of\nother situations.\n\nWith minimal work chest could be extended to serve as a communication point\nbetween multiple processes.\n\n\nKnown Failings\n--------------\n\nChest was designed to hold a moderate amount of largish numpy arrays. It\ndoesn't handle the very many small key-value pairs usecase (though could with\nsmall effort). In particular chest has the following deficiencies\n\n1. Chest is not multi-process safe. We should institute a file lock at least\n around the ``.keys`` file.\n2. Chest does not support mutation of variables on disk.\n\n\nLICENSE\n-------\n\nNew BSD. See License_\n\n\nInstall\n-------\n\n``chest`` is available through ``conda``::\n\n conda install chest\n\n``chest`` is on the Python Package Index (PyPI)::\n\n pip install chest\n\n\nExample\n-------\n\n.. code:: python\n\n >>> from chest import Chest\n >>> c = Chest()\n\n >>> # Acts like a normal dictionary\n >>> c['x'] = [1, 2, 3]\n >>> c['x']\n [1, 2, 3]\n\n >>> # Data persists to local files\n >>> c.flush()\n >>> import os\n >>> os.listdir(c.path)\n ['.keys', 'x']\n\n >>> # These files hold pickled results\n >>> import pickle\n >>> pickle.load(open(c.key_to_filename('x')))\n [1, 2, 3]\n\n >>> # Though one normally accesses these files with chest itself\n >>> c2 = Chest(path=c.path)\n >>> c2.keys()\n ['x']\n >>> c2['x']\n [1, 2, 3]\n\n >>> # Chest is configurable, so one can use json, etc. instead of pickle\n >>> import json\n >>> c = Chest(path='my-chest', dump=json.dump, load=json.load)\n >>> c['x'] = [1, 2, 3]\n >>> c.flush()\n\n >>> json.load(open(c.key_to_filename('x')))\n [1, 2, 3]\n\n\nDependencies\n------------\n\n``Chest`` supports Python 2.6+ and Python 3.2+ with a common codebase.\n\nIt currently depends on the ``heapdict`` library.\n\nIt is a light weight dependency.\n\n\n.. _shelve: https://docs.python.org/3/library/shelve.html\n.. _Shove: https://pypi.python.org/pypi/shove/0.5.6\n.. _License: https://github.com/ContinuumIO/chest/blob/master/LICENSE.txt\n.. _Redis: http://redis.io/\n.. |Build Status| image:: https://travis-ci.org/ContinuumIO/chest.png\n :target: https://travis-ci.org/ContinuumIO/chest\n.. |Coverage Status| image:: https://coveralls.io/repos/mrocklin/chest/badge.png\n :target: https://coveralls.io/r/mrocklin/chest\n.. |Version Status| image:: https://pypip.in/v/chest/badge.png\n :target: https://pypi.python.org/pypi/chest/\n.. |Downloads| image:: https://pypip.in/d/chest/badge.png\n :target: https://pypi.python.org/pypi/chest/", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ContinuumIO/chest", "keywords": "dictionary out-of-core", "license": "BSD", "maintainer": null, "maintainer_email": null, "name": "chest", "package_url": "https://pypi.org/project/chest/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/chest/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/ContinuumIO/chest" }, "release_url": "https://pypi.org/project/chest/0.2.3/", "requires_dist": null, "requires_python": null, "summary": "Simple on-disk dictionary", "version": "0.2.3" }, "last_serial": 1621451, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "8adca786f96e78b5b61c343660216c38", "sha256": "ede42695b757d48bff51a8b68aecaf27e9b4cb3968eafc879609f58d098d5d92" }, "downloads": -1, "filename": "chest-0.1.0.tar.gz", "has_sig": false, "md5_digest": "8adca786f96e78b5b61c343660216c38", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4629, "upload_time": "2014-11-29T17:22:24", "url": "https://files.pythonhosted.org/packages/8f/63/3efa3123b18623767997710b2cc4896279e509b9c6d9baf20dc5a988a615/chest-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "8ae88b66627cd06c5ca98dacf1e0962b", "sha256": "498537ad3f61a6cfd3f7a3abddc9efecb8a27b72c1e59547eee1e7738def6e8d" }, "downloads": -1, "filename": "chest-0.2.0.tar.gz", "has_sig": false, "md5_digest": "8ae88b66627cd06c5ca98dacf1e0962b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5616, "upload_time": "2015-01-14T18:10:14", "url": "https://files.pythonhosted.org/packages/ac/77/972d412712f4f2b409f79903d1e7a9e86aa373fbaca054095294ab7fefa8/chest-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "ba52d314354a27cdc632a31ab582c006", "sha256": "a80b3baf4cc636db387814bad617e1c9895e6a6bb5c566cc6c7fb032bd9ba7db" }, "downloads": -1, "filename": "chest-0.2.1.tar.gz", "has_sig": false, "md5_digest": "ba52d314354a27cdc632a31ab582c006", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9008, "upload_time": "2015-03-06T03:01:24", "url": "https://files.pythonhosted.org/packages/09/e0/6f774dc77e7d35c31936c18e66ffbaf70e4156b940f649409cad092486e9/chest-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "8027e26ac2dfcd26d485c279a3b3a2b0", "sha256": "9cfa408154d87d628ffe4d6a857cd831369a4391950117435848b9fad1b906fb" }, "downloads": -1, "filename": "chest-0.2.2.tar.gz", "has_sig": false, "md5_digest": "8027e26ac2dfcd26d485c279a3b3a2b0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9504, "upload_time": "2015-03-26T05:50:55", "url": "https://files.pythonhosted.org/packages/ba/ed/0ffddd4eb1ee96af13226b21818fe1532f37af344c677c81172bc0d116ef/chest-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "1f5fc22d0caad0fc63e6bd68cc3ca873", "sha256": "f2d1030d4720fd4c0cb258c3383d4bab764cfe441bab1366a0d186b0baf4f4d6" }, "downloads": -1, "filename": "chest-0.2.3.tar.gz", "has_sig": false, "md5_digest": "1f5fc22d0caad0fc63e6bd68cc3ca873", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9614, "upload_time": "2015-07-06T18:42:29", "url": "https://files.pythonhosted.org/packages/18/66/b883b9a26cd2f777dd04b7eedc842d31ea1567b7709b049d46eca418501e/chest-0.2.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1f5fc22d0caad0fc63e6bd68cc3ca873", "sha256": "f2d1030d4720fd4c0cb258c3383d4bab764cfe441bab1366a0d186b0baf4f4d6" }, "downloads": -1, "filename": "chest-0.2.3.tar.gz", "has_sig": false, "md5_digest": "1f5fc22d0caad0fc63e6bd68cc3ca873", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9614, "upload_time": "2015-07-06T18:42:29", "url": "https://files.pythonhosted.org/packages/18/66/b883b9a26cd2f777dd04b7eedc842d31ea1567b7709b049d46eca418501e/chest-0.2.3.tar.gz" } ] }