{ "info": { "author": "Ben Wiederhake", "author_email": "BenWiederhake.GitHub@gmx.de", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7" ], "description": "# atomic_store\n\n> Easier than a DBMS, but more fault-resistant than just a file.\n\nSometimes you need to manage a bit of state across executions.\nSometimes, a fully-blown database is just too much.\n\nThis library makes it easy to keep a *store* of stuff in a JSON file,\nin an atomic and fault-resistant manner.\n\nOther formats (like pickle and bson) are also supported,\nand arbitrary formats are possible.\n\n## Table of Contents\n\n- [Install](#install)\n- [Usage](#usage)\n- [TODOs](#todos)\n- [NOTDOs](#notdos)\n- [Contribute](#contribute)\n\n## Install\n\nJust `pip install atomic_store`. Or, if you must, `pip install -r requirements.txt`\n\nNote that the only dependency is `atomicwrites`, which has no dependencies.\n\n## Usage\n\nBy default, the store is encoded as json, written to a temporary file,\nand then atomically replaces the old file. When reading, if the file does\nnot exist, a default value is used. The default default value is `None`.\n\n### Context Manager\n\nThis program remembers all start times:\n\n```python3\nimport atomic_store\nimport time\n\nwith atomic_store.open('runs.json', default=[]) as store:\n print('Previous executions:')\n print(store.value)\n new_entry = time.strftime('%Y-%m-%d %H:%M:%S%z')\n store.value.append(new_entry)\n```\n\nLeaving the context manager takes care of all writes.\nNo intermediate values get written to disk.\n\nThis is ideal if the task runs short, and in case of any error\nyou only want to keep the old state anyway.\n\nFor advanced uses, also see the subsection on [reentrancy](#reentrancy).\n\n### Manual control\n\nThis program remembers all start times:\n\n```python3\nimport atomic_store\n\nmy_store = atomic_store.open('gathered.json', default=dict())\n\nmy_store.value['state'] = 'running'\nmy_store.value['thought'] = 'I would not eat green eggs and ham.'\nmy_store.commit()\n# ... some calculations ...\nmy_store.value['state'] = 'done'\nmy_store.value['thought'] = 'I do so like Green eggs and ham!'\nmy_store.commit()\n```\n\nOnly calls to `commit()` cause writes to the disk.\nAgain, no intermediate values get written to disk.\n\nThis is ideal if you have a long-running job with clear steps,\nand each step's output is valuable.\n\nNote that `commit()` is also available in the context manager.\n\n### Format tweaks\n\nIf you're using the json backend, and want to keep the JSON file as small as possible,\nyou can call `open` with `dump_kwargs=dict(separators=(',', ':'))`.\nThe keyword `load_kwargs` also exists.\n\n### Non-JSON formats\n\nYou can use arbitrary other formats, using the `format` keyword:\n\n```python\natomic_store.open('runs.json', default=[], format=MY_FORMAT)\n```\n\nSupported values are `None` (for JSON), `'json'`, `'pickle'`,\n`'bson'` (requires bson to be installed), and also any module or object\nproviding `dump/load` or `dumps/loads`.\nBy default, `atomic_store` assumes you operate on binary files, except when JSON is involved.\nTo override this, you can set `is_binary`.\nNote that this means you can use the modules `json`, `pickle`, and `bson` as they are.\n\nFor convenience, you can also override the abstract classes\n`atomic_store.AbstractFormatFile` or `atomic_store.AbstractFormatBstr`.\n\nIn all cases, `load_kwargs` and `dump_kwargs` are still supported.\n\n### Reentrancy\n\nIf the same `atomic_store` is used as a context manager more than once,\nthe default behavior is to write the file only when the last `with` is exited:\n\n```python\n# Assume `state.json` contains only `\"before\"`.\nmngr = atomic_store.open('mystate.json', default=[])\nwith mngr as store:\n store.value = 'outer'\n # File contains `\"before\"`: We haven't exited any context manager yet.\n with mngr as store:\n store.value = 'inner'\n # File contains `\"before\"`: We haven't exited any context manager yet.\n # File now contains `\"inner\"`, because the inner `with`-statement wrote it.\n # Read the Reentrancy section if you consider this undesired behavior.\n# File now contains `\"inner\"`, because the outer `with`-statement wrote it again.\n```\n\nIf you consider this behavior undesirable, you can either just use multiple context managers (by calling `atomic_store.open` multiple times), or by using the keyword `ignore_inner_exits=True`, like this:\n\n```python\n# Assume `state.json` contains only `\"before\"`.\nmngr = atomic_store.open('mystate.json', default=[], ignore_inner_exits=True)\nwith mngr as store:\n store.value = 'outer'\n # File contains `\"before\"`: We haven't exited any context manager yet.\n with mngr as store:\n store.value = 'inner'\n # File contains `\"before\"`: We haven't exited any context manager yet.\n # File *still* contains `\"before\"`, as the manager detected that it is still active.\n# File now contains `\"outer\"`, because the outer `with`-statement wrote it.\n```\n\n### Atomic is not magic\n\nThis library is not magical.\n\nIf two threads (or two processes, or whatever) open a store,\nmodify something, and then write concurrently, one of the results may be lost.\nHowever, the writes are guaranteed to be atomic,\nso the data is merely lost, but not corrupted.\n\n## TODOs\n\n* Figure out how to make `bson` optional\n* Publish on PyPI\n\n## NOTDOs\n\nHere are some things this project will not support:\n* Any DB backend.\n* Any multi-file backend.\n* More advanced semantics than just `commit`.\n* This includes rollback. It's just not obvious which behavior is desired when the file does not exist (Re-use `default` value? What if it was modified, as it happens with lists and dicts?), and with stacked context managers (should it rollback to the file's state? Or to the beginning of the `with`?)\n\n## Contribute\n\nFeel free to dive in! [Open an issue](https://github.com/BenWiederhake/atomic_store/issues/new) or submit PRs.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/BenWiederhake/atomic_store", "keywords": "atomic store", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "atomic-store", "package_url": "https://pypi.org/project/atomic-store/", "platform": "Any", "project_url": "https://pypi.org/project/atomic-store/", "project_urls": { "Homepage": "https://github.com/BenWiederhake/atomic_store" }, "release_url": "https://pypi.org/project/atomic-store/0.0.1/", "requires_dist": [ "atomicwrites (>=1.3.0)", "bson (>=0.5.8)" ], "requires_python": "", "summary": "A robust, atomic single-file value store", "version": "0.0.1" }, "last_serial": 5531975, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "ca013d36b81267aaddac8e63b4283b62", "sha256": "c6228ad6a631249ee50c181b25fdd852a2df43812385306b355c975b8c07e2cb" }, "downloads": -1, "filename": "atomic_store-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "ca013d36b81267aaddac8e63b4283b62", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 8017, "upload_time": "2019-07-14T20:29:20", "url": "https://files.pythonhosted.org/packages/fa/a9/3a41a6128c62036f4690ae4f5b25477711b5f95d53e9f0246e94cffae57f/atomic_store-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "662660607425ad6223d791b78f9c2ad0", "sha256": "3ce00b2819f3ee3207de79c2aeb66aee1566a4c1fdbae317c72d0ff20a3729e5" }, "downloads": -1, "filename": "atomic_store-0.0.1.tar.gz", "has_sig": false, "md5_digest": "662660607425ad6223d791b78f9c2ad0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6640, "upload_time": "2019-07-14T20:29:22", "url": "https://files.pythonhosted.org/packages/e4/4a/934ebb885d353c63d2b3f2eeda6c7fc01e254685a07c4e16947224754114/atomic_store-0.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ca013d36b81267aaddac8e63b4283b62", "sha256": "c6228ad6a631249ee50c181b25fdd852a2df43812385306b355c975b8c07e2cb" }, "downloads": -1, "filename": "atomic_store-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "ca013d36b81267aaddac8e63b4283b62", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 8017, "upload_time": "2019-07-14T20:29:20", "url": "https://files.pythonhosted.org/packages/fa/a9/3a41a6128c62036f4690ae4f5b25477711b5f95d53e9f0246e94cffae57f/atomic_store-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "662660607425ad6223d791b78f9c2ad0", "sha256": "3ce00b2819f3ee3207de79c2aeb66aee1566a4c1fdbae317c72d0ff20a3729e5" }, "downloads": -1, "filename": "atomic_store-0.0.1.tar.gz", "has_sig": false, "md5_digest": "662660607425ad6223d791b78f9c2ad0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6640, "upload_time": "2019-07-14T20:29:22", "url": "https://files.pythonhosted.org/packages/e4/4a/934ebb885d353c63d2b3f2eeda6c7fc01e254685a07c4e16947224754114/atomic_store-0.0.1.tar.gz" } ] }