{ "info": { "author": "Tim Cogan", "author_email": "timothycogan@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "# Hash Array Snapshot\n## Overview\nThis project provides a solution to the following two problems:\n1. Dataset revisions are seldom tracked effectively but can be just as important to track as source code revisions\n2. Version control systems such as Git do not handle large files (i.e. > 1GB) efficiently and struggle to track revisions of large datasets\n\nHash array snapshots solve these two problems because they are easily tracked by Git and provide a record of the data files associated with a specific source code revision.\n\n## Examples\nSuppose we have a dataset composed of images stored in folders.\n\n -dataset\n -square\n square0.png\n square1.png\n square2.png\n -triangle\n triangle0.png\n triangle1.png\n triangle2.png\n\n*$ has snap -d dataset*\n\nWill create *snapshot.has* with the following contents:\n\n34dc214a2aea8d7c254a9d6dc351e0d3c0088ad998eed6053b78877785fcdff1:triangle/triangle0.png\n566f5fa0703f5c2877c38fb3aae0fabbc5f9cdb25499b4f03ca75a6eb3827961:square/square0.png\n67240c2cee6e9c77df1192890b1cf4deb265a5a6afdb4a5ecc03e93cc5889cef:triangle/triangle2.png\ndfb6352f5d42793b58ac74f2cacf5f1f82bdb1470a30941224a0f1e34766aeb4:square/square2.png\ne361db7913f495dafee06657ea67043a49c06fa1a3c57d3ed5b1a9048455de8f:square/square1.png\nf7994454bf5a880c5741b3af8e0ababf77f8c450fe47ed8b5c6f7b9d38c9115f:triangle/triangle1.png\n\nSometime later, additional circle data is added to our dataset and the overall naming convention is changed.\n\n -dataset\n -square\n square_a.png\n square_b.png\n square_c.png\n -triangle\n triangle_a.png\n triangle_b.png\n triangle_c.png\n -circle\n circle_a.png\n circle_b.png\n circle_c.png\n\nWe can use *has check* to verify the dataset is different from what we recorded in our snapshot.\n\n*$ has check -d dataset*\n\nFiles added/modified: \n181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b:circle/circle_c.png\n61b4c705859f4158d38090c1e38e8fdc4f3d29db007f012766276aa498835cf6:circle/circle_a.png\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855:circle/circle_b.png\n\nEven though the triangle and square images have been renamed, files are tracked according to hash value rather than filename so those files are not seen as being new.\n\nWe can use our snapshot file *snapshot.has* to recover the older version of our dataset into an empty directory *tmp*.\n\n*$ has recover -b dataset -d tmp*\n\n -tmp\n -square\n square0.png\n square1.png\n square2.png\n -triangle\n triangle0.png\n triangle1.png\n triangle2.png\n\nFolder *tmp* is now identical to our previous version of folder *dataset*.\n\nWe can check the contents of *tmp* to ensure that all files have been copied successfully.\n\n*$ has check -d tmp*\n\n## Installation\nThe easiest way to install *has* is with *pip install has*.\n\nAlternatively, checkout the latest release version of *has* (e.g. *git checkout v0.0.x*), and run *sudo ./install.sh*. Open a terminal and type *has --help* to verify installation. Uninstall *has* by running *sudo ./uninstall.sh*.\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://gitlab.com/timcogan/hash_array_snapshot", "keywords": "", "license": "Apache", "maintainer": "", "maintainer_email": "", "name": "has", "package_url": "https://pypi.org/project/has/", "platform": "", "project_url": "https://pypi.org/project/has/", "project_urls": { "Homepage": "https://gitlab.com/timcogan/hash_array_snapshot" }, "release_url": "https://pypi.org/project/has/0.0.4/", "requires_dist": null, "requires_python": "", "summary": "Hash array snapshot", "version": "0.0.4" }, "last_serial": 4527953, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "b94ccae97e7474ce06bafd1da6f5c15b", "sha256": "385149875e5b3603f8519e0f43afb6d413982924673a9ae491aee5d5d134ce19" }, "downloads": -1, "filename": "has-0.0.1.tar.gz", "has_sig": false, "md5_digest": "b94ccae97e7474ce06bafd1da6f5c15b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4483, "upload_time": "2018-11-25T03:30:00", "url": "https://files.pythonhosted.org/packages/d4/87/4fa93ccdcbba9571dfaca0c56a8e58a1762f3decee299e896b6a6fae8056/has-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "53e14d71c6c99252c3b37e151f5a8a62", "sha256": "782d0345eb39f63f9655892cfda257226ffb473b32194c792d91e0da14c020b7" }, "downloads": -1, "filename": "has-0.0.2.tar.gz", "has_sig": false, "md5_digest": "53e14d71c6c99252c3b37e151f5a8a62", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4491, "upload_time": "2018-11-25T03:33:41", "url": "https://files.pythonhosted.org/packages/45/7b/82f40c57ee7cfe464308cb92debbd20e19e0544b7e0bb56de717e74263a9/has-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "df75f1226668b9c4584fba6a4f5a9069", "sha256": "0e2a9d097402f2a10da3bb5117ef5656975cfbd547709782995d75f14752d8e8" }, "downloads": -1, "filename": "has-0.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "df75f1226668b9c4584fba6a4f5a9069", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9827, "upload_time": "2018-11-26T02:46:06", "url": "https://files.pythonhosted.org/packages/18/42/a44d94c9e72eb7badd3076311fb38fb99896de315d4fc8e453b10c01f0e4/has-0.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c639f58803799e7a49679f81a9bbaf29", "sha256": "650dcd08fa0716e02ec6dfc5fa376fb66ba4ca7104bedb6c684bcfb0af42caef" }, "downloads": -1, "filename": "has-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "c639f58803799e7a49679f81a9bbaf29", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9822, "upload_time": "2018-11-26T02:46:07", "url": "https://files.pythonhosted.org/packages/a2/f3/7e202a16aee2b302dc0feed85dccc0c70f6ceec5ebfc5efda4583a937235/has-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f5964c665d620f0d15b7c34278f983e5", "sha256": "c79ec76ffca9e7e86aaf8e2bcb5c4eb3294d60a4e119e2023b68017a4ef58ebd" }, "downloads": -1, "filename": "has-0.0.3.tar.gz", "has_sig": false, "md5_digest": "f5964c665d620f0d15b7c34278f983e5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4923, "upload_time": "2018-11-26T02:46:08", "url": "https://files.pythonhosted.org/packages/0e/d5/076c5880d0dd70c27322904e89da0032b9d337bdf110515940aeb97a5b4f/has-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "b2fece7912c243560eee1aa9987193f4", "sha256": "ab237e1a99931df5288b94d16abeae66d770ad15505792f6a5d011e00341fefe" }, "downloads": -1, "filename": "has-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "b2fece7912c243560eee1aa9987193f4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9822, "upload_time": "2018-11-26T04:34:48", "url": "https://files.pythonhosted.org/packages/29/50/0def2f2be2636e01c1f2acc935330be652287ec63fc60c1e9ffcc2ad9222/has-0.0.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "00d2b66c27a209aa4d8643cabd81a221", "sha256": "be0fed65852abc2602a6f859fa78e82ca079b1f56e2ec8b1e2e9f18a952c1fd6" }, "downloads": -1, "filename": "has-0.0.4.tar.gz", "has_sig": false, "md5_digest": "00d2b66c27a209aa4d8643cabd81a221", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4925, "upload_time": "2018-11-26T04:34:50", "url": "https://files.pythonhosted.org/packages/ba/e6/e8d7c582707e4a4150189824b8ea98c5eb932130f41367401c072ea273e9/has-0.0.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b2fece7912c243560eee1aa9987193f4", "sha256": "ab237e1a99931df5288b94d16abeae66d770ad15505792f6a5d011e00341fefe" }, "downloads": -1, "filename": "has-0.0.4-py3-none-any.whl", "has_sig": false, "md5_digest": "b2fece7912c243560eee1aa9987193f4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9822, "upload_time": "2018-11-26T04:34:48", "url": "https://files.pythonhosted.org/packages/29/50/0def2f2be2636e01c1f2acc935330be652287ec63fc60c1e9ffcc2ad9222/has-0.0.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "00d2b66c27a209aa4d8643cabd81a221", "sha256": "be0fed65852abc2602a6f859fa78e82ca079b1f56e2ec8b1e2e9f18a952c1fd6" }, "downloads": -1, "filename": "has-0.0.4.tar.gz", "has_sig": false, "md5_digest": "00d2b66c27a209aa4d8643cabd81a221", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4925, "upload_time": "2018-11-26T04:34:50", "url": "https://files.pythonhosted.org/packages/ba/e6/e8d7c582707e4a4150189824b8ea98c5eb932130f41367401c072ea273e9/has-0.0.4.tar.gz" } ] }