{ "info": { "author": "", "author_email": "", "bugtrack_url": null, "classifiers": [], "description": "=========================================\nDataFS Data Management System\n=========================================\n\n.. image:: https://img.shields.io/pypi/v/datafs.svg\n :target: https://pypi.python.org/pypi/datafs\n\n.. image:: https://travis-ci.org/ClimateImpactLab/DataFS.svg?branch=master\n :target: https://travis-ci.org/ClimateImpactLab/DataFS?branch=master\n\n.. image:: https://coveralls.io/repos/github/ClimateImpactLab/DataFS/badge.svg?branch=master\n :target: https://coveralls.io/github/ClimateImpactLab/DataFS?branch=master\n\n.. image:: https://readthedocs.org/projects/datafs/badge/?version=latest\n :target: https://datafs.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n.. image:: https://pyup.io/repos/github/ClimateImpactLab/DataFS/shield.svg\n :target: https://pyup.io/repos/github/ClimateImpactLab/DataFS/\n :alt: Updates\n\n.. image:: https://api.codacy.com/project/badge/Grade/5e095453424840e092e71c42b8ad8b52\n :alt: Codacy Badge\n :target: https://www.codacy.com/app/delgadom/DataFS?utm_source=github.com&utm_medium=referral&utm_content=ClimateImpactLab/DataFS&utm_campaign=badger\n\n.. image:: https://badges.gitter.im/DataFS/Lobby.svg\n :alt: Join the chat at https://gitter.im/DataFS/Lobby\n :target: https://gitter.im/DataFS/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge\n\n\nDataFS is a package manager for data. It manages file versions, dependencies, and metadata for individual use or large organizations.\n\nConfigure and connect to a metadata `Manager `_ and multiple data `Services `_ using a specification file and you'll be sharing, tracking, and using your data in seconds.\n\n\n* Free software: MIT license\n* Documentation: https://datafs.readthedocs.io.\n\n\nFeatures\n--------\n\n* Explicit version and metadata management for teams\n* Unified read/write interface across file systems\n* Easily create out-of-the-box configuration files for users\n* Track data dependencies and usage logs\n* Use datafs from python or from the command line\n* Permissions handled by managers & services, giving you control over user access\n\n\nUsage\n-----\n\nFirst, `configure an API `_. Don't worry. It's not too bad. Check out the `quickstart `_ to follow along.\n\nWe'll assume we already have an API object created and attached to a service called \"local\". Once you have this, you can start using DataFS to create and use archives.\n\n.. code-block:: bash\n\n $ datafs create my_new_data_archive --description \"a test archive\"\n created versioned archive \n \n $ echo \"initial file contents\" > my_file.txt\n \n $ datafs update my_new_data_archive my_file.txt\n \n $ datafs cat my_new_data_archive\n initial file contents\n\nVersions are tracked explicitly. Bump versions on write, and read old versions \nif desired.\n\n.. code-block:: bash\n\n $ echo \"updated contents\" > my_file.txt\n \n $ datafs update my_new_data_archive my_file.txt --bumpversion minor\n uploaded data to . version bumped 0.0.1 --> 0.1.\n \n $ datafs cat my_new_data_archive\n updated contents\n \n $ datafs cat my_new_data_archive --version 0.0.1\n initial file contents\n\nPin versions using a requirements file to set the default version\n\n.. code-block:: bash\n\n $ echo \"my_new_data_archive==0.0.1\" > requirements_data.txt\n \n $ datafs cat my_new_data_archive\n initial file contents\n\nAll of these features are available from (and faster in) python:\n\n.. code-block:: python\n\n >>> import datafs\n >>> api = datafs.get_api()\n >>> archive = api.get_archive('my_new_data_archive')\n >>> with archive.open('r', version='latest') as f:\n ... print(f.read())\n ...\n updated contents\n\n\nIf you have permission to delete archives, it's easy to do. See `administrative tools `_ for tips on setting permissions.\n\n.. code-block:: bash\n\n $ datafs delete my_new_data_archive\n deleted archive \n\nSee `examples `_ for more extensive use cases.\n\n\n\nInstallation\n------------\n\n``pip install datafs``\n\n\nAdditionally, you'll need a manager and services:\n\nManagers:\n\n* MongoDB: ``pip install pymongo``\n* DynamoDB: ``pip install boto3``\n\nServices:\n\n* Ready out-of-the-box:\n\n - local\n - shared\n - mounted\n - zip\n - ftp\n - http/https\n - in-memory\n\n* Requiring additional packages:\n\n - AWS/S3: ``pip install boto``\n - SFTP: ``pip install paramiko``\n - XMLRPC: ``pip install xmlrpclib``\n\n\nRequirements\n------------\n\nFor now, DataFS requires python 2.7. We're working on 3x support.\n\n\nTodo\n----\n\nSee `issues `_ to see and add to our todos.\n\n\nCredits\n---------\n\nThis package was created by `Justin Simcock `_ and `Michael Delgado `_ of the `Climate Impact Lab `_. Check us out on `github `_.\n\nMajor kudos to the folks at `PyFilesystem `_. Thanks also to `audreyr `_ for the wonderful `cookiecutter `_ package, and to `Pyup `_, a constant source of inspiration and our silent third contributor.\n", "description_content_type": null, "docs_url": "https://pythonhosted.org/datafs/", "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "datafs", "package_url": "https://pypi.org/project/datafs/", "platform": null, "project_url": "https://pypi.org/project/datafs/", "project_urls": null, "release_url": "https://pypi.org/project/datafs/0.7.1/", "requires_dist": null, "requires_python": "", "summary": "", "version": "0.7.1" }, "last_serial": 2877100, "releases": { "0.1.1": [ { "comment_text": "", "digests": { "md5": "6ac951eafba5b34304b23d3693841e1b", "sha256": "d72a559edfea886c4d54de42977dd4fa597bda71dd487bc3437246bdd91e4188" }, "downloads": -1, "filename": "datafs-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6ac951eafba5b34304b23d3693841e1b", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 4136, "upload_time": "2016-12-08T04:06:28", "url": "https://files.pythonhosted.org/packages/d1/5e/f2b46552c76336305455269a4a6ef074fd9d5739b21f02cfa125177ef580/datafs-0.1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0cbde604aa79bb112c758f05c479d742", "sha256": "5d05a89f3ed07ba22e8bed7c2784649569da2624c6b58617cdb8294b09018611" }, "downloads": -1, "filename": "datafs-0.1.1.tar.gz", "has_sig": false, "md5_digest": "0cbde604aa79bb112c758f05c479d742", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 24643, "upload_time": "2016-12-08T04:06:30", "url": "https://files.pythonhosted.org/packages/72/6c/1ba2b519e33d254ee6019f4dd11afc7c6a0295c3f98d46ddf14752843600/datafs-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "14234871274cf5b6c4ec1113b9682d2c", "sha256": "e21c364cf713da03043a8a53e30a301014c2325c3dccc3534a5608688475b767" }, "downloads": -1, "filename": "datafs-0.1.2.tar.gz", "has_sig": false, "md5_digest": "14234871274cf5b6c4ec1113b9682d2c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15402, "upload_time": "2016-12-08T04:03:36", "url": "https://files.pythonhosted.org/packages/3b/a4/b1aff269d6f230d0ae1b86dfaf844c7142649c0ba1c3ebc68db341750e37/datafs-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "b3f5691c9b7263b0139d994497ff2b6c", "sha256": "da339199e2ec067d2df57a5b62e807f1f6d8eaea7a807e9ae4979156c3dcc49f" }, "downloads": -1, "filename": "datafs-0.1.3.tar.gz", "has_sig": false, "md5_digest": "b3f5691c9b7263b0139d994497ff2b6c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15471, "upload_time": "2016-12-08T21:57:01", "url": "https://files.pythonhosted.org/packages/51/2a/db63496c806758b27afda37fd2785a71739f282ef36a80d3e9f6505ccbe8/datafs-0.1.3.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "faae5801b76c8da4a3914967a79a879c", "sha256": "ae1dd75ebac8a65fee46a12db1be8cc4bca403469eef0eb37d3507a3711a98be" }, "downloads": -1, "filename": "datafs-0.3.0.tar.gz", "has_sig": false, "md5_digest": "faae5801b76c8da4a3914967a79a879c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 26340, "upload_time": "2016-12-14T03:07:53", "url": "https://files.pythonhosted.org/packages/b2/13/ef1eac340797d9473ad92b860d574643cbbee75139296800e7bf4b018f85/datafs-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "a136a4a446956e3e6fcd094d86412153", "sha256": "b07d04f124914eb74f4cae4f919de18eeb00c49e761d881eaa76066344e1759a" }, "downloads": -1, "filename": "datafs-0.4.0.tar.gz", "has_sig": false, "md5_digest": "a136a4a446956e3e6fcd094d86412153", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 29370, "upload_time": "2016-12-15T23:14:21", "url": "https://files.pythonhosted.org/packages/e6/25/ee4d2d585b2829eab4fbcd293d70f2907a773bdc60d8bfe5d21a4aed361a/datafs-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "f8212da89da150d192469b486e0372b6", "sha256": "020c7be7b455a1850ba52b847d92a9fbfa26758ecc0eb36373dcaf33bbb247ac" }, "downloads": -1, "filename": "datafs-0.4.1.tar.gz", "has_sig": false, "md5_digest": "f8212da89da150d192469b486e0372b6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 34607, "upload_time": "2016-12-20T00:40:52", "url": "https://files.pythonhosted.org/packages/d8/b8/3291e2d93347117fb92ea740f04f8ea064cbc49a8074e4ba4b7d713d17df/datafs-0.4.1.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "0336ea64470295a57826c607b9ee0f7d", "sha256": "e8ec1e199c8d3dee86025ecb8558da932f3176836c841efe11fffa6db7f680cf" }, "downloads": -1, "filename": "datafs-0.5.0.tar.gz", "has_sig": false, "md5_digest": "0336ea64470295a57826c607b9ee0f7d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 35849, "upload_time": "2016-12-21T22:08:33", "url": "https://files.pythonhosted.org/packages/c9/9f/4c66a793ad66cc8b22a3541006dc95ce97d55364875c21f2127b38755da0/datafs-0.5.0.tar.gz" } ], "0.6.1": [ { "comment_text": "", "digests": { "md5": "9a56e9c8525365e106996aabcf39a663", "sha256": "735e7547123442511279484c65f8c3e3e7f38304c892cf2bdc9d2ed94f984722" }, "downloads": -1, "filename": "datafs-0.6.1.tar.gz", "has_sig": false, "md5_digest": "9a56e9c8525365e106996aabcf39a663", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 49288, "upload_time": "2017-01-06T02:21:48", "url": "https://files.pythonhosted.org/packages/91/cd/78ff7a3d52e2227dc482a320363fad455f8f28ba1be648491ccdd053324a/datafs-0.6.1.tar.gz" } ], "0.6.2": [ { "comment_text": "", "digests": { "md5": "4db459188bca79a9b65161bbd14f2f06", "sha256": "673ab34b4fc911cb3724dffe579a7d621de2880666d0f25559ae7b76f317f0a5" }, "downloads": -1, "filename": "datafs-0.6.2.tar.gz", "has_sig": false, "md5_digest": "4db459188bca79a9b65161bbd14f2f06", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 56382, "upload_time": "2017-01-09T20:19:01", "url": "https://files.pythonhosted.org/packages/60/84/088ccdcc7980ed04721e633735ded4abde6b25a7a86e215fa566888952e8/datafs-0.6.2.tar.gz" } ], "0.6.4": [ { "comment_text": "", "digests": { "md5": "716440c56e7008ca85f1c8784e62e742", "sha256": "15d6985bc5caef9bbc978e3918b28232c4eb144af21dd5cf9c7fa6a025938749" }, "downloads": -1, "filename": "datafs-0.6.4.tar.gz", "has_sig": false, "md5_digest": "716440c56e7008ca85f1c8784e62e742", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 59910, "upload_time": "2017-01-12T02:17:45", "url": "https://files.pythonhosted.org/packages/35/52/3d9e8c16573e0ab8125123ead6ba1e360c40133ac8cb99d9ed62b068618e/datafs-0.6.4.tar.gz" } ], "0.6.5": [ { "comment_text": "", "digests": { "md5": "00c3d98d178935de435da8376439458e", "sha256": "22c34204df0b0145db845224b48dcdee87a232465079619302351b8830b20e75" }, "downloads": -1, "filename": "datafs-0.6.5.tar.gz", "has_sig": false, "md5_digest": "00c3d98d178935de435da8376439458e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 63267, "upload_time": "2017-01-13T01:40:54", "url": "https://files.pythonhosted.org/packages/fd/e2/51d7f6e7b8ed508f101d59265da9799cce441a0847ee44ad1d8c5afc7f35/datafs-0.6.5.tar.gz" } ], "0.6.6": [ { "comment_text": "", "digests": { "md5": "d31b31dcbc2343c8c6e6911647471a71", "sha256": "4f98b3c3872ba7f1bf374faab6fdb342705e2a771b3d6e2336be705e1583730d" }, "downloads": -1, "filename": "datafs-0.6.6.tar.gz", "has_sig": false, "md5_digest": "d31b31dcbc2343c8c6e6911647471a71", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 65412, "upload_time": "2017-01-20T08:45:17", "url": "https://files.pythonhosted.org/packages/dd/fe/1e4be7e1d943edeb5f2303c5430eb248321c5434e3b493237e44c56c1225/datafs-0.6.6.tar.gz" } ], "0.6.7": [ { "comment_text": "", "digests": { "md5": "44f41e21f76f5f0b5649638fdbef0ce8", "sha256": "e4decb23c6a095d07c22f6cb845d18a5ceeb7c77b973f36fb3effec551b0de3c" }, "downloads": -1, "filename": "datafs-0.6.7.tar.gz", "has_sig": false, "md5_digest": "44f41e21f76f5f0b5649638fdbef0ce8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 65671, "upload_time": "2017-02-01T22:04:04", "url": "https://files.pythonhosted.org/packages/84/b0/5bc4ca999a22f45ff6b09db18b196f967ca262bf2ba41a7d02ff52a3e680/datafs-0.6.7.tar.gz" } ], "0.6.8": [ { "comment_text": "", "digests": { "md5": "b1d9cd6945192c9370d4678d3be33cd8", "sha256": "d29aefb4bdfca0f6f866f4108ce6587948ee24e6ffb79548cb8dbf5d34938d56" }, "downloads": -1, "filename": "datafs-0.6.8.tar.gz", "has_sig": false, "md5_digest": "b1d9cd6945192c9370d4678d3be33cd8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 68535, "upload_time": "2017-02-07T20:34:53", "url": "https://files.pythonhosted.org/packages/de/08/32f3d9216b04de6ef6372bebd2c4549b432977186047014ce41b188c88c5/datafs-0.6.8.tar.gz" } ], "0.6.9": [ { "comment_text": "", "digests": { "md5": "66876bbaa461cf7fa888968071222092", "sha256": "2457ddfb2cc48f1ddda8324c7b1eb6f4060a7836590cd15f85505da5ad36c722" }, "downloads": -1, "filename": "datafs-0.6.9.tar.gz", "has_sig": false, "md5_digest": "66876bbaa461cf7fa888968071222092", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 72707, "upload_time": "2017-02-21T18:50:54", "url": "https://files.pythonhosted.org/packages/d9/50/e13e563ef819dc70886a3bfda9c52f15ab678272ddbd85e490a08aa16407/datafs-0.6.9.tar.gz" } ], "0.7.0": [ { "comment_text": "", "digests": { "md5": "43a8c1b45cc19faa9d953485dd30008a", "sha256": "84f1bb60bb2b97a1617c1a8b8ef342b1e3e5f3acf16050cc973e2c17388d6334" }, "downloads": -1, "filename": "datafs-0.7.0.tar.gz", "has_sig": false, "md5_digest": "43a8c1b45cc19faa9d953485dd30008a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 75906, "upload_time": "2017-03-09T19:42:06", "url": "https://files.pythonhosted.org/packages/07/62/0ad0926af7024bea7858a002058b0b01ff438d3aabc2dc492cd9a3dfa404/datafs-0.7.0.tar.gz" } ], "0.7.1": [ { "comment_text": "", "digests": { "md5": "6fb9508da6aae95fb4f4e40060da8add", "sha256": "ce08427080dd8912f6c7125ca58da52479277eb763a0f2bcd17f58cb6864b17c" }, "downloads": -1, "filename": "datafs-0.7.1.tar.gz", "has_sig": false, "md5_digest": "6fb9508da6aae95fb4f4e40060da8add", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 83882, "upload_time": "2017-05-02T22:10:46", "url": "https://files.pythonhosted.org/packages/2e/a5/e90586a78675dd4a8dad69a183a2d6970a579033bf12dd901fb486688117/datafs-0.7.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "6fb9508da6aae95fb4f4e40060da8add", "sha256": "ce08427080dd8912f6c7125ca58da52479277eb763a0f2bcd17f58cb6864b17c" }, "downloads": -1, "filename": "datafs-0.7.1.tar.gz", "has_sig": false, "md5_digest": "6fb9508da6aae95fb4f4e40060da8add", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 83882, "upload_time": "2017-05-02T22:10:46", "url": "https://files.pythonhosted.org/packages/2e/a5/e90586a78675dd4a8dad69a183a2d6970a579033bf12dd901fb486688117/datafs-0.7.1.tar.gz" } ] }