{ "info": { "author": "Ben Mabey", "author_email": "ben@benmabey.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License" ], "description": "==========\nprovenance\n==========\n\n|version status| |build status| |docs|\n\n\n.. |version status| image:: https://img.shields.io/pypi/v/provenance.svg\n :target: https://pypi.python.org/pypi/provenance\n :alt: Version Status\n.. |build status| image:: https://travis-ci.org/bmabey/provenance.png?branch=master\n :target: https://travis-ci.org/bmabey/provenance\n :alt: Build Status\n.. |docs| image:: https://readthedocs.org/projects/provenance/badge/?version=latest\n :target: https://provenance.readthedocs.org\n :alt: Documentation Status\n\n``provenance`` is a Python library for function-level caching and provenance that aids in\ncreating Parsimonious Pythonic |Pipelines|. By wrapping functions in the ``provenance``\ndecorator computed results are cached across various tiered stores (disk, S3, SFTP) and\n`provenance `_ (i.e. lineage) information is tracked\nand stored in an artifact repository. A central artifact repository can be used to enable\nproduction pipelines, team collaboration, and reproducible results. The library is general\npurpose but was built with machine learning pipelines in mind. By leveraging the fantastic\n`joblib`_ library object serialization is optimized for ``numpy`` and other PyData libraries.\n\nWhat that means in practice is that you can easily keep track of how artifacts (models,\nfeatures, or any object or file) are created, where they are used, and have a central place\nto store and share these artifacts. This basic plumbing is required (or at least desired!)\nin any machine learning pipeline and project. ``provenance`` can be used standalone along with\na build server to run pipelines or in conjunction with more advanced workflow systems\n(e.g. `Airflow`_, `Luigi`_).\n\n.. |Pipelines| unicode:: Pipelines U+2122\n.. _joblib: https://pythonhosted.org/joblib/\n.. _Airflow: http://airbnb.io/projects/airflow/\n.. _Luigi: https://github.com/spotify/luigi\n\nExample\n=======\n\nFor an explanation of this example please see the `Introductory Guide`_.\n\n.. code-block:: python\n\n import provenance as p\n\n p.load_config(...)\n\n import time\n\n @p.provenance()\n def expensive_add(a, b):\n time.sleep(2)\n return a + b\n\n\n @p.provenance()\n def expensive_mult(a, b):\n time.sleep(2)\n return a * b\n\n\n a1 = expensive_add(4, 3)\n a2 = expensive_add(1, 1)\n\n result = expensive_mult(a1, a2)\n\n vis.visualize_lineage(result)\n\n\n.. image:: https://raw.githubusercontent.com/bmabey/provenance/master/docs/source/images/lineage_example.png\n\n\n.. _Introductory Guide: http://provenance.readthedocs.io/en/latest/intro-guide.html\n\nInstallation\n============\n\nFor the base functionality:\n\n.. code:: bash\n\n pip install provenance\n\n\nFor the visualization module (which requires ``graphviz`` to be installed):\n\n.. code:: bash\n\n pip install provenance[vis]\n\nFor the SFTP store:\n\n.. code:: bash\n\n pip install provenance[sftp]\n\nFor everything all at once:\n\n\n.. code:: bash\n\n pip install provenance[all]\n\n\n\nCompatibility\n============\n\n``provenance`` is currently only compatible with Python 3.5 and higher. Updating it to work with Python 2.7x\nshould be easy, follow this `ticket`_ if you are interested in that.\n\n\n.. _ticket: https://github.com/bmabey/provenance/issues/32\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/bmabey/provenance", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "provenance", "package_url": "https://pypi.org/project/provenance/", "platform": "", "project_url": "https://pypi.org/project/provenance/", "project_urls": { "Homepage": "http://github.com/bmabey/provenance" }, "release_url": "https://pypi.org/project/provenance/0.12.0/", "requires_dist": [ "alembic (>=0.9.1)", "boltons (>=16.5.1)", "cloudpickle (>=0.2.1)", "joblib (>=0.10.2)", "memoized-property (>=1.0.2)", "numpy", "ordered-set (>=2.0.1)", "psutil (>=5.0.0)", "psycopg2", "s3fs (>=0.0.8)", "sqlalchemy (>=1.1.3)", "sqlalchemy-utils (>=0.32.12)", "toolz (>=0.8.2)", "wrapt (>=1.10.8)", "google-cloud; extra == 'google_storage'", "paramiko; extra == 'sftp'", "frozendict; extra == 'vis'", "graphviz; extra == 'vis'" ], "requires_python": "", "summary": "Provenance and caching library for functions, built for creating lightweight machine learning pipelines.", "version": "0.12.0" }, "last_serial": 4353652, "releases": { "0.10.0": [ { "comment_text": "", "digests": { "md5": "602d8a7e450e82702133f2aa0f6a0fc1", "sha256": "06ad0b0cd0edb3c29a00608a1a69eae7b04c7d657cd14b36ae66e5477825267a" }, "downloads": -1, "filename": "provenance-0.10.0-py3-none-any.whl", "has_sig": false, "md5_digest": "602d8a7e450e82702133f2aa0f6a0fc1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 47685, "upload_time": "2017-05-01T02:53:24", "url": "https://files.pythonhosted.org/packages/36/07/f8d93638dccd06f3025a97b8f1254e7fc5ebcd0a570eb42a04b7ca6d23f6/provenance-0.10.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f50096f84b9bab4e65a9c10d2a4e76b6", "sha256": "0f2cb7c5008624a97148bde07e61da54159ac0748a3db1442788724d6dd0c3ed" }, "downloads": -1, "filename": "provenance-0.10.0.tar.gz", "has_sig": false, "md5_digest": "f50096f84b9bab4e65a9c10d2a4e76b6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 59468, "upload_time": "2017-05-01T02:53:26", "url": "https://files.pythonhosted.org/packages/03/42/6eb8426c331a704557559260bdfdc2a1e42c2d7f8b0f745beb6255606b05/provenance-0.10.0.tar.gz" } ], "0.11.0": [ { "comment_text": "", "digests": { "md5": "3179e8bb9968c23282965cc7d7b6b9a8", "sha256": "6b7875bae7b87465f6f4b605077976dcc8b2b5b7767d742323dba0a0a7bc3d42" }, "downloads": -1, "filename": "provenance-0.11.0-py3-none-any.whl", "has_sig": false, "md5_digest": "3179e8bb9968c23282965cc7d7b6b9a8", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 58327, "upload_time": "2018-08-23T19:52:32", "url": "https://files.pythonhosted.org/packages/52/5a/7d190b6c79845f2905719e666ae15495e0478b22a85f9d764383422adb49/provenance-0.11.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ac2cab15d1f6f7200d2820944ce03c10", "sha256": "714726dea4f454824c4a27363eb469f2e3b46faff1d2769a5c860940334806a0" }, "downloads": -1, "filename": "provenance-0.11.0.tar.gz", "has_sig": false, "md5_digest": "ac2cab15d1f6f7200d2820944ce03c10", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 68764, "upload_time": "2018-08-23T19:52:34", "url": "https://files.pythonhosted.org/packages/c5/3c/8ced4f87bbde7d66237fde965a3c69e7bfaa19daba550121f3394f0b2cf2/provenance-0.11.0.tar.gz" } ], "0.12.0": [ { "comment_text": "", "digests": { "md5": "d9f5f83c31e313e8064e42592290f0b0", "sha256": "86943280d2497a75b8e0430215e82405e28e2108101fb09c2769df99fa01aed0" }, "downloads": -1, "filename": "provenance-0.12.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d9f5f83c31e313e8064e42592290f0b0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 50627, "upload_time": "2018-10-08T21:43:20", "url": "https://files.pythonhosted.org/packages/e6/50/fc2ab6431a2c4806d88c079d76206fe0515e1609f5590970ae4f8b95ce9e/provenance-0.12.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b75ff529032a5dcbdeef613f8612b702", "sha256": "2b970a05884b1d061fe89051160a73f286640957ec572f2c2d5837172c75ae41" }, "downloads": -1, "filename": "provenance-0.12.0.tar.gz", "has_sig": false, "md5_digest": "b75ff529032a5dcbdeef613f8612b702", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 61817, "upload_time": "2018-10-08T21:43:22", "url": "https://files.pythonhosted.org/packages/71/3a/765002b614c4eeec4e9905098041eaae6080599d4576fb1199d1d756ae85/provenance-0.12.0.tar.gz" } ], "0.9.0": [ { "comment_text": "", "digests": { "md5": "728b35e1812aa3ff8980d051dbe7797c", "sha256": "71bf7f742fb700e5851fbf4b4cd4c714fe485d4deb4cb839c6853ac1eff77520" }, "downloads": -1, "filename": "provenance-0.9.0-py3-none-any.whl", "has_sig": false, "md5_digest": "728b35e1812aa3ff8980d051dbe7797c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 26615, "upload_time": "2017-01-07T00:53:34", "url": "https://files.pythonhosted.org/packages/33/f2/dbdb2c39bedb7109ea999ae765a9c13770d88730a53f54c0c66b7e11a71f/provenance-0.9.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "70ca7c9a5df4e300342d016a6f873992", "sha256": "50d5f09449a8cc96c8cbd6f49769f8bede3ab507efcbed446a967627163f14e4" }, "downloads": -1, "filename": "provenance-0.9.0.tar.gz", "has_sig": false, "md5_digest": "70ca7c9a5df4e300342d016a6f873992", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 38766, "upload_time": "2017-01-07T00:53:36", "url": "https://files.pythonhosted.org/packages/e0/eb/2668ba2827d900cd7d850e7de996e5951a15cf5cae617a2baef687a77722/provenance-0.9.0.tar.gz" } ], "0.9.1": [ { "comment_text": "", "digests": { "md5": "b64a3645cc7355841f5d8f0931ab1b53", "sha256": "3b86831f4eefcb296fa1da7c20dfa402bf59752a7596289426d937f29189d384" }, "downloads": -1, "filename": "provenance-0.9.1.tar.gz", "has_sig": false, "md5_digest": "b64a3645cc7355841f5d8f0931ab1b53", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 41785, "upload_time": "2017-02-16T21:16:48", "url": "https://files.pythonhosted.org/packages/11/a4/a42a9ed392c4b7ea0e7358e9d60a40f967ddfb6eadb83d6c5aee22d041d1/provenance-0.9.1.tar.gz" } ], "0.9.2": [ { "comment_text": "", "digests": { "md5": "53860aa0db4c0e54ede3b551ba328ac7", "sha256": "395084307d71d4f56aaf0be42a0d0f510f21b37161e409a89eab27d3aaf9143f" }, "downloads": -1, "filename": "provenance-0.9.2.tar.gz", "has_sig": false, "md5_digest": "53860aa0db4c0e54ede3b551ba328ac7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42016, "upload_time": "2017-02-17T21:40:18", "url": "https://files.pythonhosted.org/packages/41/f3/638c220f846794a811211ce2d61227c4f49dba8e4ec3bbfd80aaf318fd16/provenance-0.9.2.tar.gz" } ], "0.9.3": [ { "comment_text": "", "digests": { "md5": "78e7cd419ab7c1bf0fcd3be5b227c215", "sha256": "243fff9c991459551d650761742798c3d7f83fbb29ca31ad9905600fd8af336e" }, "downloads": -1, "filename": "provenance-0.9.3-py3-none-any.whl", "has_sig": false, "md5_digest": "78e7cd419ab7c1bf0fcd3be5b227c215", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 30115, "upload_time": "2017-02-17T21:55:04", "url": "https://files.pythonhosted.org/packages/08/10/1878edee8a94db572b476d709a668dc4988443fe6dce50f01849a2867678/provenance-0.9.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5cd6af82f2c13a5ba74645b61a2b4222", "sha256": "eaebf224d1330830fb24aec35f611349732080d6b661e7fed30a78763d8f0c0a" }, "downloads": -1, "filename": "provenance-0.9.3.tar.gz", "has_sig": false, "md5_digest": "5cd6af82f2c13a5ba74645b61a2b4222", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 42007, "upload_time": "2017-02-17T21:55:06", "url": "https://files.pythonhosted.org/packages/e0/06/e045bf5078a1b37839b33ccf861b3eab4178d614d27e3acd7884956f9646/provenance-0.9.3.tar.gz" } ], "0.9.4": [ { "comment_text": "", "digests": { "md5": "958b3771d3c9c00e8989bcef0bccf452", "sha256": "a7edd56bc6e5a24c1cc06471e59cb973555da8388b4510972abc62802b738189" }, "downloads": -1, "filename": "provenance-0.9.4-py3-none-any.whl", "has_sig": false, "md5_digest": "958b3771d3c9c00e8989bcef0bccf452", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36221, "upload_time": "2017-03-22T16:55:29", "url": "https://files.pythonhosted.org/packages/1f/5e/8cba88120d622ffa5a90ffb7b0c33ef798b10e3699e66baadc1ed4b4bcba/provenance-0.9.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1a735d49329f6c61b6834adb7867fb4a", "sha256": "08f58534773857b432e9c606e56b7a7d68bbdde749615279b785611241af2db4" }, "downloads": -1, "filename": "provenance-0.9.4.tar.gz", "has_sig": false, "md5_digest": "1a735d49329f6c61b6834adb7867fb4a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 46770, "upload_time": "2017-03-22T16:55:30", "url": "https://files.pythonhosted.org/packages/c4/3e/d8edb7645d8bfb761d5c0a97770410414aa12d67dca8d4606edce27b2e92/provenance-0.9.4.tar.gz" } ], "0.9.4.1": [ { "comment_text": "", "digests": { "md5": "020017363fbfa3eb7771c6c0df77ea84", "sha256": "1210d0fc0f0cdb413e62481378bd92391597097b3ad61bf9bfaf920ddb634c9d" }, "downloads": -1, "filename": "provenance-0.9.4.1-py3-none-any.whl", "has_sig": false, "md5_digest": "020017363fbfa3eb7771c6c0df77ea84", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36220, "upload_time": "2017-03-22T17:51:07", "url": "https://files.pythonhosted.org/packages/1f/ac/4e48278593cd8ff243a53f7e70a810e275696f3d605027621f28363f86b2/provenance-0.9.4.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4c755508b766c373148b0b15d7a5d9fa", "sha256": "0c87e99ec5e861e6e3034d7e6848a5de626a180168abe641c95182bf7ee04209" }, "downloads": -1, "filename": "provenance-0.9.4.1.tar.gz", "has_sig": false, "md5_digest": "4c755508b766c373148b0b15d7a5d9fa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 46765, "upload_time": "2017-03-22T17:51:08", "url": "https://files.pythonhosted.org/packages/3e/98/fd6cacf5ad44b441a7981e6748d24d249f80aa903f8732e31acaa89ed9fa/provenance-0.9.4.1.tar.gz" } ], "0.9.4.2": [ { "comment_text": "", "digests": { "md5": "133d45bbf498ecef46ef0eb6b5c82a33", "sha256": "bbd6616b3394d8d4df9052c01377a75df6d3868432dac609e0159d43225d8170" }, "downloads": -1, "filename": "provenance-0.9.4.2-py3-none-any.whl", "has_sig": false, "md5_digest": "133d45bbf498ecef46ef0eb6b5c82a33", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 36325, "upload_time": "2017-03-23T16:36:59", "url": "https://files.pythonhosted.org/packages/0f/df/7dfde89d900ea700125a4e061acbb28f9938c4c28d52a5bd25989bf63634/provenance-0.9.4.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f11a5a30337115015f7b41696b62be56", "sha256": "67ea640ed6bbd5b5f71fc20a8abb766c1cb106575d888a27fd232181922c399b" }, "downloads": -1, "filename": "provenance-0.9.4.2.tar.gz", "has_sig": false, "md5_digest": "f11a5a30337115015f7b41696b62be56", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 46893, "upload_time": "2017-03-23T16:37:02", "url": "https://files.pythonhosted.org/packages/2f/f1/536e116a887047623bcf21f9956e74dcd4f5e40a46800d91521d84da4d98/provenance-0.9.4.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d9f5f83c31e313e8064e42592290f0b0", "sha256": "86943280d2497a75b8e0430215e82405e28e2108101fb09c2769df99fa01aed0" }, "downloads": -1, "filename": "provenance-0.12.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d9f5f83c31e313e8064e42592290f0b0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 50627, "upload_time": "2018-10-08T21:43:20", "url": "https://files.pythonhosted.org/packages/e6/50/fc2ab6431a2c4806d88c079d76206fe0515e1609f5590970ae4f8b95ce9e/provenance-0.12.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b75ff529032a5dcbdeef613f8612b702", "sha256": "2b970a05884b1d061fe89051160a73f286640957ec572f2c2d5837172c75ae41" }, "downloads": -1, "filename": "provenance-0.12.0.tar.gz", "has_sig": false, "md5_digest": "b75ff529032a5dcbdeef613f8612b702", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 61817, "upload_time": "2018-10-08T21:43:22", "url": "https://files.pythonhosted.org/packages/71/3a/765002b614c4eeec4e9905098041eaae6080599d4576fb1199d1d756ae85/provenance-0.12.0.tar.gz" } ] }