{ "info": { "author": "Fabien C. Y. Benureau", "author_email": "fabien@benureau.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Science/Research", "License :: OSI Approved :: BSD License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Scientific/Engineering", "Topic :: Software Development :: Testing" ], "description": "# The Reproducible Python Library\n*Keep track of your results.*\n\nEver produced a result for a paper, only to realize a few months later that you\ncould not reproduce it? That you had no idea which version of the code, and\nwhich parameter values were used back then?\n\nThe reproducible library, developped by the [Cognitive Neuro-Robotics\nUnit](https://groups.oist.jp/cnru) at the [Okinawa Institute of Science and\nTechnology (OIST)](http://www.oist.jp/), aims to provide an easy way to gather and save\nimportant information about the context in which a result was computed. This\nincludes details about the OS, the Python version, the time, the git commit, the\ncommand-line arguments, hashes of input and output files, and any user provided\ndata.\n\nOther Python libraries doing just that exists such as\n[Recipy](https://github.com/recipy/recipy) and\n[Sumatra](http://neuralensemble.org/sumatra/). And they are good. Do try them.\nThey each have their own design philosophy, which proved to be difficult to\ninterface with some of the workflows of the Cognitive Neuro-Robotics\nUnit lab at OIST.\n\nWith [reproducible](https://github.com/oist-cnru/reproducible.git) the goal was\nto have a small non-intrusive library allowing precise control over the data\ncollected and how to output it. In particular, the goal was to have the tracking\ninfo sitting next to\u2014or better, directly embedded in\u2014the result files. That\nmakes sending results to collaborators or packaging them for publication\nstraightforward.\n\nThe reproducible library is licensed under the [LGPL version\n3](https://www.gnu.org/licenses/lgpl-3.0.md), to allow you to use it along-side\ncode that use other licenses.\n\nThe library is in beta; expect some changes. Python 3.5 or later is officially\nsupported, but [the code runs on 2.7 and 3.4 (but not on 3.3) as\nwell](https://travis-ci.org/oist-cnru/reproducible).\n\n\n## Install\n\n`pip install reproducible`\n\n\n## Instant Tutorial\n\nSay this is your code, which is fully committed using git:\n\n```python\nimport random\nimport pickle\n\ndef walk(n):\n \"\"\"A simple random walk generator\"\"\"\n steps = [0]\n for i in range(n):\n steps.append(steps[-1] + random.choice([-1, 1]))\n return steps\n\nif __name__ == '__main__':\n random.seed(1)\n results = walk(10)\n with open('results.pickle', 'wb') as f:\n pickle.dump(results, f)\n```\n\nTo add reproducible tracking:\n\n```python\nimport random\nimport pickle\n\nimport reproducible\n\ndef walk(n):\n \"\"\"A simple random walk generator\"\"\"\n steps = [0]\n for i in range(n):\n steps.append(steps[-1] + random.choice([-1, 1]))\n return steps\n\nif __name__ == '__main__':\n # create a reproducible.Context instance, that will hold all the\n # tracked data.\n context = reproducible.Context()\n\n # recording git repository state\n # here we are okay with running our code with uncommitted changes, but\n # we record a diff of the changes in the tracked data.\n context.add_repo(path='.', allow_dirty=True, diff=True)\n\n # recording parameters; this is not necessarily needed, as the code state\n # is recorded, but it is convenient.\n seed = 1\n random.seed(seed)\n context.add_data('seed', seed)\n\n # add_data return the provided value (here 10), so you can do this:\n n = reproducible.add_data('n', 10)\n results = walk(n)\n\n with open('results.pickle', 'wb') as f:\n pickle.dump(results, f)\n # recording the SHA1 hash of the output file\n context.add_file('results.pickle', category='output')\n\n # you can examine the tracked data and add or remove from it at any moment\n # with `context.data`: it is a simple dictionary. For instance, the\n # cpu info is quite detailed. Let's remove it to keep the yaml output short.\n context.data.pop('cpuinfo')\n\n # exporting the provenance data to disk\n context.export_yaml('results_prov.yaml')\n```\n\nThis is the resulting yaml file output containing the tracking data:\n```yaml\nargv: [example_after.py]\ndata: {n: 10, seed: 1}\nfiles:\n output:\n results.pickle: {mtime: 1531381834.0666547, sha256: 395d8846640c012e3e5c642e7737173a1a74120275b37fa2ded13a211df3264e}\npackages: [gitdb2==2.0.3, GitPython==2.1.10, pip==10.0.1, py-cpuinfo==4.0.0, PyYAML==4.2b4,\n reproducible==0.1.2, setuptools==39.0.1, smmap2==2.0.3]\nplatform: Darwin-17.6.0-x86_64-i386-64bit\npython:\n branch: ''\n compiler: Clang 9.1.0 (clang-902.0.39.2)\n implementation: CPython\n revision: ''\n version: ['3', '7', '0']\nrepositories:\n .: {diff: null, dirty: false, hash: 88c1de4ba5fb5cb2564b60245f26d3226ecb20c9, version: git\n version 2.18.0}\ntimestamp: ['2018-07-12T07:50:34.033829Z']\n```\n\nSee also the The [API Reference](https://reproducible.readthedocs.io/).\n\n## Roadmap\n\n- Retrieve GPU information.\n- More configurability.\n- Optionally capture input, output (`sys.stderr`, `sys.stdout`).\n- Easy disabling/reenabling of reproducible\n- optional SHA256 in the filename of external files\n\n## Changelog\n\n**version 0.3.0**, *20190703*\nThis version introduces API and logic-breaking changes.\n- `add_file()` overwrites by default now, and category is now an optional argument.\n- `context.data()` becomes `context.data`.\n- `Context(repo_path='.', allow_dirty=False, allow_untracked=False, diff=True, cpuinfo=True)`\n becomes `Context(cpuinfo=True, pip_packages=True)`: `add_repo()` needs to be called\n explicitly now, and pip_packages queries can be made optional.\n- `reset()` does not accept any arguments anymore; remembers `__init__()` argument values instead.\n- fixed missing `reproducible.add_pip_packages()`.\n\n**version 0.2.4**, *20170809*\n- hotfix for Python 2.7---because I am stupid.\n\n**version 0.2.3**, *20170809*\n- add `json()`, `yaml()` and `requirements()` function to access the result\n of export functions programmatically.\n- YAML output is now generated using `yaml.safe_dump` rather than `yaml.dump`.\n Leads to safer and simpler output.\n\n**version 0.2.2**, *20170717*\n- fix for deprecated `save_yaml()`, `save_json()` functions.\n\n**version 0.2.1**, *20170717*\n- add readme, license to pypi package.\n\n**version 0.2.0**, *20170717*\n- renamed `save_json()` and `save_yaml()` as `export_json()` and\n `export_yaml()`. The old name remain for now with a deprecation warning.\n- `Context` instances for more flexible, non-module level, behavior, much like\n the `Random` instances of the standard `random` module.\n- `reproducible.function_args()` function to retrieve arguments from inside a\n function.\n- `reproducible.reset()` function for clearing tracked data.\n- `reproducible.export_requirements()` to create requirements files from the\n retrieved list of installed packages.\n- Fix import of the freeze command from the `pip` package.\n- Updated readme: yaml output of the example, roadmap, changlog.\n\n**version 0.1.2**, *20170611*\n- Various bug fixes.\n- The `save_json()` and `save_yaml()` functions now return the SHA256 hash of the file they produce.\n\n**version 0.1.1**, *20170608*\n- `reproducible.data()` function to access and modify the collected data.\n- more unit tests\n\n**version 0.1.0**, *20170607*\n- first version: `add_repo()`, `add_file()`, `add_data()`, `add_random_state()`, `git_info()`, `git_dirty()`, `save_json()`, `save_yaml()` functions.", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/oist-cnru/reproducible", "keywords": "replicable reproducible research", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "reproducible", "package_url": "https://pypi.org/project/reproducible/", "platform": "", "project_url": "https://pypi.org/project/reproducible/", "project_urls": { "Homepage": "https://github.com/oist-cnru/reproducible" }, "release_url": "https://pypi.org/project/reproducible/0.3.0/", "requires_dist": null, "requires_python": "", "summary": "Reproducible library", "version": "0.3.0" }, "last_serial": 5479494, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "27441df3b1654e52667d183a3e3ebf3c", "sha256": "f05d7132bdf2afe8bc935d4e516f7b81c1bc9b79d27e5da4d3c26a5e442adb44" }, "downloads": -1, "filename": "reproducible-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "27441df3b1654e52667d183a3e3ebf3c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7117, "upload_time": "2018-06-08T02:22:57", "url": "https://files.pythonhosted.org/packages/94/42/37f274a5c047e874adfeb7aae2c1a95583acd5294b2685243ac2c98f1ccf/reproducible-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "06c22065301ece188081a3b731b4adcd", "sha256": "cc377a41ccf21e2774a34f40e69580f742222350fa47c62a30c4d0e7e108070b" }, "downloads": -1, "filename": "reproducible-0.1.0.tar.gz", "has_sig": false, "md5_digest": "06c22065301ece188081a3b731b4adcd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6693, "upload_time": "2018-06-08T02:22:58", "url": "https://files.pythonhosted.org/packages/15/c8/2cbd512dc6de71af943c76552c8aac571eddc448b53892e5b89534fba6b3/reproducible-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "8485c717582ca072223e25fa6a03895c", "sha256": "be865cc4d77e00c4b06128878af0049c96a5be2e40a023913739754ebb379c6e" }, "downloads": -1, "filename": "reproducible-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "8485c717582ca072223e25fa6a03895c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7340, "upload_time": "2018-06-08T08:26:43", "url": "https://files.pythonhosted.org/packages/14/b8/1793eb3f9f7018d285ecb1d4712b4f2241e38372938d2ef19c6be9932652/reproducible-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c0b5b658df6bbcd6991c7a03edfcf666", "sha256": "d3e96134269d2b46f0343998e7740988525d684dc040c053b9a4c722cef56fb9" }, "downloads": -1, "filename": "reproducible-0.1.1.tar.gz", "has_sig": false, "md5_digest": "c0b5b658df6bbcd6991c7a03edfcf666", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6916, "upload_time": "2018-06-08T08:26:45", "url": "https://files.pythonhosted.org/packages/ec/b1/e93b7920f45d72b06f52d9a4fde2c2a5859cb9fa3632be40d4152e1c3182/reproducible-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "be20f5fe7554c7cbd848d6c16b1a8835", "sha256": "9b54881fa12a99cc2630f868107325cd2ef34ca0aaf45292b7e14449cd42c9f8" }, "downloads": -1, "filename": "reproducible-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "be20f5fe7554c7cbd848d6c16b1a8835", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7385, "upload_time": "2018-06-11T02:11:41", "url": "https://files.pythonhosted.org/packages/f3/00/343ede3fe4b0673512f066489eb4b72faa12efb411db127b5b6dee5f8444/reproducible-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "87c5a2624c6af4389700bf07aaba4024", "sha256": "de53cc4485b037a2f1426a037be9291fe8d1cf28339c491950e3e36dbfb8c824" }, "downloads": -1, "filename": "reproducible-0.1.2.tar.gz", "has_sig": false, "md5_digest": "87c5a2624c6af4389700bf07aaba4024", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6953, "upload_time": "2018-06-11T02:11:43", "url": "https://files.pythonhosted.org/packages/6a/90/80b74fa5a14f8e14e80d0fc5bcb66e7431159ddc2e0e129c261184c071a4/reproducible-0.1.2.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "8be9826c6f237795f504291e1c8f4426", "sha256": "0e92c0b27a3ce8cbb3c238bc2eed701adfde56196a6037ba781ffeb4fa751c3a" }, "downloads": -1, "filename": "reproducible-0.2.0.tar.gz", "has_sig": false, "md5_digest": "8be9826c6f237795f504291e1c8f4426", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8750, "upload_time": "2018-07-17T12:22:59", "url": "https://files.pythonhosted.org/packages/1b/e1/37dec8be6865fc7bc4cabf3c27f8d2d739579a70a5d13d6cf466e0831881/reproducible-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "c21788a296386c085598ec02aa535ded", "sha256": "3a1a0e9dbaf1ab1eaea9a8a043471af94dae34ab836561a9e677ddf5ff20a215" }, "downloads": -1, "filename": "reproducible-0.2.1.tar.gz", "has_sig": false, "md5_digest": "c21788a296386c085598ec02aa535ded", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11806, "upload_time": "2018-07-17T12:42:47", "url": "https://files.pythonhosted.org/packages/ae/a3/24a6c30890f862280b7864ca1a5eb5102f91b75d21b2b0ad060716d1fa5b/reproducible-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "843fc450d205bf7972a9cf3fece8e0da", "sha256": "ffd22a18dac9eb4387f4f90d8041423a0edd1e0cdcfeaf73f197cb4eaf5fbe30" }, "downloads": -1, "filename": "reproducible-0.2.2.tar.gz", "has_sig": false, "md5_digest": "843fc450d205bf7972a9cf3fece8e0da", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11814, "upload_time": "2018-07-19T05:05:58", "url": "https://files.pythonhosted.org/packages/a5/98/2b053626b5f37a49f0813fa0958967a308ca812e6d071c5bd7b1603c5166/reproducible-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "bed55519d2a94a38b296ee5cc969547b", "sha256": "8fad0c779cfd47999a4c49eb598a2484ed364b01b5464d64fcca366962d73f49" }, "downloads": -1, "filename": "reproducible-0.2.3-py3-none-any.whl", "has_sig": false, "md5_digest": "bed55519d2a94a38b296ee5cc969547b", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 15664, "upload_time": "2018-08-09T10:12:47", "url": "https://files.pythonhosted.org/packages/f5/5d/eb55ccdad052712b8d8cbbd4587467bb0ef2ffbad8a62f68789d44f88695/reproducible-0.2.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2c8578b4c48d971e4f7a1c9b32c5dc41", "sha256": "4db31ec298de64cd4276f409f1d6d85ba6c0057e7a86ebb5ebd1cdb2c16d71f9" }, "downloads": -1, "filename": "reproducible-0.2.3.tar.gz", "has_sig": false, "md5_digest": "2c8578b4c48d971e4f7a1c9b32c5dc41", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12243, "upload_time": "2018-08-09T10:11:04", "url": "https://files.pythonhosted.org/packages/01/64/3b796ce1e113f0ad109e004130a45829d49c3a91f67097090eed5d7d6833/reproducible-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "9d7ff94038ae41915c5fdb654325c40f", "sha256": "a7c87a1cdf74fbc32d37fcc7c994731f745adfded6ba90268e7297b99998f1e0" }, "downloads": -1, "filename": "reproducible-0.2.4-py3-none-any.whl", "has_sig": false, "md5_digest": "9d7ff94038ae41915c5fdb654325c40f", "packagetype": "bdist_wheel", "python_version": "3.6", "requires_python": null, "size": 15826, "upload_time": "2018-08-09T13:35:26", "url": "https://files.pythonhosted.org/packages/42/ac/6d863ac5efc49bfb7fba851c18f206aa957933484d4812d41f6a9ba37e29/reproducible-0.2.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e9499e595b9f9676b5147ed2f5032fd7", "sha256": "033a6daf51e46f32b7cfcecf1bfadb4ed323dfeae121cfd4ad3b4e80589d7b0a" }, "downloads": -1, "filename": "reproducible-0.2.4.tar.gz", "has_sig": false, "md5_digest": "e9499e595b9f9676b5147ed2f5032fd7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12334, "upload_time": "2018-08-09T13:35:22", "url": "https://files.pythonhosted.org/packages/e1/f4/635c1300e8128ebe2f7514049053ddde17893b530508cd38cee744a67291/reproducible-0.2.4.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "4ddd980ec4b103eda055dca863e88540", "sha256": "33cdef7da5fc46f42f89f87b444e18f41a5c50ef55938dceecbe308eb8a46f90" }, "downloads": -1, "filename": "reproducible-0.3.0.tar.gz", "has_sig": false, "md5_digest": "4ddd980ec4b103eda055dca863e88540", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13712, "upload_time": "2019-07-03T04:54:45", "url": "https://files.pythonhosted.org/packages/1d/28/3cbd3757e8e4fd8aeee6d624102bf7e1dd0e2846a1d5c8b0da4d1fb978a7/reproducible-0.3.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "4ddd980ec4b103eda055dca863e88540", "sha256": "33cdef7da5fc46f42f89f87b444e18f41a5c50ef55938dceecbe308eb8a46f90" }, "downloads": -1, "filename": "reproducible-0.3.0.tar.gz", "has_sig": false, "md5_digest": "4ddd980ec4b103eda055dca863e88540", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13712, "upload_time": "2019-07-03T04:54:45", "url": "https://files.pythonhosted.org/packages/1d/28/3cbd3757e8e4fd8aeee6d624102bf7e1dd0e2846a1d5c8b0da4d1fb978a7/reproducible-0.3.0.tar.gz" } ] }