{ "info": { "author": "Thomas Antony", "author_email": "tantony.purdue@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Software Development :: Libraries" ], "description": "# Overview\n\n**simplepipe** is a simple composable, functional pipelining library for Python. It was built to facilitate the composition of small tasks, defined as pure functions, in order to perform a complex operation. It supports single and multi-output tasks (via generator functions). **simplepipe** also allows creation of hooks that can modify the behavior of the workflow after it has been created.\n\n[![Build Status](https://travis-ci.org/thomasantony/simplepipe.svg?branch=master)](https://travis-ci.org/thomasantony/simplepipe)\n[![PyPI version](https://badge.fury.io/py/simplepipe.svg)](https://badge.fury.io/py/simplepipe)\n# Installation\n\nThe following command will install the package in your python environment from PyPI.\n\n pip install simplepipe\n\nIf you want install from the source code instead, run\n\n python setup.py install\n\n# Examples\n**simplepipe** allows you to define a list of tasks executed in a sequence that\nuses data in a workspace and returns a new, updated workspace. Each task can be\npython function, generator function or another Workflow object. The original workspace is unaffected. Method calls to `add_task`, `add_hook`, and `add_hook_point` can be chained.\n\n\n## Full-workspace tasks and composed workflows\n\nThis is the default mode for tasks if no input or output spec is given.\nThese functions must return a dict with result that will be used to update the workspace.\nWorkflow objects can themselves be used as full-workspace tasks. These functions should return\na dict that will be used with the 'update()' method on the workspace dict.\n\nsimplepipe makes sure that the input workspace dict is protected from mutations\nfrom these tasks and only updates the workspace with the returned value\n\n```python\nimport simplepipe\n\ndef do_stuff_with_workspace(workspace):\n workspace['c'] = workspace['b']*2\n return workspace\n\nwf = simplepipe.Workflow()\ndata_in = {'a': 1, 'b': 2}\nwf.add_task(do_stuff_with_workspace) # '*' is default mode\noutput = wf(data_in)\nprint(output) # Prints {'a': 1, 'b': 2, 'c': 4}\n\nwf2 = simplepipe.Workflow()\nwf2.add_task(wf) # Add another workflow as a task\nwf2.add_task(fn= lambda c: 5*c, inputs='c', outputs='d')\noutput = wf(data_in)\nprint(output) # Prints {'a': 1, 'b': 2, 'c': 4, 'd': 20}\n\n# Protection against mutator functions\ndef bad_mutator_fn(workspace):\n workspace['a'] = 'just_messing_with_a'\n return {'e': 'foobar'}\n\nwf3 = simplepipe.Workflow()\nwf3.add_task(fn=bad_mutator_fn)\noutput = wf(data_in)\nprint(output) # Prints {'a': 1, 'b': 2, 'e': 'foobar'}\n```\n\n## Single output functions\n\n```python\nimport simplepipe\n\ndef sum(a, b):\n return a+b\n\ndef twice(x):\n return 2*x\n\nwf = simplepipe.Workflow()\ndata_in = {'a': 1, 'b': 2}\nwf.add_task(sum, inputs=['a', 'b'], outputs=['c']) \\\n .add_task(twice, inputs=['c'], outputs=['d'])\noutput = wf(data_in)\nprint(output) # Prints {'a': 1, 'b': 2, 'c': 3, 'd': 6}\n```\n\n## Multi-output functions\n\nFunctions returning multiple values must use the `yield` keyword to return them\nseparately, one at a time.\n\n```python\nimport simplepipe\n\ndef sum_and_product(a, b):\n yield a+b\n yield a*b\n\nwf = simplepipe.Workflow()\ndata_in = {'a': 1, 'b': 2}\nwf.add_task(sum_and_product, inputs=['a', 'b'], outputs=['c', 'd'])\noutput = wf(data_in)\nprint(output) # Prints {'a': 1, 'b': 2, 'c': 3, 'd': 2}\n```\n\n## Hooks\n**simplepipe** also supports hooks that allow customization of the workflow after it has been created. Hook points are defined using the `add_hook_point` method. Any number of hook functions can be bound to the hook points in the work flow. Multiple hooks added at the same hook point will be executed in the order that they were added.\n\n*Note: Hook functions are not pure functions and are supposed to mutate the output workspace. They do not return anything.*\n\n```python\nimport simplepipe\n\ndef sum(a, b):\n return a+b\n\ndef twice(x):\n return 2*x\n\ndef do_after_sum(workspace):\n workspace['c'] = workspace['c']*10\n\ndef do_after_twice(workspace):\n workspace['e'] = 31337\n\n\nwf = simplepipe.Workflow()\ndata_in = {'a': 1, 'b': 2}\nwf.add_task(sum, inputs=['a', 'b'], outputs=['c'])\nwf.add_hook_point('after_sum')\nwf.add_task(twice, inputs=['c'], outputs=['d'])\nwf.add_hook_point('after_twice')\n\n# Hook functions can be inserted any time before the workflow is executed\nwf.add_hook('after_sum', do_after_sum)\nwf.add_hook('after_twice', do_after_twice)\n\noutput = wf(data_in)\nprint(output)\n# {'a': 1, 'b': 2, 'c': 30, 'd': 60, 'e': 31337}\n```\n\n#About the Author\n[Thomas Antony's LinkedIn Profile](https://www.linkedin.com/in/thomasantony)\n\n\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/thomasantony/simplepipe", "keywords": "pipeline,functional,functional programming", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "simplepipe", "package_url": "https://pypi.org/project/simplepipe/", "platform": "", "project_url": "https://pypi.org/project/simplepipe/", "project_urls": { "Homepage": "https://github.com/thomasantony/simplepipe" }, "release_url": "https://pypi.org/project/simplepipe/0.0.5.3/", "requires_dist": null, "requires_python": "", "summary": "A simple functional pipelining library for Python.", "version": "0.0.5.3" }, "last_serial": 2807653, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "06cbbcf202ef4aae82b68fd8180d0d76", "sha256": "c8971841b570939179fc9402fad0713eba2ef576c62ef783dd50e4d1748db532" }, "downloads": -1, "filename": "simplepipe-0.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "06cbbcf202ef4aae82b68fd8180d0d76", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6044, "upload_time": "2016-07-25T23:17:59", "url": "https://files.pythonhosted.org/packages/b6/24/d946d495088ed80a4a2c2c0d6a9220a3da9b8c91c39cb0476acaf2f06938/simplepipe-0.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "53d699d10684774340a1c463c7910ce6", "sha256": "ccbf2cb4b827321d56b025ea04ddfa4b64fc88bedea306bfeba5e6f77ff0c3df" }, "downloads": -1, "filename": "simplepipe-0.0.1.tar.gz", "has_sig": false, "md5_digest": "53d699d10684774340a1c463c7910ce6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3544, "upload_time": "2016-07-25T23:18:01", "url": "https://files.pythonhosted.org/packages/85/3c/fa29c3f034759d060fd5993a304381b37c14348e44b462bf23c0e657b24d/simplepipe-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "9b96d104078e6533e74b311afca7fa8e", "sha256": "cf56e4b54e6e8aeac1fa056d732b3a4351a5ee5a00b2cb7cb55a6255d2e9f6e2" }, "downloads": -1, "filename": "simplepipe-0.0.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9b96d104078e6533e74b311afca7fa8e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7541, "upload_time": "2016-07-31T21:02:41", "url": "https://files.pythonhosted.org/packages/4a/01/605067c43127ece1d64b3e296be28ea13035a4a98780e94cb4006ae197c5/simplepipe-0.0.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "53f9f076c9bb1787771d3bfa9921eab6", "sha256": "fa98249da615ae8b65e859294680a3063509636a9f2bb5158f08921bda6bb68f" }, "downloads": -1, "filename": "simplepipe-0.0.2.tar.gz", "has_sig": false, "md5_digest": "53f9f076c9bb1787771d3bfa9921eab6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4500, "upload_time": "2016-07-31T21:02:43", "url": "https://files.pythonhosted.org/packages/fe/b3/dd638a985a65399895f986fcd226f10af7512fdb8ec4ebb36b7d93185b05/simplepipe-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "a6cae71eefb026f99616c4576966e3ec", "sha256": "f366b75dc70fcdb2f8bdd89bf16a10f858b95e126f3e069a249189eeb952feec" }, "downloads": -1, "filename": "simplepipe-0.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a6cae71eefb026f99616c4576966e3ec", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7325, "upload_time": "2017-03-03T17:44:38", "url": "https://files.pythonhosted.org/packages/81/84/af2281eb2c4be4e90d29dc223841e5273a37021531079a28bb658183e3ef/simplepipe-0.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "13ebf5e4b2db111144c809df7bb9fc46", "sha256": "a1ae32ad6a31513e8f0476c30bd401d0999c45ff969a6cc3bd4c5c1a1103dec6" }, "downloads": -1, "filename": "simplepipe-0.0.3-py3.5.egg", "has_sig": false, "md5_digest": "13ebf5e4b2db111144c809df7bb9fc46", "packagetype": "bdist_egg", "python_version": "3.5", "requires_python": null, "size": 7376, "upload_time": "2017-03-03T17:44:39", "url": "https://files.pythonhosted.org/packages/b1/7d/eac30b6bf4ce39d0737beb05daa89c81d31f4903ef828a67c7bddf27be9f/simplepipe-0.0.3-py3.5.egg" }, { "comment_text": "", "digests": { "md5": "9f27b35b7c314e4387870b98d7edd628", "sha256": "a7cad0a32c880fb366e6d4afc983981d0ead8520b482dd5b5169eaf6887b58ff" }, "downloads": -1, "filename": "simplepipe-0.0.3.tar.gz", "has_sig": false, "md5_digest": "9f27b35b7c314e4387870b98d7edd628", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4612, "upload_time": "2017-03-03T17:44:41", "url": "https://files.pythonhosted.org/packages/fd/69/ecdb02291e9de2e66fe43aa4e44aa549d9f067bad738a6b253dc6f3eee3c/simplepipe-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "2b78cfff50a897fb10bdf33f6f3a0fc3", "sha256": "b29d42ab1712e33e9c18fc70dc40e5643144ff29a6c4301562f49b9b26373874" }, "downloads": -1, "filename": "simplepipe-0.0.4.macosx-10.6-x86_64.tar.gz", "has_sig": false, "md5_digest": "2b78cfff50a897fb10bdf33f6f3a0fc3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6180, "upload_time": "2017-04-04T21:02:17", "url": "https://files.pythonhosted.org/packages/c9/d7/9392d5cf8df05ab7c62fa551579ab4d776441eccee8131b66531bc7a6a69/simplepipe-0.0.4.macosx-10.6-x86_64.tar.gz" }, { "comment_text": "", "digests": { "md5": "30f7c60dcc5079595397b26162a5e139", "sha256": "8cbad8c0b74e98bbd5048b975fc6f8e208b421dbde5ecdd5fbdeab1198225c7f" }, "downloads": -1, "filename": "simplepipe-0.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "30f7c60dcc5079595397b26162a5e139", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7373, "upload_time": "2017-04-04T21:02:14", "url": "https://files.pythonhosted.org/packages/17/e4/7a319307998b7f578e4880639f4e1af09b5f7db1e30fc5c84f2c184cf6fa/simplepipe-0.0.4-py2.py3-none-any.whl" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "9772fbec95d846b847eadb6d18fb9e4f", "sha256": "6814fb2dc0bb572b96d4ab8f3a1974edd927c207fdd83bc830cf38fd0d558c79" }, "downloads": -1, "filename": "simplepipe-0.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9772fbec95d846b847eadb6d18fb9e4f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7384, "upload_time": "2017-04-04T21:42:23", "url": "https://files.pythonhosted.org/packages/64/97/3ce1eb0e238f7b83cf937e457f0cd91655792a50e3a17141fe267e060f18/simplepipe-0.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4ae3d08542ccaa52a50dc0ea416fe8d5", "sha256": "f8af88030b1ccc59a508211e1687fe0cfce504a5d744d479e66733144e215b69" }, "downloads": -1, "filename": "simplepipe-0.0.5.tar.gz", "has_sig": false, "md5_digest": "4ae3d08542ccaa52a50dc0ea416fe8d5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4673, "upload_time": "2017-04-04T21:42:24", "url": "https://files.pythonhosted.org/packages/24/42/78f59f6981b0812d8032575f05c3f99de0142c2fd3537341b5e9d313d361/simplepipe-0.0.5.tar.gz" } ], "0.0.5.2": [ { "comment_text": "", "digests": { "md5": "40c73495c69b203fdce0478ea697524f", "sha256": "e7bbd2498617126f036c32b9a3557a0374e744b4abc9d8d9ee6a693bbce984bf" }, "downloads": -1, "filename": "simplepipe-0.0.5.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "40c73495c69b203fdce0478ea697524f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7598, "upload_time": "2017-04-17T02:20:25", "url": "https://files.pythonhosted.org/packages/b8/b7/e9f6c07dfe433b615321098f8db75cab2f420c52cb4c519c2831333bd902/simplepipe-0.0.5.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8d25a2004fa2ffbd1a9faf30efee9676", "sha256": "a1d11e581e14914f6d34f4bb1c5b37ab85f2cb86cf30809ce05efaf2af3b444f" }, "downloads": -1, "filename": "simplepipe-0.0.5.2.tar.gz", "has_sig": false, "md5_digest": "8d25a2004fa2ffbd1a9faf30efee9676", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4859, "upload_time": "2017-04-17T02:20:27", "url": "https://files.pythonhosted.org/packages/35/54/40dfecfacea97088aa90ab1e572d8848c20e6e0276c62be66296494d2779/simplepipe-0.0.5.2.tar.gz" } ], "0.0.5.3": [ { "comment_text": "", "digests": { "md5": "32cf7398cbad2817816bbde11dff102c", "sha256": "6546271926941b9fa1a838d200b5d21051e4613326e026ee13c9350d9de0d144" }, "downloads": -1, "filename": "simplepipe-0.0.5.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "32cf7398cbad2817816bbde11dff102c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7573, "upload_time": "2017-04-17T02:48:55", "url": "https://files.pythonhosted.org/packages/e9/c1/d07d0f08c28e6b362407f1f1390c87106bf054758c7126364a59cb3d8dad/simplepipe-0.0.5.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0389c6840765a7fda8caff7f14564085", "sha256": "e2fd059ee62aa53f97e996d904251d296bcf85b12b6b5ffa71d3895a7b6b4654" }, "downloads": -1, "filename": "simplepipe-0.0.5.3.tar.gz", "has_sig": false, "md5_digest": "0389c6840765a7fda8caff7f14564085", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4839, "upload_time": "2017-04-17T02:48:56", "url": "https://files.pythonhosted.org/packages/21/cb/ddc454f74e6c97ee711ce934ffdb94927ce38499c09967fa175f5cee20fc/simplepipe-0.0.5.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "32cf7398cbad2817816bbde11dff102c", "sha256": "6546271926941b9fa1a838d200b5d21051e4613326e026ee13c9350d9de0d144" }, "downloads": -1, "filename": "simplepipe-0.0.5.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "32cf7398cbad2817816bbde11dff102c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7573, "upload_time": "2017-04-17T02:48:55", "url": "https://files.pythonhosted.org/packages/e9/c1/d07d0f08c28e6b362407f1f1390c87106bf054758c7126364a59cb3d8dad/simplepipe-0.0.5.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0389c6840765a7fda8caff7f14564085", "sha256": "e2fd059ee62aa53f97e996d904251d296bcf85b12b6b5ffa71d3895a7b6b4654" }, "downloads": -1, "filename": "simplepipe-0.0.5.3.tar.gz", "has_sig": false, "md5_digest": "0389c6840765a7fda8caff7f14564085", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4839, "upload_time": "2017-04-17T02:48:56", "url": "https://files.pythonhosted.org/packages/21/cb/ddc454f74e6c97ee711ce934ffdb94927ce38499c09967fa175f5cee20fc/simplepipe-0.0.5.3.tar.gz" } ] }