{ "info": { "author": "Viktor Kirillov", "author_email": "kirillovmr@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Information Technology", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# Python Data Pipeline\n\nProcess any type of data in your projects easily, control the flow of your data!\n\n![Python 3.5, 3.6, 3.7](https://img.shields.io/badge/python-3.5%2C%203.6%2C%203.7-blue.svg)\n[![PyPI version](https://badge.fury.io/py/smart-pipeline.svg)](https://badge.fury.io/py/smart-pipeline)\n\n---\n\n### Installation\nInstall `smart-pipeline` with:\n``` bash\npip install -U smart-pipeline\n```\n\n### Usage\nPackage 'smart_pipeline' provides a Pipeline class:\n``` python\n# Import Pipeline class\nfrom smart_pipeline import Pipeline\n\n# Create an instance\npl = Pipeline()\n```\n\nPipeline class has 3 types of pipes: item, data and stat.\n\n**Item pipe** modifies each item in dataset without changing the whole population of data:\n```python\ndata = [1,2,3,4,5]\n\n# Define an item function\ndef addOne(item):\n return item + 1\n\n# Adds function into pipeline\npl.addItemPipe(addOne)\n# Pass the data through pipeline\nres = pl(data)\n\n# res = [2,3,4,5,6]\n```\n\n**Data pipe** is a filter:\n``` python\ndata = [1,2,3,4,5]\n\ndef onlyOdd(item):\n return False if item%2==0 else True\n\npl.addDataPipe(onlyOdd)\nres = pl(data)\n\n# res = [1,3,5]\n```\n\n**Stat pipe** reduces over the data, passing the accumulated value to each element:\n``` python\ndata = [1,2,3,4,5]\n\n# Function that goes over all items in dataset\ndef countNumberStat(stats, item):\n stats[\"total\"] += 1\n if item%2==0:\n stats[\"even\"] += 1\n else:\n stats[\"odd\"] += 1\n return stats\n\n# Function to be called at the end with accumulated stats\ndef printNumberStat(stats):\n print(stats[\"total\"], \"items were processed in total.\")\n print(stats[\"even\"], \"of them are even.\")\n print(stats[\"odd\"], \"of them are odd\")\n\n# Make sure to pass initial state as 3rd argument\npl.addStatPipe(countNumberStat, printNumberStat, { \"total\":0, \"even\":0, \"odd\":0 })\npl(data)\n\n# Output:\n# 5 items were processed in total.\n# 2 of them are even.\n# 3 of them are odd\n```\n\n---\n### [Check out some examples](https://github.com/kirillovmr/python-pipeline/tree/master/examples)\n---\n\nIf this library solved some of your problems, please consider starring the project \ud83d\ude09\n\nAnd feel free to create pull requests!\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/kirillovmr/python-pipeline", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "smart-pipeline", "package_url": "https://pypi.org/project/smart-pipeline/", "platform": "", "project_url": "https://pypi.org/project/smart-pipeline/", "project_urls": { "Homepage": "https://github.com/kirillovmr/python-pipeline" }, "release_url": "https://pypi.org/project/smart-pipeline/1.0.0/", "requires_dist": [ "tqdm (>=4.31.1)" ], "requires_python": "", "summary": "Data Pipeline for your Python projects", "version": "1.0.0" }, "last_serial": 5620568, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "a5c68273bd9149b3a286144010fb43f0", "sha256": "8918d4b9b17f1817d11466cb9d391e12c1442d4235bd5056392b6bdd436d1dce" }, "downloads": -1, "filename": "smart_pipeline-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "a5c68273bd9149b3a286144010fb43f0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 2353, "upload_time": "2019-08-01T18:05:01", "url": "https://files.pythonhosted.org/packages/59/14/a39f4b5d465a5c093cbf49638a7c2e77e7234bb5bcf060902f33165e49f2/smart_pipeline-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "546aeafe39670ad1beefdfd837998cf5", "sha256": "24cee7a8b90fd020e3d0fa8b9f656b9155b7381b7c7706cf4bf9cc89e3e9b855" }, "downloads": -1, "filename": "smart_pipeline-0.0.1.tar.gz", "has_sig": false, "md5_digest": "546aeafe39670ad1beefdfd837998cf5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 1134, "upload_time": "2019-08-01T18:05:03", "url": "https://files.pythonhosted.org/packages/97/62/d85d787aa5f1e828adef25be8dd356fcc87cc3aa5ee88ffd5ed3c9da043f/smart_pipeline-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "75469aca4fe7951be9e1381530b59c2b", "sha256": "68228a9fa12e1d3ef27ea97ece2b65fbd7fa0488075a747248a250f7d7f5fa7e" }, "downloads": -1, "filename": "smart_pipeline-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "75469aca4fe7951be9e1381530b59c2b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6564, "upload_time": "2019-08-01T18:58:09", "url": "https://files.pythonhosted.org/packages/aa/dd/5e7f924ed32693757fdbe7d4bc271810f772051f00cb6d8c44be6dcd7e2b/smart_pipeline-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0a46d14f18abc296bf4725a2ed9ed3f8", "sha256": "7d2e9f7db3a9441272bd4937fc96d060acc2c72af9a27087b5bca216ce475390" }, "downloads": -1, "filename": "smart_pipeline-0.0.2.tar.gz", "has_sig": false, "md5_digest": "0a46d14f18abc296bf4725a2ed9ed3f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2035, "upload_time": "2019-08-01T18:58:11", "url": "https://files.pythonhosted.org/packages/ed/fb/d2281057606cb74b46a9b0f5a2bbdad520e682d182a50803825ee0f4778d/smart_pipeline-0.0.2.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "10a380a73f05823c64591bcbcd18c6e3", "sha256": "9d23804fd937b0e81bf3daaa52b6097531235cde394bf570222ec5e820306b9f" }, "downloads": -1, "filename": "smart_pipeline-1.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "10a380a73f05823c64591bcbcd18c6e3", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7393, "upload_time": "2019-08-01T20:17:23", "url": "https://files.pythonhosted.org/packages/b6/23/253ed58f9727a83a5dbc013cfeb1649151662f49c308e0193ab8c8df1ff0/smart_pipeline-1.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1cb966667e3ce947740b16a94f5d10b7", "sha256": "3a4fe02a5450a6efa906924ef6b8d275699c033c2f418674ed79202a2993e4fb" }, "downloads": -1, "filename": "smart_pipeline-1.0.0.tar.gz", "has_sig": false, "md5_digest": "1cb966667e3ce947740b16a94f5d10b7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3088, "upload_time": "2019-08-01T20:17:25", "url": "https://files.pythonhosted.org/packages/e6/88/7367105dc64f6dd5212234edbc8a0c43a906ad08ddf1d6fbb06347c23828/smart_pipeline-1.0.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "10a380a73f05823c64591bcbcd18c6e3", "sha256": "9d23804fd937b0e81bf3daaa52b6097531235cde394bf570222ec5e820306b9f" }, "downloads": -1, "filename": "smart_pipeline-1.0.0-py3-none-any.whl", "has_sig": false, "md5_digest": "10a380a73f05823c64591bcbcd18c6e3", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 7393, "upload_time": "2019-08-01T20:17:23", "url": "https://files.pythonhosted.org/packages/b6/23/253ed58f9727a83a5dbc013cfeb1649151662f49c308e0193ab8c8df1ff0/smart_pipeline-1.0.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1cb966667e3ce947740b16a94f5d10b7", "sha256": "3a4fe02a5450a6efa906924ef6b8d275699c033c2f418674ed79202a2993e4fb" }, "downloads": -1, "filename": "smart_pipeline-1.0.0.tar.gz", "has_sig": false, "md5_digest": "1cb966667e3ce947740b16a94f5d10b7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3088, "upload_time": "2019-08-01T20:17:25", "url": "https://files.pythonhosted.org/packages/e6/88/7367105dc64f6dd5212234edbc8a0c43a906ad08ddf1d6fbb06347c23828/smart_pipeline-1.0.0.tar.gz" } ] }