{ "info": { "author": "Tamas Gal", "author_email": "tgal@km3net.de", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "Intended Audience :: Science/Research", "Programming Language :: Python" ], "description": "thepipe\n=======\n\n.. image:: https://readthedocs.org/projects/thepipe/badge/?version=latest\n :target: https://thepipe.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n.. image:: https://api.codacy.com/project/badge/Grade/20a35727ae364e08845b60bdeb4b233a\n :alt: Codacy Badge\n :target: https://www.codacy.com/app/tamasgal/thepipe?utm_source=github.com&utm_medium=referral&utm_content=tamasgal/thepipe&utm_campaign=Badge_Grade\n\n.. image:: https://travis-ci.org/tamasgal/thepipe.svg?branch=master\n :alt: Travis-CI Build Status\n :target: https://travis-ci.org/tamasgal/thepipe\n\n.. image:: http://codecov.io/github/tamasgal/thepipe/coverage.svg?branch=master\n :alt: Test-coverage\n :target: http://codecov.io/github/tamasgal/thepipe?branch=master\n\n.. image:: https://img.shields.io/pypi/v/thepipe.svg?style=flat\n :alt: PyPI Package latest release\n :target: https://pypi.python.org/pypi/thepipe\n\nA simplistic, general purpose pipeline framework, which can easily be\nintegrated into existing (analysis) chains and workflows.\n\nInstallation\n------------\n``thepipe`` can be installed via ``pip``::\n\n pip install thepipe\n\nThe Pipeline\n------------\n\nHere is a basic example how to create a pipeline, add some modules to it, pass\nsome parameters and drain the pipeline.\n\nNote that pipeline modules can either be vanilla (univariate) Python functions\nor Classes which derive from ``thepipe.Module``.\n\n.. code-block:: python\n\n import thepipe as tp\n\n\n class AModule(tp.Module):\n def configure(self):\n self.cprint(\"Configuring AModule\")\n self.max_count = self.get(\"max_count\", default=23)\n self.index = 0\n\n def process(self, blob):\n self.cprint(\"This is cycle #%d\" % self.index)\n blob['index'] = self.index\n self.index += 1\n\n if self.index > self.max_count:\n self.log.critical(\"That's enough...\")\n raise StopIteration\n return blob\n\n def finish(self):\n self.cprint(\"I'm done!\")\n\n\n def a_function_based_module(blob):\n print(\"Here is the blob:\")\n print(blob)\n return blob\n\n\n pipe = tp.Pipeline()\n pipe.attach(AModule, max_count=5) # pass any parameters to the module\n pipe.attach(a_function_based_module)\n pipe.drain() # without arguments it will drain until a StopIteration is raised\n\nThis will produce the following output:\n\n.. code-block:: shell\n\n ++ AModule: Configuring AModule\n Pipeline and module initialisation took 0.000s (CPU 0.000s).\n ++ AModule: This is cycle #0\n Here is the blob:\n Blob (1 entries):\n 'index' => 0\n ++ AModule: This is cycle #1\n Here is the blob:\n Blob (1 entries):\n 'index' => 1\n ++ AModule: This is cycle #2\n Here is the blob:\n Blob (1 entries):\n 'index' => 2\n ++ AModule: This is cycle #3\n Here is the blob:\n Blob (1 entries):\n 'index' => 3\n ++ AModule: This is cycle #4\n Here is the blob:\n Blob (1 entries):\n 'index' => 4\n ++ AModule: This is cycle #5\n CRITICAL ++ AModule: That's enough...\n ++ AModule: I'm done!\n ============================================================\n 5 cycles drained in 0.000793s (CPU 0.000793s). Memory peak: 20.56 MB\n wall mean: 0.000063s medi: 0.000057s min: 0.000045s max: 0.000106s std: 0.000022s\n CPU mean: 0.000065s medi: 0.000057s min: 0.000046s max: 0.000112s std: 0.000024s", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/tamasgal/thepipe", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "thepipe", "package_url": "https://pypi.org/project/thepipe/", "platform": "any", "project_url": "https://pypi.org/project/thepipe/", "project_urls": { "Homepage": "http://github.com/tamasgal/thepipe" }, "release_url": "https://pypi.org/project/thepipe/1.0.1/", "requires_dist": null, "requires_python": ">=3.5", "summary": "A lightweight, general purpose pipeline framework.", "version": "1.0.1" }, "last_serial": 5978165, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "ea17bed5d509be0cc3e07a7bea5b923a", "sha256": "e72913406fb3c3937718d380b5bed7447f7bea9e935342da412b0c7b6e5d2603" }, "downloads": -1, "filename": "thepipe-1.0.0.tar.gz", "has_sig": false, "md5_digest": "ea17bed5d509be0cc3e07a7bea5b923a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 15484, "upload_time": "2019-03-26T11:25:04", "url": "https://files.pythonhosted.org/packages/71/21/0f97a92ed55a694697f8a3cc5b7667e2ad3b0bb90b1f3fdd16613603e2bf/thepipe-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "ca21d9332f383cc8248262f2f6c05204", "sha256": "0bee68b792ebe2a66cf8a736f220273804e055fdcda16d69837f6acc041d19d7" }, "downloads": -1, "filename": "thepipe-1.0.1.tar.gz", "has_sig": false, "md5_digest": "ca21d9332f383cc8248262f2f6c05204", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 49415, "upload_time": "2019-10-15T16:00:18", "url": "https://files.pythonhosted.org/packages/b1/a1/b37f231660dba9eff5f478920329af89214f199b9a0479a293d2d1a269e7/thepipe-1.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ca21d9332f383cc8248262f2f6c05204", "sha256": "0bee68b792ebe2a66cf8a736f220273804e055fdcda16d69837f6acc041d19d7" }, "downloads": -1, "filename": "thepipe-1.0.1.tar.gz", "has_sig": false, "md5_digest": "ca21d9332f383cc8248262f2f6c05204", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 49415, "upload_time": "2019-10-15T16:00:18", "url": "https://files.pythonhosted.org/packages/b1/a1/b37f231660dba9eff5f478920329af89214f199b9a0479a293d2d1a269e7/thepipe-1.0.1.tar.gz" } ] }