{ "info": { "author": "['Yauhen Yakimovich', 'Markus Herrmann']", "author_email": "eugeny.yakimovitch@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "License :: OSI Approved :: MIT License", "Operating System :: MacOS", "Operating System :: POSIX :: Linux", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Topic :: Scientific/Engineering :: Bio-Informatics", "Topic :: Scientific/Engineering :: Image Recognition", "Topic :: Scientific/Engineering :: Information Analysis" ], "description": "Jterator\r\n========\r\n\r\nA minimalistic pipeline engine with motivation to be flexible, but uncomplicated enough, so that no GUI is required to work with it.\r\n\r\n* use simple format like JSON for basic parametrization whenever possible\r\n** don't be afraid to keep your input settings on the disk\r\n* for performance heavy IO use HDF5 \r\n* used scientific data analysis.\r\n* UNIX pipeline is a one great idea, but it is mostly restricted to text processing\r\n\r\nModules\r\n-------\r\n\r\n\r\nThink of your pipeline as a sequence of connected modules (a linked list). Each module is a program that get and reads JSON from on STDIN file descriptor.\r\nSuch JSON contains all the input settings required by the module.\r\n\r\nGetting started \r\n---------------\r\n\r\nEach pipeline has the following layout on the disk:\r\n\r\n* **handles** folder contains all the JSON handles files, the are passed as STDIN to *modules*.\r\n* **modules** folder contains all the executable plus code for programs corresponding\r\n* **logs** folder contains all the output from STDOU and STERR streams, obtain for each executable that has been executed.\r\n* **data** folder contains all the heavy data output like HDF5, etc. These data are shared between modules. \r\n\r\nJterator allows only very simplistic type of workflow - *pipeline* (somewhat similar to a UNIX-world pipeline). Description of such workflow must be put sibling to the folder structure described about, i.e. inside the Jterator pipeline (project) folder. Recognizable file name must be one of **['JteratorPipe.json', 'jt.pipe']**. Description is a JSON format. \r\n\r\n```json\r\n{\t\r\n\t\"name\": \"Foo\",\r\n\t\"version\": \"0.0.1\",\t\r\n\t\"pipeline\": [\r\n\t\t{\r\n\t\t\t\"name\": \"Bar\",\r\n\t\t\t\"module\": \"bar\",\r\n\t\t\t\"handles\": \"handles/some_bar\"\r\n\t\t}, {\r\n\t\t\t\"name\": \"Baz\",\r\n\t\t\t\"module\": \"baz\",\r\n\t\t\t\"handles\": \"handles/some_baz\"\r\n\t\t}\r\n\t],\r\n\t\"tests\": [\r\n\t\t{\r\n\t\t\t\"type\": \"hdf5_dependency\",\r\n\t\t\t\"module_name\": \"baz\",\r\n\t\t\t\"input\": [\"bar\"],\r\n\t\t\t\"output\": [\"baz\"]\r\n\t\t}\r\n\t]\r\n}\r\n\r\n```\r\n\r\nTo *run* your first pipeline, do:\r\n\r\n```bash\r\ncd /my/first/jterator/pipeline/folder && jt run\r\n```\r\n\r\n\r\n\r\nDeveloping new modules\r\n======================\r\n\r\nThis is a small walk-through on how to develop a new module for *Jterator*. Each module as to follow a particular convention of processing input parameters. It can be written in virtually any programming language as long as such language can provide tools for working with *JSON* and *HDF5* data formats.\r\n\r\nDeveloping Jterator\r\n===================\r\n\r\nLatest code is available at https://github.com/ewiger/Jterator\r\n\r\nNose tests\r\n----------\r\n\r\nWe use nose framework to achieve code coverage with unit tests. In order to run tests, do\r\n\r\n```bash\r\ncd tests && nosetests\r\n```", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/ewiger/Jterator/tarball/master", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ewiger/Jterator", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "Jterator", "package_url": "https://pypi.org/project/Jterator/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/Jterator/", "project_urls": { "Download": "https://github.com/ewiger/Jterator/tarball/master", "Homepage": "https://github.com/ewiger/Jterator" }, "release_url": "https://pypi.org/project/Jterator/0.0.1/", "requires_dist": null, "requires_python": null, "summary": "A minimalistic pipeline engine", "version": "0.0.1" }, "last_serial": 1179677, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "5dcf67a822cb623b4afef02778294a13", "sha256": "816c77399a822a13b1997fd7a6630ac84115fa35d0403b88bdb20c343434ede8" }, "downloads": -1, "filename": "Jterator-0.0.1.tar.gz", "has_sig": false, "md5_digest": "5dcf67a822cb623b4afef02778294a13", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7961, "upload_time": "2014-08-04T18:28:55", "url": "https://files.pythonhosted.org/packages/4f/d8/8f2401e4eb490f157344d4b1c8a0e425ec362d472ff0d8a8534305cb51ff/Jterator-0.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5dcf67a822cb623b4afef02778294a13", "sha256": "816c77399a822a13b1997fd7a6630ac84115fa35d0403b88bdb20c343434ede8" }, "downloads": -1, "filename": "Jterator-0.0.1.tar.gz", "has_sig": false, "md5_digest": "5dcf67a822cb623b4afef02778294a13", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7961, "upload_time": "2014-08-04T18:28:55", "url": "https://files.pythonhosted.org/packages/4f/d8/8f2401e4eb490f157344d4b1c8a0e425ec362d472ff0d8a8534305cb51ff/Jterator-0.0.1.tar.gz" } ] }