{ "info": { "author": "Fabrice Salvaire", "author_email": "fabrice.salvaire@orange.fr", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Framework :: AsyncIO", "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: CPython", "Topic :: System :: Distributed Computing", "Topic :: System :: Hardware :: Symmetric Multi-processing" ], "description": ".. -*- Mode: rst -*-\n\n.. -*- Mode: rst -*-\n\n..\n |siditaUrl|\n |siditaHomePage|_\n |siditaDoc|_\n |sidita@github|_\n |sidita@readthedocs|_\n |sidita@readthedocs-badge|\n |sidita@pypi|_\n\n.. |ohloh| image:: https://www.openhub.net/accounts/230426/widgets/account_tiny.gif\n :target: https://www.openhub.net/accounts/fabricesalvaire\n :alt: Fabrice Salvaire's Ohloh profile\n :height: 15px\n :width: 80px\n\n.. |siditaUrl| replace:: https://fabricesalvaire.github.io/sidita\n\n.. |siditaHomePage| replace:: sidita Home Page\n.. _siditaHomePage: https://fabricesalvaire.github.io/sidita\n\n.. |sidita@readthedocs-badge| image:: https://readthedocs.org/projects/sidita/badge/?version=latest\n :target: http://sidita.readthedocs.org/en/latest\n\n.. |sidita@github| replace:: https://github.com/FabriceSalvaire/sidita\n.. .. _sidita@github: https://github.com/FabriceSalvaire/sidita\n\n.. |sidita@pypi| replace:: https://pypi.python.org/pypi/sidita\n.. .. _sidita@pypi: https://pypi.python.org/pypi/sidita\n\n.. |Build Status| image:: https://travis-ci.org/FabriceSalvaire/sidita.svg?branch=master\n :target: https://travis-ci.org/FabriceSalvaire/sidita\n :alt: sidita build status @travis-ci.org\n\n.. |Pypi Version| image:: https://img.shields.io/pypi/v/sidita.svg\n :target: https://pypi.python.org/pypi/sidita\n :alt: sidita last version\n\n.. |Pypi License| image:: https://img.shields.io/pypi/l/sidita.svg\n :target: https://pypi.python.org/pypi/sidita\n :alt: sidita license\n\n.. |Pypi Python Version| image:: https://img.shields.io/pypi/pyversions/sidita.svg\n :target: https://pypi.python.org/pypi/sidita\n :alt: sidita python version\n\n.. coverage test\n.. https://img.shields.io/pypi/status/Django.svg\n.. https://img.shields.io/github/stars/badges/shields.svg?style=social&label=Star\n.. -*- Mode: rst -*-\n\n.. |Python| replace:: Python\n.. _Python: http://python.org\n\n.. |PyPI| replace:: PyPI\n.. _PyPI: https://pypi.python.org/pypi\n\n.. |Sphinx| replace:: Sphinx\n.. _Sphinx: http://sphinx-doc.org\n\n.. |asyncio| replace:: asyncio\n.. _asyncio: https://docs.python.org/3/library/asyncio.html\n\n.. |asyncio event loop| replace:: asyncio event loop\n.. _asyncio event loop: https://docs.python.org/3/library/asyncio-eventloops.html\n\n.. |asyncio queue| replace:: asyncio queue\n.. _asyncio queue: https://docs.python.org/3/library/asyncio-queue.html\n\n.. |asyncio subprocess| replace:: asyncio subprocess\n.. _asyncio subprocess: https://docs.python.org/3/library/asyncio-subprocess.html\n\n.. |multiprocessing module| replace:: multiprocessing module\n.. _multiprocessing module: https://docs.python.org/3.6/library/multiprocessing.html\n\n.. |Celery| replace:: Celery\n.. _Celery: http://www.celeryproject.org\n\n============\n sidita\n============\n\n|Pypi License|\n|Pypi Python Version|\n\n|Pypi Version|\n\n* Quick Link to `Production Branch `_\n* Quick Link to `Devel Branch `_\n\nOverview\n========\n\nWhat is sidita ?\n---------------------\n\nSidita is a Python module which implements a distributed task queue featuring an intermediate\nsolution between the |multiprocessing module|_ and a task scheduler like |Celery|_.\n\nThe Sidita use case corresponds to the case where you need to run CPU bound tasks in parallel and\nyou require an immunity to crashes, memory leaks and overruns. Theses requirements are met by a\nmonitored subprocess implementation.\n\nWhere is the Documentation ?\n----------------------------\n\nThe documentation is available on the |siditaHomePage|_.\n\nWhat are the main features ?\n----------------------------\n\n.. -*- Mode: rst -*-\n\n\n.. no title here\n\n* Must be a simple solution to circumvent to the GIL limitation : i.e. distribute and run tasks in parallel on local CPU cores\n* Must be a lightweight solution to Celery : no broker\n* Should be portable on main OS like Unix and Windows\n* Scheduler is implemented using an |asyncio event loop|_ and |asyncio queue|_\n* Scheduler queue can be limited on size so as to prevent a memory overshoot at startup\n* Workers are spawned using |asyncio subprocess|_ and communicate through a pipe (stdin, stdout)\n* Pass any pickable object as request and response\n* Scheduler monitors workers and restart them when they was killed or they exceeded a memory or timeout threshold\n* Provide basic metrics on workers\n\nTechnical details in brief:\n\n* scheduler run a producer and N consumers coroutines in an |asyncio|_ event loop\n* the producer coroutine awaits on :code:`queue.put(task)` ( if a size limit is set )\n* each consumer coroutine wraps a worker subprocess and await data on the stdout pipe\n\n.. -*- Mode: rst -*-\n\n\n.. no title here\n\nTechnical details in brief:\n\n* scheduler run a producer and N consumers coroutines in an |asyncio|_ event loop\n* the producer coroutine awaits on :code:`queue.put(task)` ( if a size limit is set )\n* each consumer coroutine wraps a worker subprocess and await data on the stdout pipe\n\nHow to install it ?\n-------------------\n\nLook at the `installation `_ section in the documentation.\n\nCredits\n=======\n\nAuthors: `Fabrice Salvaire `_\n\nNews\n====\n\n.. -*- Mode: rst -*-\n\n\n.. no title here\n\nV0 2018-05-01\n-------------\n\nStarted project\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/FabriceSalvaire/sidita", "keywords": "distributed task queue asyncio multiprocessing", "license": "GPLv3", "maintainer": "", "maintainer_email": "", "name": "sidita", "package_url": "https://pypi.org/project/sidita/", "platform": "any", "project_url": "https://pypi.org/project/sidita/", "project_urls": { "Homepage": "https://github.com/FabriceSalvaire/sidita" }, "release_url": "https://pypi.org/project/sidita/1.0.1/", "requires_dist": [ "PyYAML (>=3.10)", "psutil (>=5.4.3)" ], "requires_python": "", "summary": "Simple Distributed Task Queue : multiprocessing < sidita < celery", "version": "1.0.1" }, "last_serial": 3822397, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "60d305e11df29003845a062edcc13e6e", "sha256": "782d75b0dcd7fbd46d8825fff2bc78d85e08ccde6ed584a7aa0cef22a146777c" }, "downloads": -1, "filename": "sidita-1.0.0-py2.py3-none-any.whl", "has_sig": true, "md5_digest": "60d305e11df29003845a062edcc13e6e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 15868, "upload_time": "2018-04-30T22:35:01", "url": "https://files.pythonhosted.org/packages/c4/8d/6fb7439547caa48b95abbbfd87fa7066d286b726189a662101a29bc0c7d3/sidita-1.0.0-py2.py3-none-any.whl" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "2ec04bad585051fba390db67ca1f848f", "sha256": "6b33b9279d285b219f5cfa66e17428a97669e3922b714228cacce36a3e02d73e" }, "downloads": -1, "filename": "sidita-1.0.1-py2.py3-none-any.whl", "has_sig": true, "md5_digest": "2ec04bad585051fba390db67ca1f848f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 15892, "upload_time": "2018-04-30T22:49:44", "url": "https://files.pythonhosted.org/packages/80/b8/53000883cfe1921037eedd6451809a055a2e39afb93b2e9445ba2b8b6862/sidita-1.0.1-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2ec04bad585051fba390db67ca1f848f", "sha256": "6b33b9279d285b219f5cfa66e17428a97669e3922b714228cacce36a3e02d73e" }, "downloads": -1, "filename": "sidita-1.0.1-py2.py3-none-any.whl", "has_sig": true, "md5_digest": "2ec04bad585051fba390db67ca1f848f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 15892, "upload_time": "2018-04-30T22:49:44", "url": "https://files.pythonhosted.org/packages/80/b8/53000883cfe1921037eedd6451809a055a2e39afb93b2e9445ba2b8b6862/sidita-1.0.1-py2.py3-none-any.whl" } ] }