{ "info": { "author": "Dan O'Reilly", "author_email": "oreilldf@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Natural Language :: English", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "aioprocessing\n=============\n[![Build Status](https://travis-ci.org/dano/aioprocessing.svg?branch=master)](https://travis-ci.org/dano/aioprocessing)\n\n\n`aioprocessing` provides asynchronous, [`asyncio`](https://docs.python.org/3/library/asyncio.html) compatible, coroutine \nversions of many blocking instance methods on objects in the [`multiprocessing`](https://docs.python.org/3/library/multiprocessing.html) \nlibrary. Here's an example demonstrating the `aioprocessing` versions of \n`Event`, `Queue`, and `Lock`:\n\n```python\n import time\n import asyncio\n import aioprocessing\n import multiprocessing\n\n\n def func(queue, event, lock, items):\n \"\"\" Demo worker function.\n\n This worker function runs in its own process, and uses\n normal blocking calls to aioprocessing objects, exactly \n the way you would use oridinary multiprocessing objects.\n\n \"\"\"\n with lock:\n event.set()\n for item in items:\n time.sleep(3)\n queue.put(item+5)\n queue.close()\n\n @asyncio.coroutine\n def example(queue, event, lock):\n l = [1,2,3,4,5]\n p = aioprocessing.AioProcess(target=func, args=(queue, event, lock, l))\n p.start()\n while True:\n result = yield from queue.coro_get()\n if result is None:\n break\n print(\"Got result {}\".format(result))\n yield from p.coro_join()\n\n @asyncio.coroutine\n def example2(queue, event, lock):\n yield from event.coro_wait()\n with (yield from lock):\n yield from queue.coro_put(78)\n yield from queue.coro_put(None) # Shut down the worker\n\n if __name__ == \"__main__\":\n loop = asyncio.get_event_loop()\n queue = aioprocessing.AioQueue()\n lock = aioprocessing.AioLock()\n event = aioprocessing.AioEvent()\n tasks = [\n asyncio.ensure_future(example(queue, event, lock)), \n asyncio.ensure_future(example2(queue, event, lock)),\n ]\n loop.run_until_complete(asyncio.wait(tasks))\n loop.close()\n```\n\nPython 3.5 syntax is supported, too. This means the `example2` function above \ncould look like this:\n\n```python\n async def example2(queue, event, lock):\n await event.coro_wait()\n async with lock:\n await queue.coro_put(78)\n await queue.coro_put(None) # Shut down the worker\n```\n\nThe aioprocessing objects can be used just like their multiprocessing\nequivalents - as they are in `func` above - but they can also be \nseamlessly used inside of `asyncio` coroutines, without ever blocking\nthe event loop.\n\n\nHow does it work?\n-----------------\n\nIn most cases, this library makes blocking calls to `multiprocessing` methods\nasynchronous by executing the call in a [`ThreadPoolExecutor`](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor), using\n[`asyncio.run_in_executor()`](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.BaseEventLoop.run_in_executor). \nIt does *not* re-implement multiprocessing using asynchronous I/O. This means \nthere is extra overhead added when you use `aioprocessing` objects instead of \n`multiprocessing` objects, because each one is generally introducing a\n`ThreadPoolExecutor` containing at least one [`threading.Thread`](https://docs.python.org/2/library/threading.html#thread-objects). It also means \nthat all the normal risks you get when you mix threads with fork apply here, too \n(See http://bugs.python.org/issue6721 for more info).\n\nThe one exception to this is `aioprocessing.AioPool`, which makes use of the \nexisting `callback` and `error_callback` keyword arguments in the various \n[`Pool.*_async`](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.apply_async) methods to run them as `asyncio` coroutines. Note that \n`multiprocessing.Pool` is actually using threads internally, so the thread/fork\nmixing caveat still applies.\n\nEach `multiprocessing` class is replaced by an equivalent `aioprocessing` class,\ndistinguished by the `Aio` prefix. So, `Pool` becomes `AioPool`, etc. All methods\nthat could block on I/O also have a coroutine version that can be used with `asyncio`. For example, `multiprocessing.Lock.acquire()` can be replaced with `aioprocessing.AioLock.coro_acquire()`. You can pass an `asyncio` EventLoop object to any `coro_*` method using the `loop` keyword argument. For example, `lock.coro_acquire(loop=my_loop)`.\n\nNote that you can also use the `aioprocessing` synchronization primitives as replacements \nfor their equivalent `threading` primitives, in single-process, multi-threaded programs \nthat use `asyncio`.\n\n\nWhat parts of multiprocessing are supported?\n--------------------------------------------\n\nMost of them! All methods that could do blocking I/O in the following objects\nhave equivalent versions in `aioprocessing` that extend the `multiprocessing`\nversions by adding coroutine versions of all the blocking methods.\n\n- `Pool`\n- `Process`\n- `Pipe`\n- `Lock`\n- `RLock`\n- `Semaphore`\n- `BoundedSemaphore`\n- `Event`\n- `Condition`\n- `Barrier`\n- `connection.Connection`\n- `connection.Listener`\n- `connection.Client`\n- `Queue`\n- `JoinableQueue`\n- `SimpleQueue`\n- All `managers.SyncManager` `Proxy` versions of the items above (`SyncManager.Queue`, `SyncManager.Lock()`, etc.).\n\n\nWhat versions of Python are compatible?\n---------------------------------------\n\n`aioprocessing` will work out of the box on Python 3.4+.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/dano/aioprocessing", "keywords": "asyncio multiprocessing coroutine", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "aioprocessing", "package_url": "https://pypi.org/project/aioprocessing/", "platform": "", "project_url": "https://pypi.org/project/aioprocessing/", "project_urls": { "Homepage": "https://github.com/dano/aioprocessing" }, "release_url": "https://pypi.org/project/aioprocessing/1.0.1/", "requires_dist": null, "requires_python": "", "summary": "A Python 3.3+ library that integrates the multiprocessing module with asyncio.", "version": "1.0.1" }, "last_serial": 4020838, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "d519eaa5bc795c0ed9fe7615063b13ec", "sha256": "49c770a1026099f351d1383bf70cdd0e89f6fcdf3ea770ac0b0523a4b0506568" }, "downloads": -1, "filename": "aioprocessing-0.0.1.tar.gz", "has_sig": false, "md5_digest": "d519eaa5bc795c0ed9fe7615063b13ec", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8504, "upload_time": "2014-09-20T05:30:59", "url": "https://files.pythonhosted.org/packages/cc/e8/c03a42f33657ae377a2285a53d1568f4915f31aa61fea848dbb50da99983/aioprocessing-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "0fbacf44bfe8f0478cb966def94f5b3f", "sha256": "1aa20e60c3215396762c5ae0e9cba25d2ea89c71f978dc4ee6bf15efe3b02486" }, "downloads": -1, "filename": "aioprocessing-0.0.2.linux-x86_64.tar.gz", "has_sig": false, "md5_digest": "0fbacf44bfe8f0478cb966def94f5b3f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18779, "upload_time": "2016-11-24T05:23:31", "url": "https://files.pythonhosted.org/packages/46/8c/13e99b7af7dfdc3de1497fe9c549938ba83f51d1bc33fd611130d828ef6c/aioprocessing-0.0.2.linux-x86_64.tar.gz" }, { "comment_text": "", "digests": { "md5": "59bee8c6eda94009dd9e3086cead59a3", "sha256": "9123e25c0eeef2e9c09f76205159f7c709028f04dcade502602f7353bc778307" }, "downloads": -1, "filename": "aioprocessing-0.0.2.tar.gz", "has_sig": false, "md5_digest": "59bee8c6eda94009dd9e3086cead59a3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9198, "upload_time": "2016-11-24T05:23:33", "url": "https://files.pythonhosted.org/packages/75/2a/0012c515104f4ff573f1c2273a01fb3f5946fc9c77a328360a57489884ae/aioprocessing-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "b6fd692127b6c5d816bf18260f8a2acf", "sha256": "2b354b3e99224e613bc0558fda94c3c04c53a86390e1a6e1fae5ba7a36de6901" }, "downloads": -1, "filename": "aioprocessing-0.0.3.tar.gz", "has_sig": false, "md5_digest": "b6fd692127b6c5d816bf18260f8a2acf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10653, "upload_time": "2017-02-01T13:05:57", "url": "https://files.pythonhosted.org/packages/9c/d9/8057ceb331b5744539bc4c1eb371a0fd918a85e08e011e8b23e17651eddf/aioprocessing-0.0.3.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "5f72f29e2e67f73f376dc5c841b32081", "sha256": "7cc8cb25499017655868759db91c36c30e820c64da582aeef418ab424f7c3579" }, "downloads": -1, "filename": "aioprocessing-1.0.0.tar.gz", "has_sig": false, "md5_digest": "5f72f29e2e67f73f376dc5c841b32081", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10657, "upload_time": "2017-05-31T13:32:01", "url": "https://files.pythonhosted.org/packages/1c/6b/239dc26f33d146e0bb3b753c31977dc6f94902d847930dcb085c5f99a086/aioprocessing-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "7a6be1d600cda7e117caabd70db2c180", "sha256": "9b88f4e51b7358e7a53ea2b6cfd1e000848270a1b09532a410b6d38e015b7de5" }, "downloads": -1, "filename": "aioprocessing-1.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7a6be1d600cda7e117caabd70db2c180", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12827, "upload_time": "2018-07-01T19:51:06", "url": "https://files.pythonhosted.org/packages/a2/87/edcfc09731bd94f9776e34587939948ba5f67b995773990e5896657e9655/aioprocessing-1.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "868ead8536e0bcaa5e573f6df8da7192", "sha256": "b6952b476586c2a2e0d2802f42d73e6898ab474213ffda788d720a3fb57b01fb" }, "downloads": -1, "filename": "aioprocessing-1.0.1.tar.gz", "has_sig": false, "md5_digest": "868ead8536e0bcaa5e573f6df8da7192", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12143, "upload_time": "2018-07-01T19:51:07", "url": "https://files.pythonhosted.org/packages/38/b1/7c24e264f240f89310ed95bf5257566deb4ae719e306e7eb6e654459383d/aioprocessing-1.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7a6be1d600cda7e117caabd70db2c180", "sha256": "9b88f4e51b7358e7a53ea2b6cfd1e000848270a1b09532a410b6d38e015b7de5" }, "downloads": -1, "filename": "aioprocessing-1.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "7a6be1d600cda7e117caabd70db2c180", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 12827, "upload_time": "2018-07-01T19:51:06", "url": "https://files.pythonhosted.org/packages/a2/87/edcfc09731bd94f9776e34587939948ba5f67b995773990e5896657e9655/aioprocessing-1.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "868ead8536e0bcaa5e573f6df8da7192", "sha256": "b6952b476586c2a2e0d2802f42d73e6898ab474213ffda788d720a3fb57b01fb" }, "downloads": -1, "filename": "aioprocessing-1.0.1.tar.gz", "has_sig": false, "md5_digest": "868ead8536e0bcaa5e573f6df8da7192", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12143, "upload_time": "2018-07-01T19:51:07", "url": "https://files.pythonhosted.org/packages/38/b1/7c24e264f240f89310ed95bf5257566deb4ae719e306e7eb6e654459383d/aioprocessing-1.0.1.tar.gz" } ] }