{ "info": { "author": "Dominic Slee", "author_email": "domslee1@gmail.com", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python :: 2", "Programming Language :: Python :: 3" ], "description": "# multiprocessing-wrap\n\n[![Build Status](https://travis-ci.org/domsleee/multiprocessing-wrap.svg?branch=master)](https://travis-ci.org/domsleee/multiprocessing-wrap)\n[![PyPI shield](https://img.shields.io/pypi/v/multiprocessing-wrap.svg?style=flat-square)](https://pypi.org/project/multiprocessing-wrap/)\n\nA simple interface for writing concurrent scripts. Get the most out of `multiprocessing` without all the boilerplate and confusing syntax!\n\n## Features\n\n* Sensible error propagation - having a stack trace showing where your code speeds debugging and development\n* Built-in loading bar as default using [tqdm](https://github.com/tqdm/tqdm)\n* Uses [dill](https://github.com/uqfoundation/dill) for pickling, which extends the types that can be passed to your workers (see [here](http://docs.python.org/library/pickle.html#what-can-be-pickled-and-unpickled) for documentation of the limitations of python's default pickling)\n\n## Installation\n\nTo install using pip:\n\n~~~bash\npip install multiprocessing-wrap\n~~~\n\n## Usage\n\nYou can use the functional `multiprocess` for single line multiprocessing:\n\n~~~python\nfrom multiprocess import multiprocess\n\nf = lambda: print(1)\nmultiprocess(f, [(), (), ()])\n~~~\n\n~~~bash\n1 \n1 \n1\n~~~\n\nOtherwise you can use the `Multiprocess` class to use the more explicit `add_tasks` and `do_tasks` directives:\n\n~~~python\nfrom multiprocess import Multiprocess\n\nm = Multiprocess(show_loading_bar=False)\nf = lambda: print(1)\nm.add_tasks(f, [(), (), ()])\nm.do_tasks() # blocking\nm.close()\n~~~\n\nA more involved example of sorting numbers using `sleep`. Since the worker function is run in a different process, to transfer data between the processes we use a thread-safe `Queue`. Note the following only works if you have at least 2 threads:\n~~~python\nfrom multiprocess import multiprocess, Queue\nfrom time import sleep\n\ndef sleep_sort():\n q = Queue()\n def f(q, x):\n sleep(x)\n q.push(x)\n\n multiprocess(f, [(q, 1,), (q, 2,)])\n print('SORTED')\n while not q.empty():\n print(q.pop())\n\nsleep_sort()\n~~~\n\n~~~bash \n1 \n2\n~~~\n\n## Error handling\nErrors from within a process are propagated back to the parent with stack information. For example:\n\n~~~python\nfrom multiprocess import Multiprocess\n\nm = Multiprocess()\ndef f(x):\n raise ValueError('bad error')\n\nm.add_tasks(f, [(1,)])\nm.do_tasks()\nm.close()\n~~~\n\n~~~bash\nTraceback (most recent call last):\n File \"\", line 1, in \n File \"/Users/dom/Documents/git/Multiprocess/multiprocess/multiprocessClass.py\", line 61, in do_tasks\n self._check_for_exceptions()\n File \"/Users/dom/Documents/git/Multiprocess/multiprocess/multiprocessClass.py\", line 79, in _check_for_exceptions\n \"\\n\".join(['ERROR: ' + str(e) for e in exceptions]))\nmultiprocess.multiprocessClass.MultiprocessProcessException: 1 errors occurred:\nERROR: Error in function call \"f((1,))\"\nTraceback (most recent call last):\n File \"/Users/dom/Documents/git/Multiprocess/multiprocess/multiprocessClass.py\", line 95, in my_worker\n fn(*rem_args)\n File \"\", line 2, in f\nValueError: bad errorr\n~~~\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/domsleee/multiprocess", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "multiprocessing-wrap", "package_url": "https://pypi.org/project/multiprocessing-wrap/", "platform": "", "project_url": "https://pypi.org/project/multiprocessing-wrap/", "project_urls": { "Homepage": "https://github.com/domsleee/multiprocess" }, "release_url": "https://pypi.org/project/multiprocessing-wrap/0.0.3/", "requires_dist": [ "tqdm", "dill" ], "requires_python": "", "summary": "Elegant multiprocessing without the boilerplate and confusing syntax", "version": "0.0.3" }, "last_serial": 4951753, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "1e9ab3eeabbee1dc464053a42eff2cdc", "sha256": "05240c7b66146dfab19f81f1d7bda9058153a76de42c118c5da6d725972c5fb8" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "1e9ab3eeabbee1dc464053a42eff2cdc", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 3217, "upload_time": "2019-03-17T07:12:41", "url": "https://files.pythonhosted.org/packages/e3/4d/3ee5dc9ccaf8b97f9cf073c5ac1d2394b588ac2bd9287a7956fa7f81059a/multiprocessing_wrap-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "12422fd95ccaecf831c8e2a4990946c6", "sha256": "cf96ddc9b54752fc6801093209f68ee35993835cf0c35fee1d55b7ee468b079c" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.1.tar.gz", "has_sig": false, "md5_digest": "12422fd95ccaecf831c8e2a4990946c6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2941, "upload_time": "2019-03-17T07:12:44", "url": "https://files.pythonhosted.org/packages/8d/57/57702d5fa4d343d8dd84353d512b9a603536b9cab4dbe9e11a9efe546c93/multiprocessing_wrap-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "c44f380fa764a2c4e54dea2dec2fd5a9", "sha256": "3dc85bcdbd6080180137e7682b4943da27dd345389b2ce9a2e34fd9c03f43983" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "c44f380fa764a2c4e54dea2dec2fd5a9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6040, "upload_time": "2019-03-17T10:40:05", "url": "https://files.pythonhosted.org/packages/91/c1/78f87a016b8662efcd652f5fe1547a76ff6afbc24b7c1d09cf5b99f04217/multiprocessing_wrap-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "37638c792b30bc716d879e19b97a0626", "sha256": "4329a6207303e61f54627b6d4fbdb7a9de693146573ed82f48872744dea86263" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.2.tar.gz", "has_sig": false, "md5_digest": "37638c792b30bc716d879e19b97a0626", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4031, "upload_time": "2019-03-17T10:40:07", "url": "https://files.pythonhosted.org/packages/51/6e/36baca7e2b08108aa796a86f6224b45a5b81fbb6af8e6c0e8bb752553a37/multiprocessing_wrap-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "b9310c75c808eac05fee886ab05bcd1b", "sha256": "4139db31968493d7e7b1dcff0e98314b3ab9c0f36776446bf42a253bfd890607" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "b9310c75c808eac05fee886ab05bcd1b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4698, "upload_time": "2019-03-18T00:39:39", "url": "https://files.pythonhosted.org/packages/91/3f/9143c8dee6c76c7ddc4827685abe5cf23397988a05e6a8e09a4badb7fe69/multiprocessing_wrap-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b7b9ca4acabb7ebd48ae541eef4fd6f9", "sha256": "4497d841cbcedda2a3f6b4592d94d859c4d1d660145e7f8674dc1581d4ecb934" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.3.tar.gz", "has_sig": false, "md5_digest": "b7b9ca4acabb7ebd48ae541eef4fd6f9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4024, "upload_time": "2019-03-18T00:39:41", "url": "https://files.pythonhosted.org/packages/2d/33/121a72638608bed986c1dbfcc01b0f61f8c2a49bae53f7d40025dd3c619c/multiprocessing_wrap-0.0.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b9310c75c808eac05fee886ab05bcd1b", "sha256": "4139db31968493d7e7b1dcff0e98314b3ab9c0f36776446bf42a253bfd890607" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.3-py3-none-any.whl", "has_sig": false, "md5_digest": "b9310c75c808eac05fee886ab05bcd1b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 4698, "upload_time": "2019-03-18T00:39:39", "url": "https://files.pythonhosted.org/packages/91/3f/9143c8dee6c76c7ddc4827685abe5cf23397988a05e6a8e09a4badb7fe69/multiprocessing_wrap-0.0.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b7b9ca4acabb7ebd48ae541eef4fd6f9", "sha256": "4497d841cbcedda2a3f6b4592d94d859c4d1d660145e7f8674dc1581d4ecb934" }, "downloads": -1, "filename": "multiprocessing_wrap-0.0.3.tar.gz", "has_sig": false, "md5_digest": "b7b9ca4acabb7ebd48ae541eef4fd6f9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4024, "upload_time": "2019-03-18T00:39:41", "url": "https://files.pythonhosted.org/packages/2d/33/121a72638608bed986c1dbfcc01b0f61f8c2a49bae53f7d40025dd3c619c/multiprocessing_wrap-0.0.3.tar.gz" } ] }