{ "info": { "author": "Hagai Helman Tov", "author_email": "hagai.helman@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6" ], "description": "# meanwhile - Very Easy Multithreading\n\nIf you want to call the same function on many inputs, in a multithreaded \nfashion, ```meanwhile``` makes it easy.\n\nIt can make your code significantly faster, especially if the function requires\nI/O operations, like file access or HTTP(S) queries.\n\n\n## Installation\n\n```bash\npip3 install meanwhile\n```\n\n\n## Simple Usage Example\n\nSuppose you have a function named ```test_url```, that gets a URL, downloads\nits content, and tests whether it contains the word \"meanwhile\". Also,\nsuppose you have a file, named ```urls.txt```, where each line contains a URL\nyou would like to apply ```test_url``` to.\n\nYou can do the following:\n\n```python\n>>> from meanwhile import Job\n>>> job = Job(test_url, 10) # at most 10 threads will be used concurrently.\n>>> urls = open(\"urls.txt\").read().splitlines()\n>>> job.add_many(urls)\n>>> job.wait()\n>>> results = job.get_results()\n```\n\nThe target function (in this case: ```test_url```) should get one argument, and\nthis argument should be hashable.\n\nNote that if your function prints output, you probably want to use \n```meanwhile.print()``` instead of Python's built-in ```print()``` function.\nThis function prevents conflicts both with other threads, and with the progress\nupdates shown by the ```wait``` method.\n\n\n## In More Details\n\nThe ```Job``` object holds a queue of inputs to process. It automatically \nspawns and kills threads, as needed (up to the maximal number of concurrent \nthreads set by the user).\n\nThe methods ```add``` and ```add_many``` can be used to add inputs to the queue.\n\nThe method ```wait``` stops until the queue is empty. By default, it shows the\njob's progress, like this:\n\n```\npending: 19\t running: 20\t finished: 11\t failed: 0\n```\n\nIn order to ```wait``` without showing the progress, one can set the keyword\nargument ```show_status``` to be ```False```. Also, it's possible to show the\njob's progress without ```wait```-ing, using the method ```print_status```.\n\n```wait``` also supports a keyword argument, ```timeout```. If it is set, the\nmethod will unconditionally return after ```timeout``` seconds. ```wait``` can\nalso be stopped safely by a ```KeyboardInterrupt```.\n\nThe return values of the target function can be inspected using the methods\n```get_result```, ```has_result``` and ```get_results```.\n\n\n## Fix Mistakes On The Fly\n\n```meanwhile``` also makes it easy to debug your code and fix it while the job\nis already in progress.\n\nFirst, almost everything can be changed at any time. For example: \n\n* You can always add new inputs to the queue using ```add``` and ```add_many``` \n(even after all previous inputs were already processed, and all threads were \nkilled); \n* You can always change the maximal number of threads allowed to run\n concurrently using the method ```set_n_threads```;\n* You can always change the target function using the method ```set_target```\n (this will apply only to inputs that weren't successfully processed yet).\n\nAlso, you can inspect the exceptions raised by the target function using the\nmethod ```get_exceptions``` (and also ```has_exception``` and\n```get_exception```).\n\nAfter you fix the cause for the exceptions, you can use the methods ```retry```,\n```retry_many``` and ```retry_all``` to return inputs that raised exceptions\ninto the job's queue.\n\nIn case your target function sometimes randomly fails (i.e. raises exception),\nyou can also use the method ```set_n_attempts```, to make the job retry inputs\nautomatically for a limited number of attempts (and note that the ```Job``` \nclass'es constructor also can take the keyword argument ```n_attempts```).\n\nFinally, note the methods ```pause```, ```resume``` and ```kill```, that also\ncan be useful when you debug a job in progress (don't be afraid of ```kill```:\nit is just equivalent to ```set_n_threads(0)```).\n\n\n## Resource Reuse\n\nSometimes it's useful to have a thread reusing resources whlie sequentially \nprocessing inputs. For example, if your target function makes an HTTPS request,\nyou may want to reuse the same session, in order to save the TLS handshake time.\n\nThat's why ```meanwhile``` allows you to provide a target *factory* instead of a\ntarget function. The factory is used to create a different target function for \neach thread spawned.\n\nA target factory can be either a function that does not take arguments, and\nreturns a target function, or a callable class (that is initialized without\narguments, and its ```__call__``` method is used as the target function.\n\nIn order to provide a target factory instead of target function, one must set \nthe keyword argument ```factory``` to be ```True``` (this is true for both the\n```Job``` class constructor and the ```set_target``` method).\n\n\n## Module Reference\n\nFor the full module reference, see ```help(meanwhile)```.", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/hagai-helman/meanwhile", "keywords": "threading,threads,multithreading", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "meanwhile", "package_url": "https://pypi.org/project/meanwhile/", "platform": "", "project_url": "https://pypi.org/project/meanwhile/", "project_urls": { "Homepage": "https://github.com/hagai-helman/meanwhile" }, "release_url": "https://pypi.org/project/meanwhile/1.1.0/", "requires_dist": null, "requires_python": "~=3.6", "summary": "Very easy multithreading", "version": "1.1.0" }, "last_serial": 5491816, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "c852ff9903d57443eeebe44d9aadd54d", "sha256": "bf8af332fdc217c943b992eacf389d4c641f6c6c99bc57a13eda4c82d697e03e" }, "downloads": -1, "filename": "meanwhile-1.0.0.tar.gz", "has_sig": false, "md5_digest": "c852ff9903d57443eeebe44d9aadd54d", "packagetype": "sdist", "python_version": "source", "requires_python": "~=3.6", "size": 5618, "upload_time": "2019-05-25T15:36:34", "url": "https://files.pythonhosted.org/packages/07/7a/39936b93b9478bc68959737b97fbb105b6325a1083df7846ac0adc03b457/meanwhile-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "8792a6295992d1d9dc03cb5d97ee9773", "sha256": "2cdf575c764db33159d43fe616d31b119c0df52ae8687b34d00747210f329885" }, "downloads": -1, "filename": "meanwhile-1.0.1.tar.gz", "has_sig": false, "md5_digest": "8792a6295992d1d9dc03cb5d97ee9773", "packagetype": "sdist", "python_version": "source", "requires_python": "~=3.6", "size": 5707, "upload_time": "2019-05-25T18:59:25", "url": "https://files.pythonhosted.org/packages/17/6c/b7210410aa0e4e50239d287ae8462c4d337d44954a99823e00fef8b92f97/meanwhile-1.0.1.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "d163304efdcc4bd8b4a1a1d5b116f831", "sha256": "00ad2c15861cd9398968d4979ec6d656be86bc42b3ce48a9ecb475643a2dc221" }, "downloads": -1, "filename": "meanwhile-1.1.0.tar.gz", "has_sig": false, "md5_digest": "d163304efdcc4bd8b4a1a1d5b116f831", "packagetype": "sdist", "python_version": "source", "requires_python": "~=3.6", "size": 7388, "upload_time": "2019-07-05T14:45:08", "url": "https://files.pythonhosted.org/packages/bb/f1/8c4b8faf55969bb3556e4bf4e80182872251af7a377f53492815813efbb5/meanwhile-1.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d163304efdcc4bd8b4a1a1d5b116f831", "sha256": "00ad2c15861cd9398968d4979ec6d656be86bc42b3ce48a9ecb475643a2dc221" }, "downloads": -1, "filename": "meanwhile-1.1.0.tar.gz", "has_sig": false, "md5_digest": "d163304efdcc4bd8b4a1a1d5b116f831", "packagetype": "sdist", "python_version": "source", "requires_python": "~=3.6", "size": 7388, "upload_time": "2019-07-05T14:45:08", "url": "https://files.pythonhosted.org/packages/bb/f1/8c4b8faf55969bb3556e4bf4e80182872251af7a377f53492815813efbb5/meanwhile-1.1.0.tar.gz" } ] }