{
    "info": {
        "author": "Santiago C Paredes",
        "author_email": "santiago.paredes2012@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "# spychronous\nA simple synchronous job runner for parallel processing tasks.\n\n## spychronous in action\nTL;DR Here's a quick example:\n```python\nfrom spychronous import SynchronousJob as Job\n\ndef get_plus_one(num):\n    return num + 1\n\nnums = [1, 2, 3]\nplus_one_job = Job(func=get_plus_one, items=nums)\n\nprint plus_one_job.run_multi_processed()\n# OUTPUT\n# [2, 3, 4]\n```\n## Working with spychronous\nSay you've written a function that will be repeatedly called to transform a list:\n```python\n>>> def get_plus_one(item):\n...     return item + 1\n...\n>>> items = [1, 2, 3]\n>>> for item in items:\n...     get_plus_one(item)\n...\n2\n3\n4\n```\nUsing a spychronous `Job` instead, you can _parallel process_ the list-transformation (and repeatedly apply your function to each item in your list):\n```python\n>>> from spychronous import SynchronousJob as Job\n>>> plus_one_job = Job(func=get_plus_one, items=items)\n>>> plus_one_job.run_multi_processed()\n[2, 3, 4]\n```\nNow imagine your function changes to require _more_ arguments than simply an item from your list, like a `multiplier`:\n``` python\n>>> from time import sleep\n>>> from random import random\n>>> \n>>> def get_number(item, multiplier):\n...    sleep(random()*5) # your function executes at variable speed\n...    return item * multiplier\n```\n_Notice the first parameter will hold a single `item` from `Job`'s `items` and additional parameters (like `multiplier`) are listed in the signature afterwards._\n\n`Job` accounts for this by using `args`:\n``` python\n>>>\tmultiplier = 2\n>>>\tadditional_function_args = [multiplier]\n>>>\tnumbers_job = Job(func=get_number, items=items, args=additional_function_args)\n>>>\tnumbers_job.run_multi_processed()\n[8, 4, 6]\n```\n_Notice there's no guaranteed order of the output. See [Coupling output to input](#coupling-output-to-input) for tips on working with this behavior._\n#### Important Configurable Features\nIf you need to set process pool size, set worker timeouts, handle Ctrl-C, or run your job single-processed (to debug, for instance), the following features come in handy.\n###### Process Pool Size\n```python\nJob(...processes=20) # default is 4\n```\n###### Worker Timeouts\n```python\nminutes = 60\nJob(...timeout=5*minutes)\n```\n###### Non-Daemonic Process Pool\nThe default `multiprocessing.pool.Pool` disallows spawning processes within processes.  This can be properly circumvented with the `no_daemon` attribute.\n```python\nJob(...no_daemon=True)\n```\n###### Child-Exception Suppression\nIf you don't want a `Job`'s children (i.e. workers) to die if another raises an exception i.e. you want remaining items in your list to be processed, you can suppress those exceptions and log them instead.\n```python\nJob(...suppress_worker_exceptions=True)\n```\n###### Predictable Ctrl-C Handling\n`Job` also handles `SIGINT` gracefully by intentionally killing its workers ASAP, and then killing itself afterwards:\n```python\n>>> from time import sleep, strftime, gmtime\n>>> from random import random\n>>> def print_item(item):\n...     sleep(random()*50)\n...     print strftime(\"%H:%M:%S\", gmtime()), item\n...\n>>> printing_job = Job(func=print_item, items=items)\n>>> items = [1, 2, 3]\n>>> # GOAL: print the following: 1) item and item's timestamp 2) Ctrl-C's timestamp\n>>> try:\n...     printing_job.run_multi_processed()\n... finally:\n...     from time import gmtime, strftime, sleep\n...     print '', strftime(\"%H:%M:%S\", gmtime()), 'user issued Ctrl-C'\n...\n20:39:50 3\n20:39:52 1\n^C 20:41:53 user issued Ctrl-C\nTraceback (most recent call last):\n... Your typical stacktrace here ...\nKeyboardInterrupt\n```\n###### Run Job with Single Process (for dev-ing, debugging, etc.)\nWhen you plan to parallelize a job, it's helpful to develop with single-processed job execution first (and debug likewise) and then switch to multi-processed job execution when you're ready.  The spychronous `Job` can facilitate this with the `run_single_processed` instance method.\n\n_Utilizing run_single_processed: A Use Case..._\n\n* The following example illustrates the aforementioned proposal for development, debugging, and deployment with `run_single_processed`.\n* Illustrated using 3 different iterations of the same program:\n```python\n# 1st iteration: Development\nfrom spychronous import SynchronousJob as Job\ndef get_plus_one(num):\n    return num + 1/0\n\nnums = [1, 2, 3]\njob = Job(func=get_plus_one, items=nums)\njob.run_single_processed()\n# OUTPUT\n# Traceback (most recent call last):\n#   File \"<stdin>\", line 1, in <module>\n#   File \"spychronous.py\", line 33, in run_single_processed\n#     raise e\n# ZeroDivisionError: integer division or modulo by zero\n```\n```python\n# 2nd iteration: Debugging\nfrom spychronous import SynchronousJob as Job\ndef get_plus_one(num):\n    import pdb;pdb.set_trace()\n    return num + 1/0\n\nnums = [1, 2, 3]\njob = Job(func=get_plus_one, items=nums)\njob.run_single_processed()\n# OUTPUT\n# <stdin>(5)get_plus_one()\n# (Pdb) print item\n# 3\n```\n```python\n# 3rd iteration: Multiprocessing the Job\nfrom spychronous import SynchronousJob as Job\ndef get_plus_one(num):\n    return num\n\nnums = [1, 2, 3]\njob = Job(func=get_plus_one, items=nums)\njob.run_multi_processed() # notice method name changed from 'single' to 'multi'\n# OUTPUT\n# [1, 3, 2]\n```\n\n#### Working with instance methods\nIn order to call an instance method on a list of objects, simply wrap the instance method call in a trivial method:\n```python\n>>> class Cat(object):\n...     def __init__(self, name):\n...         self.name = name\n...     def meow(self):\n...         print 'meow, my name is', self.name\n... \n>>> def make_cat_meow(cat):\n...     cat.meow()\n...\n>>> dave = Cat('Dave')\n>>> meow_job = Job(func=make_cat_meow, items=[dave])\n>>> meow_job.run_multi_processed()\nmeow, my name is Dave\n```\n\n#### Coupling output to input\nIn order to preserve a relationship between input and output, simply wrap your function in one that couples the IO:\n```python\n>>> def get_num_times_2(num):\n...     return num * 2\n...\n>>> def double_num(num): # this is the coupling function.\n... \treturn (num, get_num_times_2(num))\n...\n>>> doubling_job = Job(func=double_num, items=[5])\n>>> doubling_job.run_multi_processed()\n[(5, 10)]\n```\n\n## TODO: AsynchronousJob, Thread Pools\nThe next step for spychronous is an asynchrounous job runner.  This is being developed in the develop branch.  After that, thread pools will be implemented as an alternative to process pools.\n\n## Why spychronous?\nI made spychronous because I wanted a clean out-of-the-box solution to quickly replace loops that I wanted to parallel process.  I wanted to hide burdensome configuration and process management from the user. I wanted a solution that would gracefully handle `SIGINT`.\n\nLastly, it was a fun programming exercise for me.\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "https://github.com/scparedes/spychronous/releases/download/1.0.post4/spychronous-1.0.post4.tar.gz",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/scparedes/spychronous",
        "keywords": "multiprocessing,multiprocess,multi,process,synchronous,job,runner,simple",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "spychronous",
        "package_url": "https://pypi.org/project/spychronous/",
        "platform": "",
        "project_url": "https://pypi.org/project/spychronous/",
        "project_urls": {
            "Download": "https://github.com/scparedes/spychronous/releases/download/1.0.post4/spychronous-1.0.post4.tar.gz",
            "Homepage": "https://github.com/scparedes/spychronous"
        },
        "release_url": "https://pypi.org/project/spychronous/1.0.post4/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "A simple synchronous job runner for parallel processing tasks in Python.",
        "version": "1.0.post4"
    },
    "last_serial": 5368229,
    "releases": {
        "1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "86822a17bb86d5e6ff3948bd53464387",
                    "sha256": "48be874b2caa7b70eb964e494a77dbbc1daf535bfcea672c644ec2ec3d8f78ee"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "86822a17bb86d5e6ff3948bd53464387",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2764,
                "upload_time": "2019-06-03T02:27:44",
                "url": "https://files.pythonhosted.org/packages/c5/70/ddfdfd8982b09a40af32b94b02e5e2b82ab6e4c2d158c2b11381839d4f23/spychronous-1.0.tar.gz"
            }
        ],
        "1.0.post1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "36a8ee654a7ff27fe47ddd7d59954287",
                    "sha256": "f2d99415f972f62ef3f5e538898dfa5073f7aca86b0a208cf552d11c55bc557f"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post1-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "36a8ee654a7ff27fe47ddd7d59954287",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 6744,
                "upload_time": "2019-06-04T13:29:30",
                "url": "https://files.pythonhosted.org/packages/34/03/9b61a449d781433ee24b700a736a5a10fbb7156f08d06a0040160e22d210/spychronous-1.0.post1-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8645c80014f9ffb773142ae715de61ea",
                    "sha256": "8535ff8e91420233731284310c03d76024e2677d3c02641f87d8c97a08edbcd8"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post1.tar.gz",
                "has_sig": false,
                "md5_digest": "8645c80014f9ffb773142ae715de61ea",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2769,
                "upload_time": "2019-06-03T02:50:13",
                "url": "https://files.pythonhosted.org/packages/83/36/663cf503bc45f3b1c4c45cfb5278436d32b40a567c7c1f46fcc1610db75e/spychronous-1.0.post1.tar.gz"
            }
        ],
        "1.0.post2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0167729605fe45c004ac8ca6cc530ad7",
                    "sha256": "43a5401360f3a73f7f0e259bc3bd1b3e33d339a70577a1bf64aeb88fab1fec91"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post2-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "0167729605fe45c004ac8ca6cc530ad7",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 6749,
                "upload_time": "2019-06-04T13:29:32",
                "url": "https://files.pythonhosted.org/packages/f3/8b/50302db67deec0246468074682d9f469160ff94b63cb3b77fc3d890e4bad/spychronous-1.0.post2-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0806cc6e3dadb3e460478e8818f329e5",
                    "sha256": "58303d6b7be9166ab0cef96cf9144bbf2446dd597827a6edec4c959794cb16fe"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post2.tar.gz",
                "has_sig": false,
                "md5_digest": "0806cc6e3dadb3e460478e8818f329e5",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10163,
                "upload_time": "2019-06-04T13:36:48",
                "url": "https://files.pythonhosted.org/packages/e6/14/bd04aedeefcca55334ef9f8a1415de701cabf45f052174f83e2dd9584ac2/spychronous-1.0.post2.tar.gz"
            }
        ],
        "1.0.post3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3f9515c3b0f53243fa72f3dd58d32119",
                    "sha256": "b3a7b43714950a44cec535e618a7892a51182dc4cd0b542d1320c319130fbb22"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post3-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "3f9515c3b0f53243fa72f3dd58d32119",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 6757,
                "upload_time": "2019-06-04T15:02:41",
                "url": "https://files.pythonhosted.org/packages/18/14/69ef174d5e28ed692aa5009eaba8b336bc2d48e0f2ec2a7e1d335c65535b/spychronous-1.0.post3-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "09d17ce8e1f82441db42ed11eda901d4",
                    "sha256": "3228d9fded1ab51df79899ec2912576a3a7f3cd1d69fb2b279f243c115adac68"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post3.tar.gz",
                "has_sig": false,
                "md5_digest": "09d17ce8e1f82441db42ed11eda901d4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10161,
                "upload_time": "2019-06-04T15:02:42",
                "url": "https://files.pythonhosted.org/packages/e8/70/b14769dccb44afcca6cef111f58b411469c95e08e41e9431dd8a0b1d0816/spychronous-1.0.post3.tar.gz"
            }
        ],
        "1.0.post4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a855a2c77b9988645a09570d47e243aa",
                    "sha256": "fa53ca100e0b425e3c1175df891a6e109ab0c4380207bc9598b9c03e2b9f6153"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post4-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "a855a2c77b9988645a09570d47e243aa",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 7649,
                "upload_time": "2019-06-06T17:13:25",
                "url": "https://files.pythonhosted.org/packages/0e/2f/bfc0a2218d138fe1acd4e49f1ee9d2a31c5f986421237fc3a337c88594d0/spychronous-1.0.post4-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "cd4488af663c2e1612c25caa76bb0a10",
                    "sha256": "196cab6fba098d83bed36eb6f226427909837db7a69f50ad66611dfb60862905"
                },
                "downloads": -1,
                "filename": "spychronous-1.0.post4.tar.gz",
                "has_sig": false,
                "md5_digest": "cd4488af663c2e1612c25caa76bb0a10",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10272,
                "upload_time": "2019-06-06T17:13:28",
                "url": "https://files.pythonhosted.org/packages/f6/cf/eb1ea055cf872967e22eaa5a38381f59112f3bdffc2b24ea89f2c6bbe330/spychronous-1.0.post4.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "a855a2c77b9988645a09570d47e243aa",
                "sha256": "fa53ca100e0b425e3c1175df891a6e109ab0c4380207bc9598b9c03e2b9f6153"
            },
            "downloads": -1,
            "filename": "spychronous-1.0.post4-py2-none-any.whl",
            "has_sig": false,
            "md5_digest": "a855a2c77b9988645a09570d47e243aa",
            "packagetype": "bdist_wheel",
            "python_version": "py2",
            "requires_python": null,
            "size": 7649,
            "upload_time": "2019-06-06T17:13:25",
            "url": "https://files.pythonhosted.org/packages/0e/2f/bfc0a2218d138fe1acd4e49f1ee9d2a31c5f986421237fc3a337c88594d0/spychronous-1.0.post4-py2-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "cd4488af663c2e1612c25caa76bb0a10",
                "sha256": "196cab6fba098d83bed36eb6f226427909837db7a69f50ad66611dfb60862905"
            },
            "downloads": -1,
            "filename": "spychronous-1.0.post4.tar.gz",
            "has_sig": false,
            "md5_digest": "cd4488af663c2e1612c25caa76bb0a10",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10272,
            "upload_time": "2019-06-06T17:13:28",
            "url": "https://files.pythonhosted.org/packages/f6/cf/eb1ea055cf872967e22eaa5a38381f59112f3bdffc2b24ea89f2c6bbe330/spychronous-1.0.post4.tar.gz"
        }
    ]
}