{
    "info": {
        "author": "Shaun Viguerie",
        "author_email": "shaunvig114@gmail",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# pytchfork [![PyPI version](https://badge.fury.io/py/pytchfork.svg)](https://badge.fury.io/py/pytchfork) [![Build Status](https://travis-ci.org/shaunvxc/pytchfork.svg?branch=master)](https://travis-ci.org/shaunvxc/pytchfork) [![Coverage Status] (https://coveralls.io/repos/shaunvxc/pytchfork/badge.svg?branch=master&service=github)](https://coveralls.io/github/shaunvxc/pytchfork)\n\nPytchfork simplifies working with python's multiprocessing package.  By abstracting away the common boilerplate associated with forking processes and managing queues, pytchfork will allow you to write cleaner and more readable multiprocessing code.\n\n#Usage\n###Decorator\nYou can easily mark methods to be run using multiple processes by invoking the pytchfork decorator:\n```python\nfrom pytchfork import pytchfork\nfrom multiprocessing import Queue\n\n@pytchfork(3)\ndef do_work(queue):\n  data = queue.get()\n  process(data)\n\nqueue = Queue()\n...\ndo_work(queue) # this call will fork 3 processes\n```\nPytchfork can also manage queues for worker processes.  Just provide the necessary references to the decorator and\nit will take care of polling the queue and passing data to the workers.\n\n```python\nfrom pytchfork import pytchfork\nfrom multiprocessing import Queue\n\n@pytchfork(3, read_from=work_queue, write_to=done_queue, sentinel=\"DONE\")\ndef process_data(data):\n  processed_data = do_something(data)\n  return processed_data\n\nprocess_data() # this call will fork 3 processes that read from work_queue & write to done_queue.\n```\n####Redis\nPytchfork processes can also be configured to read from and write to Redis instances (currently only Redis Lists are supported).  To do so, simply pass the `redis_uri` and `redis_port` to the pytchfork decorator (In addition to string values Redis will use to key the `work_queue` and `done_queue`).  \n\n```python\nfrom pytchfork import pytchfork\n\n@pytchfork(2, read_from=\"work_queue\", write_to=\"done_queue\", redis_uri='localhost', redis_port=6379)\ndef process_data(data):\n  processed_data = do_something(data)\n  return processed_data\n\nprocess_data() # this will fork 2 processes that read from/write to a local redis instance   \n```\n\nIn the above snippet, these processes will run continuously as daemons.  For smaller tasks with fixed amounts of input data, this might not be desirable.\n\nTo get the processes to exit upon completetion, pass the `sentinel` argument to the decorator. In order for this to work, the `redis_work_queue` must clearly mark the ending with `N` occurrences of the `sentinel`, where `N` is the desired number of processes. \n\nBelow is an example (verbosity for clarity):\n\n```python\nfrom pytchfork import pytchfork\nimport redis\n\nredis_client = redis.StrictRedis(host='localhost', port=6379)\nfill_redis_with_work_tasks(redis_client, \"work_queue\")\n\nnum_procs = 2\nsentinel = \"DONE\"\n# mark ending here:\nfor x in range(0, num_procs):\n  redis_client.lpush(\"work_queue\", sentinel)\n\n# provide the sentinel to the decorator\n@pytchfork(num_procs, read_from=\"work_queue\", write_to=\"done_queue\", sentinel=sentinel, redis_uri=uri, redis_port=port)\ndef process_data(data):\n  processed_data = do_something(data)\n  return processed_data\n\nprocess_data() # this will fork 2 processes that read/write to redis. Each process will\n               # exit upon dequeueing a sentinel value from the redis work queue\n```\nFor further reference on this, see the `test_redis()` method in `tests/test_decorator.py`.\n\n###Context Manager\nYou can also use the context manager to get hold of a multiprocessing.Pool object, without having to manage the lifecycle of the pool.  I.e.:\n\n```python\nfrom pytchfork import pytchfork\n    ...\n    with pytchfork(num_procs) as forked:\n        res = forked.map_async(process_data, data, callback=callback)\n```\n\nThis construct ensures that the worker processes will be closed, joined and terminated upon the completion of the code in the block. \n\n##Installation\n *Requirements*: Python >= 2.7\n\n`$ pip install pytchfork`\n\n## Contributing\n1. Fork it ( https://github.com/shaunvxc/pytchfork/fork )\n1. Create your feature branch (`git checkout -b new-feature`)\n1. Commit your changes (`git commit -am 'Add some feature'`)\n1. Run the tests (`make test`)\n1. Push change to the branch (`git push origin new-feature`)\n1. Create a Pull Request\n\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/shaunvxc/pytchfork",
        "keywords": null,
        "license": "MIT",
        "maintainer": null,
        "maintainer_email": null,
        "name": "pytchfork",
        "package_url": "https://pypi.org/project/pytchfork/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/pytchfork/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/shaunvxc/pytchfork"
        },
        "release_url": "https://pypi.org/project/pytchfork/0.0.2/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "multiprocessing decorator with queue management & redis features",
        "version": "0.0.2"
    },
    "last_serial": 1962693,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "49413721fb8435ef28ac771a1c3cd7d0",
                    "sha256": "4db2a49d91c657ceb3f05ee28ba6491517e4b953d8723f6e11957b83bf639d6d"
                },
                "downloads": -1,
                "filename": "pytchfork-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "49413721fb8435ef28ac771a1c3cd7d0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3354,
                "upload_time": "2016-02-18T03:52:55",
                "url": "https://files.pythonhosted.org/packages/3e/5c/0cc77c8142ed016a8b3ba909c76b29b9db03bccdf160090bb1147e8d0720/pytchfork-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cbc585fa6271fb76586255fbb6ac90bb",
                    "sha256": "903af234b2bacbad76149b1f0584995e8eee8bd109fb1ba826e939a4b68dacd6"
                },
                "downloads": -1,
                "filename": "pytchfork-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "cbc585fa6271fb76586255fbb6ac90bb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5662,
                "upload_time": "2016-02-18T06:49:56",
                "url": "https://files.pythonhosted.org/packages/5d/e7/7e590316c73aa8ad8cd51314a71e399a67e81421fc6c90f4da80e2e5abbb/pytchfork-0.0.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "cbc585fa6271fb76586255fbb6ac90bb",
                "sha256": "903af234b2bacbad76149b1f0584995e8eee8bd109fb1ba826e939a4b68dacd6"
            },
            "downloads": -1,
            "filename": "pytchfork-0.0.2.tar.gz",
            "has_sig": false,
            "md5_digest": "cbc585fa6271fb76586255fbb6ac90bb",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 5662,
            "upload_time": "2016-02-18T06:49:56",
            "url": "https://files.pythonhosted.org/packages/5d/e7/7e590316c73aa8ad8cd51314a71e399a67e81421fc6c90f4da80e2e5abbb/pytchfork-0.0.2.tar.gz"
        }
    ]
}