{
    "info": {
        "author": "David Cramer",
        "author_email": "dcramer@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Environment :: Console",
            "Intended Audience :: Developers",
            "Programming Language :: Python",
            "Topic :: Software Development",
            "Topic :: Utilities"
        ],
        "description": "Taskmaster\n----------\n\nTaskmaster is a simple distributed queue designed for handling large numbers of one-off tasks.\n\nWe built this at DISQUS to handle frequent, but uncommon tasks like \"migrate this data to a new schema\".\n\nWhy?\n----\n\nYou might ask, \"Why not use Celery?\". Well the answer is simply that normal queueing requires (not literally,\nbut it'd be painful without) you to buffer all tasks into a central location. This becomes a problem when you\nhave a large amount of tasks, especially when they contain a large amount of data.\n\nImagine you have 1 billion tasks, each weighing in at 5k. Thats, uncompressed, at minimum 4 terabytes of storage\nrequired just to keep that around, and gains you very little.\n\nTaskmaster on the other hand is designed to take a resumable iterator, and only pull in a maximum number of\njobs at a time (using standard Python Queue's). This ensures a consistent memory pattern that can scale linearly.\n\nRequirements\n------------\n\nRequirements **should** be handled by setuptools, but if they are not, you will need the following Python packages:\n\n* progressbar\n* pyzmq (zeromq)\n* gevent\n* gevent_zeromq\n\n\nA note on Gevent\n----------------\n\nBeing that Taskmaster uses gevent for both its iterator task (master) and its consumers, your application will need\nto correctly implement non-blocking gevent compatible callers. In most cases this won't be a problem, but if you're\nusing the network you'll need to look for a compatible library for your adapter. For example, there is an alternative\nversion of ``psycopg2`` designed for gevent called ``gevent-psycopg2``.\n\nUsage\n-----\n\nCreate an iterator, and callback::\n\n    # taskmaster/example.py\n    def get_jobs(last=0):\n        # last would be sent if state was resumed\n        # from a previous run\n        for i in xrange(last, 100000000):\n            # jobs yielded must be serializeable with pickle\n            yield i\n\n    def handle_job(i):\n        # this **must** be idempotent, as resuming the process may execute a job\n        # that had already been run\n        print \"Got %r!\" % i\n\n\nSpawn a master::\n\n    $ tm-master taskmaster.example\n\nYou can also pass keyword arguments for the master::\n\n    $ tm-master taskmaster.example argument=value\n\nSpawn a slave::\n\n    $ tm-slave taskmaster.example\n\nOr spawn 8 slaves (each containing a threadpool)::\n\n    $ tm-spawn taskmaster.example 8\n\nDont like the magical function discover for master/slave? Specify your own targets::\n\n    $ tm-master taskmaster.example:get_jobs\n    $ tm-slave taskmaster.example:handle_job\n\nMaybe you simply need to run things on the same server?\n\n::\n\n    $ tm-run taskmaster/example.py 8\n\n.. note:: All arguments are optional, and the address will default to ``tcp://0.0.0.0:3050``.",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/dcramer/taskmaster",
        "keywords": null,
        "license": "Apache License 2.0",
        "maintainer": null,
        "maintainer_email": null,
        "name": "taskmaster",
        "package_url": "https://pypi.org/project/taskmaster/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/taskmaster/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/dcramer/taskmaster"
        },
        "release_url": "https://pypi.org/project/taskmaster/0.8.2/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "UNKNOWN",
        "version": "0.8.2"
    },
    "last_serial": 844780,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7c742aa2b3a5478a4f40f42cfdd5b31b",
                    "sha256": "9876bbf8528d68d269a779989950b5a6a28218c9e9ee9d562e737ccf72ab64fb"
                },
                "downloads": -1,
                "filename": "taskmaster-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "7c742aa2b3a5478a4f40f42cfdd5b31b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3971,
                "upload_time": "2012-05-04T23:41:53",
                "url": "https://files.pythonhosted.org/packages/c6/22/073444243756bde392631e4b0bbdbaebde69fd778faa3a04787985c49002/taskmaster-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ee1a5f0b48a77c9c8afbd655930ac61a",
                    "sha256": "81f0da9fd8d78baab043f460b8ae68514783a902109fa26907226983cd7c63ef"
                },
                "downloads": -1,
                "filename": "taskmaster-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ee1a5f0b48a77c9c8afbd655930ac61a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4406,
                "upload_time": "2012-05-05T00:35:38",
                "url": "https://files.pythonhosted.org/packages/fd/b0/edf82fd65acca95ee32591e0db9ca2361e8ac4b2b4c3b955ae98405fde5d/taskmaster-0.2.0.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b924c6cd0a709bc1d2119810b21e7046",
                    "sha256": "2e089c0c096bbda91d831ccab32b81ceb1d544c9c9866b0557b6aca9ed64bfbb"
                },
                "downloads": -1,
                "filename": "taskmaster-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "b924c6cd0a709bc1d2119810b21e7046",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4518,
                "upload_time": "2012-05-05T01:15:31",
                "url": "https://files.pythonhosted.org/packages/93/ce/fc0dbc647a8c719fc54a5bd28633e6788525612ad0b51ead24a1a697c3cb/taskmaster-0.3.0.tar.gz"
            }
        ],
        "0.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e082930e4ff6020bd0ed720a9e913953",
                    "sha256": "437fffe2ea281144450f4777956103ca41d7b58bca9658c7956597b2bb74ae16"
                },
                "downloads": -1,
                "filename": "taskmaster-0.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "e082930e4ff6020bd0ed720a9e913953",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4565,
                "upload_time": "2012-05-07T21:18:07",
                "url": "https://files.pythonhosted.org/packages/e8/ea/89ef69c1fc327606f98232e93a32a8b0b9887ea9b668bdb48561fff66ad0/taskmaster-0.3.1.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ba5ebd6f8911785eaa2a579e06e92a7c",
                    "sha256": "461b6a44e72f7c564d2a1c31040b08c1d874a23d5698481e88bd44f7996cc133"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ba5ebd6f8911785eaa2a579e06e92a7c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4857,
                "upload_time": "2012-05-15T02:54:52",
                "url": "https://files.pythonhosted.org/packages/50/55/2995f5012013497a87769809558e7894d5e2a9d398de1a160e9a86b9f720/taskmaster-0.4.0.tar.gz"
            }
        ],
        "0.4.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "03b574f4a13c14f591091f4bf56f0f5f",
                    "sha256": "02b32903a51371128483b593fbe1f0fb9697aa2764c9808923012ded85745897"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.1.tar.gz",
                "has_sig": false,
                "md5_digest": "03b574f4a13c14f591091f4bf56f0f5f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4867,
                "upload_time": "2012-05-15T04:00:42",
                "url": "https://files.pythonhosted.org/packages/68/31/2452842ecdfb0d4cdc32e97524cb9c55ab52094c51050c132ec52e6df8cc/taskmaster-0.4.1.tar.gz"
            }
        ],
        "0.4.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9741c2a1369882d69b7476d0448c6826",
                    "sha256": "36fb2285b9a862251968943af89a762715e8a593be7fe13e0f9d92d1e3db1af1"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.2.tar.gz",
                "has_sig": false,
                "md5_digest": "9741c2a1369882d69b7476d0448c6826",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4871,
                "upload_time": "2012-05-15T04:10:04",
                "url": "https://files.pythonhosted.org/packages/34/c1/67c4494b7a78d75d51f374bdfee6019fc51688097ad70d4c54f2c6994cd8/taskmaster-0.4.2.tar.gz"
            }
        ],
        "0.4.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d2618159378ca8bbeaf993ae6b6e4a3d",
                    "sha256": "ab2929bc9a0797eb5dafd81fb8414c9e7dcb0091450b51d1d2175ab295998168"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.3.tar.gz",
                "has_sig": false,
                "md5_digest": "d2618159378ca8bbeaf993ae6b6e4a3d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4868,
                "upload_time": "2012-05-15T04:11:26",
                "url": "https://files.pythonhosted.org/packages/15/83/d3ab0b361e3d03f75de2bce66ec85c3e70523019f013260d7f26f3de187c/taskmaster-0.4.3.tar.gz"
            }
        ],
        "0.4.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0133001c1ff08e9e9281d264af61f252",
                    "sha256": "31f8e3e805f086a885515d51c3ca8ee96df3b378336bd5867f417faa8302c267"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.4.tar.gz",
                "has_sig": false,
                "md5_digest": "0133001c1ff08e9e9281d264af61f252",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5027,
                "upload_time": "2012-05-15T21:21:32",
                "url": "https://files.pythonhosted.org/packages/2f/96/80f3531738827aec6e68e7beb4a66b150d0b78336c333074196b54cac42e/taskmaster-0.4.4.tar.gz"
            }
        ],
        "0.4.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0bf23dc0681bb20869d1dd7c01bea942",
                    "sha256": "5f7ded262b1e30687ea1b66bc08e7638c921ab16d65b3766221929a290a63db7"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.5.tar.gz",
                "has_sig": false,
                "md5_digest": "0bf23dc0681bb20869d1dd7c01bea942",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5069,
                "upload_time": "2012-05-15T21:30:42",
                "url": "https://files.pythonhosted.org/packages/e5/00/2d378a9d27ea216eea274c2687066364940f3b4c1cac7ff485cceec57dfb/taskmaster-0.4.5.tar.gz"
            }
        ],
        "0.4.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "41a78f7d614055957adf6cd4e09902d1",
                    "sha256": "2fd3cf22c35a3f88e30d95e2e3f216136f94b7ef37244ffc50292306ed8eeb6d"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.6.tar.gz",
                "has_sig": false,
                "md5_digest": "41a78f7d614055957adf6cd4e09902d1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5057,
                "upload_time": "2012-05-15T21:32:56",
                "url": "https://files.pythonhosted.org/packages/11/7e/b82048608eca9c94dba79e30667e4e041dfef73416aac09730e29f85cb87/taskmaster-0.4.6.tar.gz"
            }
        ],
        "0.4.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "bb7273f514d354b46ecca22ef8277345",
                    "sha256": "699c707bf725fe65244cb8122dc080c675022474b6750a73074efb8bddecb496"
                },
                "downloads": -1,
                "filename": "taskmaster-0.4.7.tar.gz",
                "has_sig": false,
                "md5_digest": "bb7273f514d354b46ecca22ef8277345",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5050,
                "upload_time": "2012-05-15T21:52:51",
                "url": "https://files.pythonhosted.org/packages/55/e2/415248a79d12a802404299b4b51c028d816d33f64a5221fe4764b6cb90b9/taskmaster-0.4.7.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ed1372d70d1737bbfbfe57215110ad04",
                    "sha256": "cf78328b7be4958e3da29303605accdff2f909fba51062621c240a824c0bf53c"
                },
                "downloads": -1,
                "filename": "taskmaster-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ed1372d70d1737bbfbfe57215110ad04",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5368,
                "upload_time": "2012-05-16T02:14:54",
                "url": "https://files.pythonhosted.org/packages/c4/2f/f70510368f39fe260f3068bf572900a86998ba34b7fc4ce4dc63f720e299/taskmaster-0.5.0.tar.gz"
            }
        ],
        "0.5.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6a12f464bc111a1baeda716a6559e089",
                    "sha256": "659aa8c97ab3e7ab609c80d0c08ef737616fb4a1e43123b22786a72015448764"
                },
                "downloads": -1,
                "filename": "taskmaster-0.5.2.tar.gz",
                "has_sig": false,
                "md5_digest": "6a12f464bc111a1baeda716a6559e089",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5353,
                "upload_time": "2012-05-16T02:19:57",
                "url": "https://files.pythonhosted.org/packages/43/b3/0a205f64d089a15b148434f64994942e32cc7cc7ab97ef407be3d0c13170/taskmaster-0.5.2.tar.gz"
            }
        ],
        "0.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "83a9c88d76f596071a7a2c81268737eb",
                    "sha256": "8b5036c6ff70709fc238dae4314c34bdcf68030c21b325b1d5d55d58e8f3addf"
                },
                "downloads": -1,
                "filename": "taskmaster-0.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "83a9c88d76f596071a7a2c81268737eb",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6712,
                "upload_time": "2012-08-07T22:53:23",
                "url": "https://files.pythonhosted.org/packages/b3/f6/42951bf87141e3f0e3bca5dde765d0b813beea35c5c92fd99ef34652110a/taskmaster-0.6.0.tar.gz"
            }
        ],
        "0.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad1ab4bf6469cb8a84e1f9f25a5885e4",
                    "sha256": "f3ea6806ca9fe5a259af69fd121a1be133b9d05a671ba1d663961119ebc6503c"
                },
                "downloads": -1,
                "filename": "taskmaster-0.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ad1ab4bf6469cb8a84e1f9f25a5885e4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7543,
                "upload_time": "2012-11-06T22:04:09",
                "url": "https://files.pythonhosted.org/packages/f5/0d/22d784cd758f271bd4b1836ea2fc9bf6463370e42f9d6c6127d0b7865792/taskmaster-0.7.0.tar.gz"
            }
        ],
        "0.7.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b6d4aa827014869c1c4e5f1e0361753c",
                    "sha256": "59a4d0d688e4ca1468f5180a4366ea685d57c8f7a01423270a2ce6ae32a6f891"
                },
                "downloads": -1,
                "filename": "taskmaster-0.7.1.tar.gz",
                "has_sig": false,
                "md5_digest": "b6d4aa827014869c1c4e5f1e0361753c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7645,
                "upload_time": "2012-11-06T22:50:01",
                "url": "https://files.pythonhosted.org/packages/e7/14/3aafb4360f3ab2825ac9448a9da0316b49a9997a7fa2342b4f565e080b27/taskmaster-0.7.1.tar.gz"
            }
        ],
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "590f542e2099821461c08be226255174",
                    "sha256": "e51c18d44458a4a13d2a777c78844bd3469bec9554124bd4c817beb2661e43c9"
                },
                "downloads": -1,
                "filename": "taskmaster-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "590f542e2099821461c08be226255174",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7920,
                "upload_time": "2013-01-08T21:37:38",
                "url": "https://files.pythonhosted.org/packages/dc/5e/e100a7f3698a809b4c8a43494cb3d5a4403ab58a85f2330e031c7b3fd4f5/taskmaster-0.8.0.tar.gz"
            }
        ],
        "0.8.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6ec3f995398b7fbbcee3b4e3581a4117",
                    "sha256": "f846dba3d5ce0d769651c8db10c4a2ce99ec646e5a78667cdfcd342630201653"
                },
                "downloads": -1,
                "filename": "taskmaster-0.8.1.tar.gz",
                "has_sig": false,
                "md5_digest": "6ec3f995398b7fbbcee3b4e3581a4117",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7921,
                "upload_time": "2013-01-11T01:12:05",
                "url": "https://files.pythonhosted.org/packages/d6/d8/f5b2295bbb5894c7e729b27acc1c5d9e85814b1332745a6a07ce17d511d3/taskmaster-0.8.1.tar.gz"
            }
        ],
        "0.8.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "75896f73ece9fc3f0028bec574df91e3",
                    "sha256": "3aadb62af79bef317f6edc16b13d116a71c48835769530772676501042746aaa"
                },
                "downloads": -1,
                "filename": "taskmaster-0.8.2.tar.gz",
                "has_sig": false,
                "md5_digest": "75896f73ece9fc3f0028bec574df91e3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 7993,
                "upload_time": "2013-08-21T05:01:39",
                "url": "https://files.pythonhosted.org/packages/ff/56/e3a3cfcd2e4ad48f6af3c63ec9b0c1417d0ea7c3db247f7e1bc96aba806a/taskmaster-0.8.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "75896f73ece9fc3f0028bec574df91e3",
                "sha256": "3aadb62af79bef317f6edc16b13d116a71c48835769530772676501042746aaa"
            },
            "downloads": -1,
            "filename": "taskmaster-0.8.2.tar.gz",
            "has_sig": false,
            "md5_digest": "75896f73ece9fc3f0028bec574df91e3",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 7993,
            "upload_time": "2013-08-21T05:01:39",
            "url": "https://files.pythonhosted.org/packages/ff/56/e3a3cfcd2e4ad48f6af3c63ec9b0c1417d0ea7c3db247f7e1bc96aba806a/taskmaster-0.8.2.tar.gz"
        }
    ]
}