{
    "info": {
        "author": "Tim Savannah",
        "author_email": "kata198@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 6 - Mature",
            "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.5",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Topic :: System :: Distributed Computing"
        ],
        "description": "disttask\n========\n\nDisttask is a utility which provides the ability to distribute a task across a fixed number of processes and collate the output, for better utilization of multiprocessing.\n\nUse it with existing single-threaded/process tools and scripts to take full advantage of your computer's resources.\n\nProvides the ability to distribute a task across a fixed number of processes, for better utilization of multiprocessing.\n\n**Usage**\n\n\tUsage: disttask [cmd] [concurrent tasks] [argset]\n\n\n\tUse a %s in [cmd] where you want the args to go. use %d for the pipe number.\n\n\tTo run a list of commands (job server), have '%s' be your full command.\n\n\tIf you need a literal %s or %d, use %%s or %%d.\n\n\n\n\t\tOptions:\n\n\n\t\t\t-nc or --no-collate          By default, the output will be held until the task is completed, so output is not intermixed.\n\n\t\t\t\t\t\t\t\t\t\t   By providing \"-nc\" or \"--no-collate\", instead each line that comes in from any running task\n\n\t\t\t\t\t\t\t\t\t\t   is printed, prefixed with the argset in square-brackets (e.x.  \"[arg1] Some message\"\n\n\n\n\t\tArgsets from stdin\n\n\n\t\t\tIf argset is '--', then the argset items will be read from stdin instead of being provided on the commandline.\n\n\t\t\tExecution begins immediately, so you can use disttask as a job manager with another process feeding in items\n\n\t\t\tas they become available.\n\n\n\n\t\tMax Concurrency\n\n\n\t\t\tYou may use \"0\" or \"MAX\" as the \"concurrent tasks\" parameter to execute all items in the argset simultaneously\n\n\n\t\tExample Usage\n\n\t\t\tdisttask \"ssh root@%s hostname\" 3 host1 host2 host3 host4 host5 host6 # Connect and get hostname on 6 hosts, 3 at a time.\n\n\n\tdisttask version 2.2.0\n\n\n**Concurrency**\n\nThe application runs at most \"concurrent task\" number of processes (good to match total number of processors available to this number - 1).\nIt captures stdout and stderr to ensure that any output is not intertwined between the applications.\n\n**Output**\n\nThe output is written in whole upon completion of each task, unless \"-nc\" or \"--no-collate\" argument is provided, in which case output is printed as soon as it is available, each line prefixed with the item name in square brackets.\n\n\n**Specifying Command**\n\nEach command should specify a \"%s\" to where each argument will go (one argument from @argset per application). To feed entire commands, use simply \"%s\" as the \"cmd\" and provide the commands in your \"argset\".\nThe commands are executed by a shell, and thus can contain shell expressions.\n\n\"%d\" is also available as the pipe number, which can be used to differentiate the running processes.\n\nYou can also have a program or file pipe output to disttask, by providing '\\-\\-' as the argset. Disttask will run and read from stdin, treating each line as an argset item, and will terminate when stdin has been closed and all subprocesses have completed.\n\n\nExamples\n--------\n\nUse disttask to connect to various hosts in parallel and execute commands:\n\n\n\tdisttask \"echo -n 'pipe %d: ' && ssh root@%s hostname\" 2 \"host1\" \"host2\" \"host3\" \"host4\"\n\n\tpipe 0: host1\n\n\tpipe 1: host2\n\n\tpipe 0: host3\n\n\tpipe 1: host4\n\nSame example, using \"no-collate\" option:\n\n\tdisttask \"echo -n 'pipe %d: ' && ssh root@%s hostname\" 2 \"host1\" \"host2\" \"host3\" \"host4\"\n\n\t[host1] pipe 0: host1\n\n\t[host2] pipe 1: host2\n\n\t[host3] pipe 0: host3\n\n\t[host4] pipe 1: host4\n\n\nRun pyflakes, using 10 simultaneous processes, on all python files in subdirectories (requires shopt -s globstar. Notice the backticks, not single-quotes.)\n\n\n\tdisttask \"pyflakes %s\" 10 `echo **/*.py`\n\nUsage: disttask [cmd] [concurrent tasks] [argset]\n\nUse a %s in [cmd] where you want the args to go. use %d for the pipe number.\nTo run a list of commands (job server), have '%s' be your full command.\n\n\n    Options:\n\n       -nc or --no-collate          By default, the output will be held until the task is completed, so output is not intermixed.\n                                       By providing \"-nc\" or \"--no-collate\", instead each line that comes in from any running task\n                                       is printed, prefixed with the argset in square-brackets (e.x.  \"[arg1] Some message\"\n\n\n   Argsets from stdin\n\n      If argset is '--', then the argset items will be read from stdin instead of being provided on the commandline.\n      Execution begins immediately, so you can use disttask as a job manager with another process feeding in items\n      as they become available.\n\n\n   Max Concurrency\n\n      You may use \"0\" or \"MAX\" as the \"concurrent tasks\" parameter to execute all items in the argset simultaneously\n\n\n   Example Usage\n\n      disttask \"ssh root@%s hostname\" 3 host1 host2 host3 host4 host5 host6 # Connect and get hostname on 6 hosts, 3 at a time.\n\ndisttask version 2.1.1",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "",
        "keywords": "disttask,multiprocessing,distributed,concurrent,execution,command,xargs,parallel,threading,pipes,task,management",
        "license": "GPLv3",
        "maintainer": "Tim Savannah",
        "maintainer_email": "kata198@gmail.com",
        "name": "disttask",
        "package_url": "https://pypi.org/project/disttask/",
        "platform": "",
        "project_url": "https://pypi.org/project/disttask/",
        "project_urls": null,
        "release_url": "https://pypi.org/project/disttask/2.4.0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Utility which distributes a task across a fixed number of processes, and collates the output, for better utilization of multiprocessing",
        "version": "2.4.0"
    },
    "last_serial": 4294335,
    "releases": {
        "1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1e6196415ea381253668d8df9cfd1832",
                    "sha256": "9fd29bc9c4a4469492e108cdd27425bbd060dc19d90936b9b0e20fb8109348bf"
                },
                "downloads": -1,
                "filename": "disttask-1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1e6196415ea381253668d8df9cfd1832",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 1257,
                "upload_time": "2015-03-23T18:52:54",
                "url": "https://files.pythonhosted.org/packages/6c/88/718f4851aafcd01d880b488cfbb75e73da2285b1ef000de37dd365ac63a2/disttask-1.0.tar.gz"
            }
        ],
        "1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2a5792d234b0a484cc2306cb9d80b925",
                    "sha256": "c31b1114a9367793a27187d9274f31b16a9b74920118d59a3a85f6688fe82b5e"
                },
                "downloads": -1,
                "filename": "disttask-1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "2a5792d234b0a484cc2306cb9d80b925",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2663,
                "upload_time": "2015-04-08T17:12:24",
                "url": "https://files.pythonhosted.org/packages/a5/32/e4a1facfb1785addef412f60a29ca41c6ee5f735b85792167bee76f091bb/disttask-1.1.tar.gz"
            }
        ],
        "2.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fdb0e125e5011c05766f83e8bc292520",
                    "sha256": "86d426d531f963f899ac31d5b7b74b845002ff824846fba2107d241321584b7a"
                },
                "downloads": -1,
                "filename": "disttask-2.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "fdb0e125e5011c05766f83e8bc292520",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3894,
                "upload_time": "2016-01-27T21:40:10",
                "url": "https://files.pythonhosted.org/packages/f4/1a/ad950640880e62ff1de47e84b3ed0e0d1890e801a743cda4ce079bf1d881/disttask-2.0.0.tar.gz"
            }
        ],
        "2.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "113fb2c4c89bc6d0cdb5eebff843cc78",
                    "sha256": "ba548291a1cf5bcb3ffd8cacb1b85e90cfaef83b73246cf011098a4c34eb209c"
                },
                "downloads": -1,
                "filename": "disttask-2.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "113fb2c4c89bc6d0cdb5eebff843cc78",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3928,
                "upload_time": "2016-01-28T19:22:24",
                "url": "https://files.pythonhosted.org/packages/ed/fe/4d7df458ad6b003c5f6570887cffc5f4065323ef2e48ae28b05aa78d0182/disttask-2.0.1.tar.gz"
            }
        ],
        "2.0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9f99999ffe2e5261eb324c533da0a33a",
                    "sha256": "be3855382d674148bb661fbf88081072c10e1df3178268aae78b9b5b1e5743d9"
                },
                "downloads": -1,
                "filename": "disttask-2.0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "9f99999ffe2e5261eb324c533da0a33a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 16183,
                "upload_time": "2016-01-28T19:23:52",
                "url": "https://files.pythonhosted.org/packages/8c/00/9736c0a48b64f0b0a8067306211e1db3ec54a9d5ccf506bae41a2ce38f17/disttask-2.0.1.0.tar.gz"
            }
        ],
        "2.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "79aaed7341301d16bbd6a6e09a504a3f",
                    "sha256": "93131fb0a28ed77d913ec7706e5c2eb18517cca3c5710ebb0bcf051f6f24fa12"
                },
                "downloads": -1,
                "filename": "disttask-2.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "79aaed7341301d16bbd6a6e09a504a3f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18878,
                "upload_time": "2016-09-23T20:40:05",
                "url": "https://files.pythonhosted.org/packages/c6/0b/557f43c50e4042aa1f11c15c560e26ec9fd90b3608143c754b6b9dbb83e9/disttask-2.1.0.tar.gz"
            }
        ],
        "2.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fbb60404ace3c922cc701c9d47545b41",
                    "sha256": "3aaafef136c8b354d113a71d640b8dba96a9d70f58759bbc80cc48fe10e9753c"
                },
                "downloads": -1,
                "filename": "disttask-2.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "fbb60404ace3c922cc701c9d47545b41",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17327,
                "upload_time": "2016-09-27T00:42:25",
                "url": "https://files.pythonhosted.org/packages/ab/76/208e58cbbcf4a7420f5a938dc0cc3dd1b7d7875e39d35784da0b87a65504/disttask-2.1.1.tar.gz"
            }
        ],
        "2.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "16adb05c42a40f1b0b814e7376f1686b",
                    "sha256": "f19b159967c2e97b57dae7f238e2521a93d8eff1cc143e032cc996904a0c06df"
                },
                "downloads": -1,
                "filename": "disttask-2.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "16adb05c42a40f1b0b814e7376f1686b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17614,
                "upload_time": "2016-09-27T00:57:24",
                "url": "https://files.pythonhosted.org/packages/57/ae/a7df8b14b23cab446114913628c86c19099fb8e670dfe3a3fb3d4c55da2d/disttask-2.2.0.tar.gz"
            }
        ],
        "2.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "115a6c563f73056638f3c53553549a69",
                    "sha256": "b55a595c237fab0a07e022e668f00e627b6275f66415927e4e7f8a9f938f629c"
                },
                "downloads": -1,
                "filename": "disttask-2.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "115a6c563f73056638f3c53553549a69",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 21089,
                "upload_time": "2016-09-27T20:24:21",
                "url": "https://files.pythonhosted.org/packages/a4/5f/175f0861c4f20389a87b62a0115c3155b2acdbc4eba542c13aecf11fc3ab/disttask-2.2.1.tar.gz"
            }
        ],
        "2.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3ae30937b6c73ac68156bf210b4e63d7",
                    "sha256": "4b9f23d68fdc8442ce207debf91b6620f98e4a9f026c648da035a22b7c310333"
                },
                "downloads": -1,
                "filename": "disttask-2.2.2.tar.gz",
                "has_sig": false,
                "md5_digest": "3ae30937b6c73ac68156bf210b4e63d7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19589,
                "upload_time": "2017-04-26T02:28:04",
                "url": "https://files.pythonhosted.org/packages/30/30/98f303ba95cb1d27d0c735f9b8b2e73816617894fa31dbf395888360ee63/disttask-2.2.2.tar.gz"
            }
        ],
        "2.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8f81aeefd934a7959339f7c233c7d914",
                    "sha256": "b2936e820f93e13fb7bd54d7792b187dddf59e3572ed75a2dbb75a9ae99bcfa8"
                },
                "downloads": -1,
                "filename": "disttask-2.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8f81aeefd934a7959339f7c233c7d914",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20059,
                "upload_time": "2018-02-09T21:00:59",
                "url": "https://files.pythonhosted.org/packages/97/ae/f0e56e21ba1be3afaa5f76b322ef32eef0691329b6639ef977e81e62f91c/disttask-2.3.0.tar.gz"
            }
        ],
        "2.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "89366c9ff5fa19e90fe5bddef3ac6a2c",
                    "sha256": "8147b114c7bb4b5839fb9e5774cc966ab6c290e1d2ed9c8ed481f6671c3987e7"
                },
                "downloads": -1,
                "filename": "disttask-2.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "89366c9ff5fa19e90fe5bddef3ac6a2c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20726,
                "upload_time": "2018-09-20T21:25:30",
                "url": "https://files.pythonhosted.org/packages/0c/f6/d08cf0f398988d1ce8a5588439c894bd459a600f09ce920403dc276134b8/disttask-2.4.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "89366c9ff5fa19e90fe5bddef3ac6a2c",
                "sha256": "8147b114c7bb4b5839fb9e5774cc966ab6c290e1d2ed9c8ed481f6671c3987e7"
            },
            "downloads": -1,
            "filename": "disttask-2.4.0.tar.gz",
            "has_sig": false,
            "md5_digest": "89366c9ff5fa19e90fe5bddef3ac6a2c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 20726,
            "upload_time": "2018-09-20T21:25:30",
            "url": "https://files.pythonhosted.org/packages/0c/f6/d08cf0f398988d1ce8a5588439c894bd459a600f09ce920403dc276134b8/disttask-2.4.0.tar.gz"
        }
    ]
}