{
    "info": {
        "author": "Maxim Kochurov",
        "author_email": "maxim.v.kochurov@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3",
            "Topic :: Scientific/Engineering"
        ],
        "description": "Batch bsub launcher\n===================\n\nPure python script to create batched GPU jobs using BSUB\n\nInstallation / Update\n---------------------\n\n::\n\n    pip install -U git+https://github.com/ferrine/gpu-batch.sub\n    # or\n    pip install -U gpu-batch-sub\n\nExample Config\n--------------\n\nDefault config for script (location ``~/.gpubatch.conf``) should look\nlike\n\n::\n\n    > cat config\n    [gpubatch]\n    batch=-1\n    gpu=1\n    ; use ';' for comments\n    ;paths are relative\n    out=bsub-log/out\n    err=bsub-log/err\n    hosts=host1 host2 host3\n    queue=normal\n    header=\n        #BSUB option1\n        #BSUB option2\n        #BSUB option3\n        custom code you want\n\nExamples\n--------\n\n::\n\n    # batch by -1\n    # yields 1 job\n    > gpu-batch.sub 'python script_1.py' 'python script_2.py' 'python script_2.py --other-args'\n\n    # batch by 2\n    # yields 2 jobs\n    > gpu-batch.sub -b 2 'python script_1.py' 'python script_2.py' 'python script_2.py --other-args'\n\n    # run from file\n    > gpu-batch.sub -b 2 -f filewithjobs1 filewithjobs2 filewithjobs3\n    > cat filewithjobs1\n    multiline \\ # comments are ok\n        job number one\n    # comments here are ok too\n    multiline \\\n        job number two\n\n    # naming jobs\n    # special syntax is applied (no spaces allowed in jobname)\n    gpu-batch.sub 'jobname : python script1.py'\n\n    # running sequential jobs\n    # yields 2 jobs with sequentially running commands\n    > gpu-batch.sub -b 2 -s 'python script_1.py' 'python script_2.py' 'python script_2.py --other-args'\n\nChecking Command Submission\n---------------------------\n\n``--debug`` flag helps to print expected submissions to LSF\n\n::\n\n    > gpu-batch.sub --debug --batch 2 command1 command2 named:command3\n    >>>>>>>>>>\n    #SUBMIT: 0\n    vvvvvvvvvv\n    #!/bin/sh\n    #BSUB -J gpu-batch.sub\n    #BSUB -o bsub-log/out/gpu-batch.sub-%J-stats.out\n    #BSUB -q normal\n    #BSUB -n 1\n    #BSUB -gpu \"num=1:j_exclusive=no:mode=shared\"\n    cd ${LS_SUBCWD}\n    mkdir -p bsub-log/out\n    mkdir -p bsub-log/err\n    {\n    command1 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-0.0.0.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-0.0.0.err ;\n    } & {\n    command2 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-0.1.0.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-0.1.0.err ;\n    } & wait\n\n    >>>>>>>>>>\n    #SUBMIT: 1\n    vvvvvvvvvv\n    #!/bin/sh\n    #BSUB -J gpu-batch.sub\n    #BSUB -o bsub-log/out/gpu-batch.sub-%J-stats.out\n    #BSUB -q normal\n    #BSUB -n 1\n    #BSUB -gpu \"num=1:j_exclusive=no:mode=shared\"\n    cd ${LS_SUBCWD}\n    mkdir -p bsub-log/out\n    mkdir -p bsub-log/err\n    {\n    command3 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-1.0.0-named.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-1.0.0-named.err ;\n    } & wait\n\nRunning commands from file\n--------------------------\n\n::\n\n    > cat commands\n    command1\n    command2\n    <sequential> # indicates sequential jobs start\n    command3\n    command4\n    </sequential> # indicates sequential jobs end\n    > gpu-batch.sub --debug -b 2 -f commands\n    >>>>>>>>>>\n    #SUBMIT: 0\n    vvvvvvvvvv\n    #!/bin/sh\n    #BSUB -J gpu-batch.sub\n    #BSUB -o bsub-log/out/gpu-batch.sub-%J-stats.out\n    #BSUB -q normal\n    #BSUB -n 1\n    #BSUB -gpu \"num=1:j_exclusive=no:mode=shared\"\n    cd ${LS_SUBCWD}\n    mkdir -p bsub-log/out\n    mkdir -p bsub-log/err\n    {\n    command1 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-0.0.0.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-0.0.0.err ;\n    } & {\n    command2 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-0.1.0.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-0.1.0.err ;\n    } & wait\n\n    >>>>>>>>>>\n    #SUBMIT: 1\n    vvvvvvvvvv\n    #!/bin/sh\n    #BSUB -J gpu-batch.sub\n    #BSUB -o bsub-log/out/gpu-batch.sub-%J-stats.out\n    #BSUB -q normal\n    #BSUB -n 1\n    #BSUB -gpu \"num=1:j_exclusive=no:mode=shared\"\n    cd ${LS_SUBCWD}\n    mkdir -p bsub-log/out\n    mkdir -p bsub-log/err\n    {\n    command3 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-1.0.0.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-1.0.0.err ;\n    command4 >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-1.0.1.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-1.0.1.err ;\n    } & wait\n\nRunning single command, no quotes\n---------------------------------\n\n::\n\n    > gpu-batch.sub --debug -C python main.py\n    >>>>>>>>>>\n    #SUBMIT: 0\n    vvvvvvvvvv\n    #!/bin/sh\n    #BSUB -J gpu-batch.sub\n    #BSUB -o bsub-log/out/gpu-batch.sub-%J-stats.out\n    #BSUB -q normal\n    #BSUB -n 1\n    #BSUB -gpu \"num=1:j_exclusive=no:mode=shared\"\n    cd ${LS_SUBCWD}\n    mkdir -p bsub-log/out\n    mkdir -p bsub-log/err\n    {\n    python main.py >\\\n      bsub-log/out/gpu-batch.sub-${LSB_JOBID}-0.0.0.out 2> bsub-log/err/gpu-batch.sub-${LSB_JOBID}-0.0.0.err ;\n    } & wait\n\nProgram Description\n-------------------\n\n::\n\n    usage: gpu-batch.sub [-h] [--batch BATCH] [--sequential] [--gpu GPU]\n                         [--out OUT] [--err ERR] [--name NAME] [--hosts HOSTS]\n                         [--files FILES [FILES ...]] [-C [C [C ...]]]\n                         [--queue QUEUE] [--exclusive] [--debug]\n                         [--bsub-bin BSUB_BIN] [--version]\n                         [jobs [jobs ...]]\n\n    gpu-batch.sub is a util to wrap submissions to LSF in a batch. It\n    automatically collects jobs, prepares submission file you can check beforehand\n    with `--debug` flag. `gpu-batch.sub` asks LSF for desired number of GPU per\n    batch and allocates them in shared or exclusive (not recommended) mode.\n\n    positional arguments:\n      jobs                  Jobs to execute (e.g. 'python script.py') enclosed as\n                            strings, you can specify either files or explicit jobs\n                            in command line. Multiline jobs in files are\n                            supported. Optional naming schema for jobs has the\n                            following syntax 'name:command' (default: [])\n\n    optional arguments:\n      -h, --help            show this help message and exit\n      --batch BATCH, -b BATCH\n                            Number of jobs in batch where -1 stands for unlimited\n                            batch (default: -1)\n      --sequential, -s      Make all jobs sequential within bsub submit (default:\n                            False)\n      --gpu GPU, -g GPU     Number of gpu per batch (default: 1)\n      --out OUT, -o OUT     Output path for stdout (default: bsub-log/out)\n      --err ERR, -e ERR     Output path for stderr (default: bsub-log/err)\n      --name NAME, -n NAME  Name for job, defaults to base directory of execution\n                            (default: $(basename `pwd`))\n      --hosts HOSTS         Space or comma separated allowed hosts. Empty string\n                            holds for ALL visible hosts. It is suggested to\n                            specify hosts in `.conf` file. Passing hosts in\n                            command line looks like `--hosts ''` for ALL or\n                            `--hosts 'host1,host2'` for 2 hosts (default: )\n      --files FILES [FILES ...], -f FILES [FILES ...]\n                            Read jobs from files. File can contain multiline jobs\n                            for readability (default: [])\n      -C [C [C ...]]        Single command, does not require quotes (default: [])\n      --queue QUEUE, -q QUEUE\n                            Queue name (default: normal)\n      --exclusive, -x       Exclusive mode allocates GPU only for 1 separate job.\n                            (default: no)\n      --debug               Print submissions and exit (default: False)\n      --bsub-bin BSUB_BIN   bsub binary path (default: bsub)\n      --version             Print version and exit (default: False)\n\n    Default settings are stored in `$HOME/.gpubatch.conf`. They will override the\n    help message as well. Possible settings for config file: batch, gpu, hosts,\n    header, queue. Header will be appended to LSF submission file as is, there is\n    no default extra header.\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/ferrine/gpu-batch.sub",
        "keywords": "",
        "license": "Apache License, Version 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "gpu-batch-sub",
        "package_url": "https://pypi.org/project/gpu-batch-sub/",
        "platform": "",
        "project_url": "https://pypi.org/project/gpu-batch-sub/",
        "project_urls": {
            "Homepage": "https://github.com/ferrine/gpu-batch.sub"
        },
        "release_url": "https://pypi.org/project/gpu-batch-sub/0.1.2/",
        "requires_dist": null,
        "requires_python": ">=3.5",
        "summary": "Batch bsub launcher",
        "version": "0.1.2"
    },
    "last_serial": 4795625,
    "releases": {
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a1e93fd7ca61100237a0c4309d21928c",
                    "sha256": "752e1f6ff614686ac058159b5953b26fff72b67f2b0e84253b95e5051edaf5da"
                },
                "downloads": -1,
                "filename": "gpu-batch-sub-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "a1e93fd7ca61100237a0c4309d21928c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5693,
                "upload_time": "2018-06-22T21:59:17",
                "url": "https://files.pythonhosted.org/packages/9f/da/5fa2b41d56ad7996876fdb5a59771dc97f78c293b28e1e4717d372e331d6/gpu-batch-sub-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dbbbb661acfc3c2498018c6f5ac5194e",
                    "sha256": "04fdead5bade1853a8357d3456f39f1269f481d319b3536f6e30b99f88ef6409"
                },
                "downloads": -1,
                "filename": "gpu-batch-sub-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "dbbbb661acfc3c2498018c6f5ac5194e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5700,
                "upload_time": "2018-07-01T13:39:05",
                "url": "https://files.pythonhosted.org/packages/3c/3f/34e46456e6257c1fc2d53b5a1c950f9dd5b00a2555d8ebc13d60f310ed26/gpu-batch-sub-0.0.3.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1886464b078bcbc78d02277e6292f44c",
                    "sha256": "a7d849d20499dde518ac6e4f07a233e74ba0a731320f1af9ed6ba1d00bef0466"
                },
                "downloads": -1,
                "filename": "gpu_batch_sub-0.1.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1886464b078bcbc78d02277e6292f44c",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5",
                "size": 11698,
                "upload_time": "2019-02-08T14:01:41",
                "url": "https://files.pythonhosted.org/packages/6d/92/afa05e7e1dadcdec3639e9e7fda86e3ecb4ab71a791424dedac3b5493baf/gpu_batch_sub-0.1.1-py3-none-any.whl"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6a47a56a96204ad571c8f0440359c8f6",
                    "sha256": "eb918296e34f361477036964968fb34733fb87001e02ff907c00f909034fd1e3"
                },
                "downloads": -1,
                "filename": "gpu_batch_sub-0.1.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6a47a56a96204ad571c8f0440359c8f6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.5",
                "size": 11700,
                "upload_time": "2019-02-08T14:04:07",
                "url": "https://files.pythonhosted.org/packages/89/97/4b1d1908d8bcfff49afc5ec8e5861401025f99e1be62c42fdbcc0c0e442e/gpu_batch_sub-0.1.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "25b8f1849d70dc22e3499d4f1ab93984",
                    "sha256": "5f75d50ed113412ec22f94ab61b948b8702bb58d81551fddb5bdac8af771a182"
                },
                "downloads": -1,
                "filename": "gpu-batch-sub-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "25b8f1849d70dc22e3499d4f1ab93984",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.5",
                "size": 7329,
                "upload_time": "2019-02-08T14:04:09",
                "url": "https://files.pythonhosted.org/packages/95/85/37fab88d7e6a335f8dea2c8f3adae04b78c2f6de058edf03e9f366fb2a52/gpu-batch-sub-0.1.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "6a47a56a96204ad571c8f0440359c8f6",
                "sha256": "eb918296e34f361477036964968fb34733fb87001e02ff907c00f909034fd1e3"
            },
            "downloads": -1,
            "filename": "gpu_batch_sub-0.1.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "6a47a56a96204ad571c8f0440359c8f6",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.5",
            "size": 11700,
            "upload_time": "2019-02-08T14:04:07",
            "url": "https://files.pythonhosted.org/packages/89/97/4b1d1908d8bcfff49afc5ec8e5861401025f99e1be62c42fdbcc0c0e442e/gpu_batch_sub-0.1.2-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "25b8f1849d70dc22e3499d4f1ab93984",
                "sha256": "5f75d50ed113412ec22f94ab61b948b8702bb58d81551fddb5bdac8af771a182"
            },
            "downloads": -1,
            "filename": "gpu-batch-sub-0.1.2.tar.gz",
            "has_sig": false,
            "md5_digest": "25b8f1849d70dc22e3499d4f1ab93984",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.5",
            "size": 7329,
            "upload_time": "2019-02-08T14:04:09",
            "url": "https://files.pythonhosted.org/packages/95/85/37fab88d7e6a335f8dea2c8f3adae04b78c2f6de058edf03e9f366fb2a52/gpu-batch-sub-0.1.2.tar.gz"
        }
    ]
}