{
    "info": {
        "author": "Michael Milligan, Andrea Zonca, Mike Gilbert",
        "author_email": "milligan@umn.edu, m code@andreazonca.com, mike@nau.edu",
        "bugtrack_url": null,
        "classifiers": [
            "Intended Audience :: Developers",
            "Intended Audience :: Science/Research",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: BSD License",
            "Programming Language :: Python",
            "Programming Language :: Python :: 3"
        ],
        "description": "# batchspawner for Jupyterhub\n\n[![Build Status](https://travis-ci.org/jupyterhub/batchspawner.svg?branch=master)](https://travis-ci.org/jupyterhub/batchspawner)\n\nThis is a custom spawner for [Jupyterhub](https://jupyterhub.readthedocs.io/) that is designed for installations on clusters using batch scheduling software.\n\nThis began as a generalization of [mkgilbert's batchspawner](https://github.com/mkgilbert/slurmspawner) which in turn was inspired by [Andrea Zonca's blog post](http://zonca.github.io/2015/04/jupyterhub-hpc.html 'Run jupyterhub on a Supercomputer') where he explains his implementation for a spawner that uses SSH and Torque. His github repo is found [here](http://www.github.com/zonca/remotespawner 'RemoteSpawner').\n\nThis package formerly included WrapSpawner and ProfilesSpawner, which provide mechanisms for runtime configuration of spawners.  These have been split out and moved to the [`wrapspawner`](https://github.com/jupyterhub/wrapspawner) package.\n\n## Installation\n1. from root directory of this repo (where setup.py is), run `pip install -e .`\n\n   If you don't actually need an editable version, you can simply run\n      `pip install batchspawner`\n\n2. add lines in jupyterhub_config.py for the spawner you intend to use, e.g.\n\n   ```python\n      c = get_config()\n      c.JupyterHub.spawner_class = 'batchspawner.TorqueSpawner'\n   ```\n3. Depending on the spawner, additional configuration will likely be needed.\n\n## Batch Spawners\n\n### Overview\n\nThis file contains an abstraction layer for batch job queueing systems (`BatchSpawnerBase`), and implements\nJupyterhub spawners for Torque, Moab, SLURM, SGE, HTCondor, LSF, and eventually others.\nCommon attributes of batch submission / resource manager environments will include notions of:\n  * queue names, resource manager addresses\n  * resource limits including runtime, number of processes, memory\n  * singleuser child process running on (usually remote) host not known until runtime\n  * job submission and monitoring via resource manager utilities\n  * remote execution via submission of templated scripts\n  * job names instead of PIDs\n\n`BatchSpawnerBase` provides several general mechanisms:\n  * configurable traits `req_foo` that are exposed as `{foo}` in job template scripts\n  * configurable command templates for submitting/querying/cancelling jobs\n  * a generic concept of job-ID and ID-based job state tracking\n  * overrideable hooks for subclasses to plug in logic at numerous points\n\n### Example\n\nEvery effort has been made to accomodate highly diverse systems through configuration\nonly. This example consists of the (lightly edited) configuration used by the author\nto run Jupyter notebooks on an academic supercomputer cluster.\n\n   ```python\n   # Select the Torque backend and increase the timeout since batch jobs may take time to start\n   c.JupyterHub.spawner_class = 'batchspawner.TorqueSpawner'\n   c.Spawner.http_timeout = 120\n\n   #------------------------------------------------------------------------------\n   # BatchSpawnerBase configuration\n   #    These are simply setting parameters used in the job script template below\n   #------------------------------------------------------------------------------\n   c.BatchSpawnerBase.req_nprocs = '2'\n   c.BatchSpawnerBase.req_queue = 'mesabi'\n   c.BatchSpawnerBase.req_host = 'mesabi.xyz.edu'\n   c.BatchSpawnerBase.req_runtime = '12:00:00'\n   c.BatchSpawnerBase.req_memory = '4gb'\n   #------------------------------------------------------------------------------\n   # TorqueSpawner configuration\n   #    The script below is nearly identical to the default template, but we needed\n   #    to add a line for our local environment. For most sites the default templates\n   #    should be a good starting point.\n   #------------------------------------------------------------------------------\n   c.TorqueSpawner.batch_script = '''#!/bin/sh\n   #PBS -q {queue}@{host}\n   #PBS -l walltime={runtime}\n   #PBS -l nodes=1:ppn={nprocs}\n   #PBS -l mem={memory}\n   #PBS -N jupyterhub-singleuser\n   #PBS -v {keepvars}\n   module load python3\n   {cmd}\n   '''\n   # For our site we need to munge the execution hostname returned by qstat\n   c.TorqueSpawner.state_exechost_exp = r'int-\\1.mesabi.xyz.edu'\n   ```\n\n## Provide different configurations of BatchSpawner\n\n### Overview\n\n`ProfilesSpawner`, available as part of the [`wrapspawner`](https://github.com/jupyterhub/wrapspawner)\npackage, allows the Jupyterhub administrator to define a set of different spawning configurations,\nboth different spawners and different configurations of the same spawner.\nThe user is then presented a dropdown menu for choosing the most suitable configuration for their needs.\n\nThis method provides an easy and safe way to provide different configurations of `BatchSpawner` to the\nusers, see an example below.\n\n### Example\n\nThe following is based on the author's configuration (at the same site as the example above)\nshowing how to give users access to multiple job configurations on the batch scheduled\nclusters, as well as an option to run a local notebook directly on the jupyterhub server.\n\n   ```python\n   # Same initial setup as the previous example\n   c.JupyterHub.spawner_class = 'wrapspawner.ProfilesSpawner'\n   c.Spawner.http_timeout = 120\n   #------------------------------------------------------------------------------\n   # BatchSpawnerBase configuration\n   #   Providing default values that we may omit in the profiles\n   #------------------------------------------------------------------------------\n   c.BatchSpawnerBase.req_host = 'mesabi.xyz.edu'\n   c.BatchSpawnerBase.req_runtime = '12:00:00'\n   c.TorqueSpawner.state_exechost_exp = r'in-\\1.mesabi.xyz.edu'\n   #------------------------------------------------------------------------------\n   # ProfilesSpawner configuration\n   #------------------------------------------------------------------------------\n   # List of profiles to offer for selection. Signature is:\n   #   List(Tuple( Unicode, Unicode, Type(Spawner), Dict ))\n   # corresponding to profile display name, unique key, Spawner class,\n   # dictionary of spawner config options.\n   #\n   # The first three values will be exposed in the input_template as {display},\n   # {key}, and {type}\n   #\n   c.ProfilesSpawner.profiles = [\n      ( \"Local server\", 'local', 'jupyterhub.spawner.LocalProcessSpawner', {'ip':'0.0.0.0'} ),\n      ('Mesabi - 2 cores, 4 GB, 8 hours', 'mesabi2c4g12h', 'batchspawner.TorqueSpawner',\n         dict(req_nprocs='2', req_queue='mesabi', req_runtime='8:00:00', req_memory='4gb')),\n      ('Mesabi - 12 cores, 128 GB, 4 hours', 'mesabi128gb', 'batchspawner.TorqueSpawner',\n         dict(req_nprocs='12', req_queue='ram256g', req_runtime='4:00:00', req_memory='125gb')),\n      ('Mesabi - 2 cores, 4 GB, 24 hours', 'mesabi2c4gb24h', 'batchspawner.TorqueSpawner',\n         dict(req_nprocs='2', req_queue='mesabi', req_runtime='24:00:00', req_memory='4gb')),\n      ('Interactive Cluster - 2 cores, 4 GB, 8 hours', 'lab', 'batchspawner.TorqueSpawner',\n         dict(req_nprocs='2', req_host='labhost.xyz.edu', req_queue='lab',\n             req_runtime='8:00:00', req_memory='4gb', state_exechost_exp='')),\n      ]\n   ```\n\n\n## Changelog\n\n### v0.8.1 (bugfix release)\n\n* Fix regression: single-user server binding address is overwritten by previous session server address, resulting in failure to start.  Issue #76\n\n### v0.8.0 (compatible with JupyterHub 0.5.0 through 0.8.1/0.9dev)\n\n* SlurmSpawner: Remove `--uid` for (at least) Slurm 17.11 compatibility.  If you use `sudo`, this should not be necessary, but because this is security related you should check that user management is as you expect.  If your configuration does not use `sudo` then you may need to add the `--uid` option in a custom `batch_script`.\n* add base options `req_ngpus` `req_partition` `req_account` and `req_options` \n* Fix up logging\n* Merge `user_options` with the template substitution vars instead of having it as a separate key\n* Update ip/port handling for JupyterHub 0.8\n* Add `LICENSE` (BSD3) and `CONTRIBUTING.md`\n* Add `LsfSpawner` for IBM LFS\n* Add `MultiSlurmSpawner`\n* Add `MoabSpawner`\n* Add `condorSpawner`\n* Add `GridEngineSpawner`\n* SlurmSpawner: add `req_qos` option\n* WrapSpawner and ProfilesSpawner, which provide mechanisms for runtime configuration of spawners, have been split out and moved to the [`wrapspawner`](https://github.com/jupyterhub/wrapspawner) package\n* Enable CI testing via Travis-CI\n\n\n### v0.3 (tag: jhub-0.3, compatible with JupyterHub 0.3.0)\n\n* initial release containing `TorqueSpawner` and `SlurmSpawner`\n\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://jupyter.org",
        "keywords": "Interactive,Interpreter,Shell,Web,Jupyter",
        "license": "BSD",
        "maintainer": "",
        "maintainer_email": "",
        "name": "batchspawner",
        "package_url": "https://pypi.org/project/batchspawner/",
        "platform": "Linux",
        "project_url": "https://pypi.org/project/batchspawner/",
        "project_urls": {
            "About Jupyterhub": "http://jupyterhub.readthedocs.io/en/latest/",
            "Bug Reports": "https://github.com/jupyterhub/batchspawner/issues",
            "Homepage": "http://jupyter.org",
            "Jupyter Project": "http://jupyter.org",
            "Source": "https://github.com/jupyterhub/batchspawner/"
        },
        "release_url": "https://pypi.org/project/batchspawner/0.8.1/",
        "requires_dist": [
            "jupyterhub (>=0.5)"
        ],
        "requires_python": "~=3.3",
        "summary": "Batchspawner: A spawner for Jupyterhub to spawn notebooks using batch resource managers.",
        "version": "0.8.1"
    },
    "last_serial": 3828467,
    "releases": {
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e8141c45ddb84048bfeaaedf87badf22",
                    "sha256": "4e4b315464d8aebc63dd48f748c8a629e91ab5bf3d99b82a73b68ce879e10ef1"
                },
                "downloads": -1,
                "filename": "batchspawner-0.8.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e8141c45ddb84048bfeaaedf87badf22",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "~=3.3",
                "size": 11475,
                "upload_time": "2018-04-24T23:37:24",
                "url": "https://files.pythonhosted.org/packages/b3/a9/24e0be1fc97fd92be1a654cace9c904082e9a14fb2e93f22df011ae2939c/batchspawner-0.8.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "864f8382c7485d38268400aad93d853f",
                    "sha256": "848ef9ce81e5bcd5d03aef79ca2220da19d232632222358e01e865c1e218f661"
                },
                "downloads": -1,
                "filename": "batchspawner-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "864f8382c7485d38268400aad93d853f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "~=3.3",
                "size": 15349,
                "upload_time": "2018-04-24T23:37:25",
                "url": "https://files.pythonhosted.org/packages/d3/77/fa6deb5a97bb69f1c631c86613e7bd8322f02b8fa8ef3c346971a909d8fe/batchspawner-0.8.0.tar.gz"
            }
        ],
        "0.8.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "616eecba8d20642131097a1c92727874",
                    "sha256": "f191b12b6019d70c56139e6a4f89b45c494b576bcd36c53bcc53f1e748c69b49"
                },
                "downloads": -1,
                "filename": "batchspawner-0.8.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "616eecba8d20642131097a1c92727874",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": "~=3.3",
                "size": 11664,
                "upload_time": "2018-05-02T20:50:37",
                "url": "https://files.pythonhosted.org/packages/e4/07/e3cb8ae9a045dac146d647dfe528cf286130ae4a2a8f36311a0f3aea4185/batchspawner-0.8.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "0d72644a54dea8c2ce10e6825bcbf571",
                    "sha256": "861648f8b39d7c564582e0b705325008461487a9bc656c6b96f85230127a3702"
                },
                "downloads": -1,
                "filename": "batchspawner-0.8.1.tar.gz",
                "has_sig": false,
                "md5_digest": "0d72644a54dea8c2ce10e6825bcbf571",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": "~=3.3",
                "size": 15641,
                "upload_time": "2018-05-02T20:50:39",
                "url": "https://files.pythonhosted.org/packages/41/1e/1acb77443cc6c45173ff2373f2fb384b72b51ab99795bed659f9ba4d86da/batchspawner-0.8.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "616eecba8d20642131097a1c92727874",
                "sha256": "f191b12b6019d70c56139e6a4f89b45c494b576bcd36c53bcc53f1e748c69b49"
            },
            "downloads": -1,
            "filename": "batchspawner-0.8.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "616eecba8d20642131097a1c92727874",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "~=3.3",
            "size": 11664,
            "upload_time": "2018-05-02T20:50:37",
            "url": "https://files.pythonhosted.org/packages/e4/07/e3cb8ae9a045dac146d647dfe528cf286130ae4a2a8f36311a0f3aea4185/batchspawner-0.8.1-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "0d72644a54dea8c2ce10e6825bcbf571",
                "sha256": "861648f8b39d7c564582e0b705325008461487a9bc656c6b96f85230127a3702"
            },
            "downloads": -1,
            "filename": "batchspawner-0.8.1.tar.gz",
            "has_sig": false,
            "md5_digest": "0d72644a54dea8c2ce10e6825bcbf571",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "~=3.3",
            "size": 15641,
            "upload_time": "2018-05-02T20:50:39",
            "url": "https://files.pythonhosted.org/packages/41/1e/1acb77443cc6c45173ff2373f2fb384b72b51ab99795bed659f9ba4d86da/batchspawner-0.8.1.tar.gz"
        }
    ]
}