{
    "info": {
        "author": "Matthew Rocklin",
        "author_email": "mrocklin@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "Dask on DRMAA\n=============\n\n|Build Status| |PyPI Release| |conda-forge Release|\n\nDeploy a Dask.distributed_ cluster on top of a cluster running a\nDRMAA_-compliant job scheduler.\n\n\nExample\n-------\n\nLaunch from Python\n\n.. code-block:: python\n\n   from dask_drmaa import DRMAACluster\n   cluster = DRMAACluster()\n\n   from dask.distributed import Client\n   client = Client(cluster)\n   cluster.start_workers(2)\n\n   >>> future = client.submit(lambda x: x + 1, 10)\n   >>> future.result()\n   11\n\nOr launch from the command line::\n\n    $ dask-drmaa 10  # starts local scheduler and ten remote workers\n\n\nInstall\n-------\n\nPython packages are available from PyPI and can be installed with ``pip``::\n\n    pip install dask-drmaa\n\nAlso ``conda`` packages are available from conda-forge::\n\n    conda install -c conda-forge dask-drmaa\n\nAdditionally the package can be installed from GitHub with the latest changes::\n\n    pip install git+https://github.com/dask/dask-drmaa.git --upgrade\n\nor::\n\n    git clone git@github.com:dask/dask-drmaa.git\n    cd dask-drmaa\n    pip install .\n\nYou must have the DRMAA system library installed and be able to submit jobs\nfrom your local machine. Please make sure to set the environment variable\n``DRMAA_LIBRARY_PATH`` to point to the location of ``libdrmaa.so`` for your\nsystem.\n\n\nTesting\n-------\n\nThis repository contains a Docker-compose testing harness for a Son of Grid\nEngine cluster with a master and two slaves.  You can initialize this system\nas follows:\n\n.. code-block:: bash\n\n   docker-compose build\n   ./start-sge.sh\n\nIf you have done this previously and need to refresh your solution you can do\nthe following\n\n.. code-block:: bash\n\n   docker-compose stop\n   docker-compose build --no-cache\n   ./start-sge.sh\n\nAnd run tests with py.test in the master docker container\n\n.. code-block:: bash\n\n  docker exec -it sge_master /bin/bash -c \"cd /dask-drmaa; python setup.py develop\"\n  docker exec -it sge_master /bin/bash -c \"cd /dask-drmaa; py.test dask_drmaa --verbose\"\n\n\nAdaptive Load\n-------------\n\nDask-drmaa can adapt to scheduler load, deploying more workers on the grid when\nit has more work, and cleaning up these workers when they are no longer\nnecessary.  This can simplify setup (you can just leave a cluster running) and\nit can reduce load on the cluster, making IT happy.\n\nTo enable this, call the ``adapt`` method of a ``DRMAACluster``.  You can\nsubmit computations to the cluster without ever explicitly creating workers.\n\n.. code-block:: python\n\n   from dask_drmaa import DRMAACluster\n   from dask.distributed import Client\n\n   cluster = DRMAACluster()\n   cluster.adapt()\n   client = Client(cluster)\n\n   futures = client.map(func, seq)  # workers will be created as necessary\n\n\nExtensible\n----------\n\nThe DRMAA interface is the lowest common denominator among many different job\nschedulers like SGE, SLURM, LSF, Torque, and others.  However, sometimes users\nneed to specify parameters particular to their cluster, such as resource\nqueues, wall times, memory constraints, etc..\n\nDRMAA allows users to pass native specifications either when constructing the\ncluster or when starting new workers:\n\n.. code-block:: python\n\n   cluster = DRMAACluster(template={'nativeSpecification': '-l h_rt=01:00:00'})\n   # or\n   cluster.start_workers(10, nativeSpecification='-l h_rt=01:00:00')\n\n\nRelated Work\n------------\n\n* DRMAA_: The Distributed Resource Management Application API, a high level\n  API for general use on traditional job schedulers\n* drmaa-python_: The Python bindings for DRMAA\n* DaskSGE_: An earlier dask-drmaa implementation\n* `Son of Grid Engine`_: The default implementation used in testing\n* Dask.distributed_: The actual distributed computing library this launches\n\n.. _DRMAA: https://www.drmaa.org/\n.. _drmaa-python: http://drmaa-python.readthedocs.io/en/latest/\n.. _`Son of Grid Engine`: https://arc.liv.ac.uk/trac/SGE\n.. _dasksge: https://github.com/mfouesneau/dasksge\n.. _Dask.distributed: http://distributed.readthedocs.io/en/latest/\n.. _DRMAA: https://www.drmaa.org/\n\n\n.. |Build Status| image:: https://travis-ci.org/dask/dask-drmaa.svg?branch=master\n   :target: https://travis-ci.org/dask/dask-drmaa\n\n.. |PyPI Release| image:: https://img.shields.io/pypi/v/dask-drmaa.svg\n   :target: https://pypi.python.org/pypi/dask-drmaa\n\n.. |conda-forge Release| image:: https://img.shields.io/conda/vn/conda-forge/dask-drmaa.svg\n   :target: https://github.com/conda-forge/dask-drmaa-feedstock\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://github.com/dask/dask-drmaa/",
        "keywords": "",
        "license": "BSD",
        "maintainer": "",
        "maintainer_email": "",
        "name": "dask-drmaa",
        "package_url": "https://pypi.org/project/dask-drmaa/",
        "platform": "",
        "project_url": "https://pypi.org/project/dask-drmaa/",
        "project_urls": {
            "Homepage": "http://github.com/dask/dask-drmaa/"
        },
        "release_url": "https://pypi.org/project/dask-drmaa/0.2.1/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Dask on DRMAA",
        "version": "0.2.1"
    },
    "last_serial": 3833010,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1cda20b30bc9139591e15d9119b76ade",
                    "sha256": "c98041835727539fd541a2a08963831c6b1f3371784df145ab48298d5de19aff"
                },
                "downloads": -1,
                "filename": "dask-drmaa-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1cda20b30bc9139591e15d9119b76ade",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 24602,
                "upload_time": "2017-11-10T06:02:15",
                "url": "https://files.pythonhosted.org/packages/b5/88/6ecb9e020d1040ee16cc66475a75585bdf16c5b3456b877b6b3a02e35e8b/dask-drmaa-0.1.0.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f22e409300228c3f130394d0a2040604",
                    "sha256": "999f8e037c3cfd56a885eb9b15bfb06e552bf37292c1434e153a8f4527afb644"
                },
                "downloads": -1,
                "filename": "dask-drmaa-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "f22e409300228c3f130394d0a2040604",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25045,
                "upload_time": "2018-03-09T16:29:20",
                "url": "https://files.pythonhosted.org/packages/63/c4/aa7bab3bb3d16d3466fb2e1b65598f058ae569ba75f569af44a4d2688011/dask-drmaa-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b71efa1d6b5f78c1005c543d127fd3d0",
                    "sha256": "5d5ff119babc89d37bd6eb44e4f2297d5766d8e8614871d940a22f656f761030"
                },
                "downloads": -1,
                "filename": "dask-drmaa-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "b71efa1d6b5f78c1005c543d127fd3d0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25053,
                "upload_time": "2018-03-09T20:19:42",
                "url": "https://files.pythonhosted.org/packages/f2/ad/fef83ca39d1601528b7d07f206c4ad8274675ce86e352b3e34150ecab8f9/dask-drmaa-0.1.2.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7b7dfb86941aa34191c27e60603d87a8",
                    "sha256": "8bc9a730d5cd7f9b1d9541f1076c2300f5590f09c6c38b0f3d2d21b00852c746"
                },
                "downloads": -1,
                "filename": "dask-drmaa-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "7b7dfb86941aa34191c27e60603d87a8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25411,
                "upload_time": "2018-04-02T19:33:41",
                "url": "https://files.pythonhosted.org/packages/42/a3/13af82bda7585fbbd7762369a89b3bf03ba0d2c684bd0737c4ae8824c580/dask-drmaa-0.2.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "727264379162be849cca557be823bcc4",
                    "sha256": "8b02524c38138358da2076c3405a340b493a6f07f2a2865e6805edce073ca6d2"
                },
                "downloads": -1,
                "filename": "dask-drmaa-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "727264379162be849cca557be823bcc4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 25953,
                "upload_time": "2018-05-04T03:02:29",
                "url": "https://files.pythonhosted.org/packages/00/47/9ee6f6be4e073ddfbe9a4afde31c3b2e574c26aa9ab13aebb5ad418f1fb2/dask-drmaa-0.2.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "727264379162be849cca557be823bcc4",
                "sha256": "8b02524c38138358da2076c3405a340b493a6f07f2a2865e6805edce073ca6d2"
            },
            "downloads": -1,
            "filename": "dask-drmaa-0.2.1.tar.gz",
            "has_sig": false,
            "md5_digest": "727264379162be849cca557be823bcc4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 25953,
            "upload_time": "2018-05-04T03:02:29",
            "url": "https://files.pythonhosted.org/packages/00/47/9ee6f6be4e073ddfbe9a4afde31c3b2e574c26aa9ab13aebb5ad418f1fb2/dask-drmaa-0.2.1.tar.gz"
        }
    ]
}