{ "info": { "author": "Kriszti\u00e1n Sz\u0171cs", "author_email": "krisztian.szucs@lensa.com", "bugtrack_url": null, "classifiers": [], "description": "[![Build Status](http://52.0.47.203:8000/api/badges/lensacom/dask.mesos/status.svg)](http://52.0.47.203:8000/lensacom/dask.mesos)\n[![Join the chat at https://gitter.im/lensacom/dask.mesos](https://badges.gitter.im/lensacom/dask.mesos.svg)](https://gitter.im/lensacom/dask.mesos?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)\n\n[Apache Mesos](http://mesos.apache.org/) backend for [Dask](https://github.com/dask/dask) scheduling library.\nRun distributed python dask workflows on your Mesos cluster.\n\n## Notable Features\n\n - distributively run tasks in docker container\n - specify resource requirements per task\n - bin packing for optimized resource utilization\n\n## Installation\n\n**Prerequisits:** [satyr](https://github.com/lensacom/satyr), [dask](https://github.com/dask/dask.git), [toolz](https://pypi.python.org/pypi/toolz). All of them should be installed w/ the following commands:\n\n`pip install dask.mesos` or use [lensa/dask.mesos](https://hub.docker.com/r/lensa/dask.mesos/) Docker image\n\nConfiguration:\n- `MESOS_MASTER=zk://127.0.0.1:2181/mesos`\n- `ZOOKEEPER_HOST=127.0.0.1:2181`\n\n\n## Example\n\n```python\nfrom __future__ import absolute_import, division, print_function\n\nfrom dask_mesos import mesos, MesosExecutor\n\n\n@mesos(cpus=0.1, mem=64)\ndef add(x, y):\n \"\"\"Run add on mesos with specified resources\"\"\"\n return x + y\n\n\n@mesos(cpus=0.3, mem=128, image='lensa/dask.mesos')\ndef mul(x, y):\n \"\"\"Run mul on mesos in specified docker image\"\"\"\n return x * y\n\n\nwith MesosExecutor(name='dask') as executor:\n \"\"\"This context handles Mesos scheduler's lifecycle\"\"\"\n a, b = 23, 89\n alot = add(a, b)\n gigalot = mul(alot, add(10, 2))\n\n gigalot.compute(get=executor.get) # (a + b) * (10 + 2)\n executor.compute([alot, gigalot]) # list of futures\n```\n\n\n## Configuring dask.mesos Tasks\n\nYou can configure your mesos tasks in your decorator, currently the following options are available:\n\n* **cpus**: The amount of cpus to use for the task.\n* **mem**: Memory in MB to reserver for the task.\n* **disk**: The amount of disk to use for the task.\n* **image**: A docker image name. If not set, mesos containerizer will be used.\n\nBoth mem and cpus are defaults to some low values set in _satyr_ so you're encouraged to override them here. More options like constraints, other resources are on the way.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/lensacom/dask.mesos", "keywords": "mesos dask multiprocessing scheduling satyr", "license": "Apache License, Version 2.0", "maintainer": null, "maintainer_email": null, "name": "dask.mesos", "package_url": "https://pypi.org/project/dask.mesos/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/dask.mesos/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/lensacom/dask.mesos" }, "release_url": "https://pypi.org/project/dask.mesos/0.2.2/", "requires_dist": null, "requires_python": null, "summary": "Apache Mesos backend for Dask scheduling library", "version": "0.2.2" }, "last_serial": 2297783, "releases": { "0.1.3": [ { "comment_text": "", "digests": { "md5": "edc7054148357b50cb7e815546fb443d", "sha256": "21cbc104afbb58ec3f1450977056eaf5fc91653875f79f62cde276137360c4e8" }, "downloads": -1, "filename": "dask.mesos-0.1.3.tar.gz", "has_sig": false, "md5_digest": "edc7054148357b50cb7e815546fb443d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7692, "upload_time": "2016-05-09T14:01:00", "url": "https://files.pythonhosted.org/packages/37/69/a63b1dbdd300e800b21b45f411507bcb87279b8be7b88f979262948af67d/dask.mesos-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "24a516355ff1f3be8e70aeb41aab4eb6", "sha256": "6ef2750c102fbb3c6f8054a3cb346f49fff519602de9cb8c67966f956f5b1044" }, "downloads": -1, "filename": "dask.mesos-0.1.4.tar.gz", "has_sig": false, "md5_digest": "24a516355ff1f3be8e70aeb41aab4eb6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7809, "upload_time": "2016-05-13T11:11:16", "url": "https://files.pythonhosted.org/packages/c0/74/ed43404d67e50b2089b3951035077ba560fd2081c5c3a8ce085722c1c5fc/dask.mesos-0.1.4.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "d7b821de7761211d52913077de83c236", "sha256": "83d75028784721c75f321ad048dd8e0dc5958a21cca174734f71d47ca2388038" }, "downloads": -1, "filename": "dask.mesos-0.2.1.tar.gz", "has_sig": false, "md5_digest": "d7b821de7761211d52913077de83c236", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12070, "upload_time": "2016-06-13T11:37:49", "url": "https://files.pythonhosted.org/packages/10/3f/cf5c60efe92b84130a391c5d048b6c52e2a02147f0b32eeedc559486f4cc/dask.mesos-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "5e91a6c1dfc2e5b6c18ee9ed9d0d8383", "sha256": "8398a0a85db1ca442c2dc2536da33fbcebd700ef4fa67427e19683f06121dca4" }, "downloads": -1, "filename": "dask.mesos-0.2.2.tar.gz", "has_sig": false, "md5_digest": "5e91a6c1dfc2e5b6c18ee9ed9d0d8383", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12165, "upload_time": "2016-08-23T12:43:23", "url": "https://files.pythonhosted.org/packages/d9/ee/9b16a06511ebaefa9391d326d601e77dd942328ffa4b7cfbd5d70e5dc14a/dask.mesos-0.2.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5e91a6c1dfc2e5b6c18ee9ed9d0d8383", "sha256": "8398a0a85db1ca442c2dc2536da33fbcebd700ef4fa67427e19683f06121dca4" }, "downloads": -1, "filename": "dask.mesos-0.2.2.tar.gz", "has_sig": false, "md5_digest": "5e91a6c1dfc2e5b6c18ee9ed9d0d8383", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12165, "upload_time": "2016-08-23T12:43:23", "url": "https://files.pythonhosted.org/packages/d9/ee/9b16a06511ebaefa9391d326d601e77dd942328ffa4b7cfbd5d70e5dc14a/dask.mesos-0.2.2.tar.gz" } ] }