{ "info": { "author": "Shane Breeze", "author_email": "sdb15@ic.ac.uk", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Utilities" ], "description": "```\n pysge + +\n /\\ /\\ \\ O \\\n / \\\\ /\\ /\\\\\\ \\_/|\\_\\\n / \\\\/ \\\\ / \\\\\\ \\\\\n / \\/ \\\\\\ \\/'\n / / \\\\\\ \\\\_\n / / \\\\\\ -\n```\n\nCall python functions on (IC) SGE batch cluster - with tqdm progress bars.\nDesigned to work inside a jupyter notebook.\n\n## How to use\n\n[Docs](https://shane-breeze.github.io/pysge/).\n\nHigh-level interface use:\n\n```\nimport pysge\n\ndef myfunc(a, b):\n return a + b\n\ntasks = [\n {\"task\": myfunc, \"args\": (1, 2), \"kwargs\": {}},\n {\"task\": myfunc, \"args\": (3, 4), \"kwargs\": {}},\n {\"task\": myfunc, \"args\": (5, 6), \"kwargs\": {}},\n]\n\nresults = pysge.local_submit(tasks)\nresults = pysge.mp_submit(tasks, ncores=3)\nresults = pysge.sge_submit(tasks, \"name\", \"/tmp/pysge-temporaries\", options=\"-q hep.q\")\n```\n\nThe return value is a list of results for each task, in order of tasks.\n\n# How it works\n\nFor SGE batch system a working area is created and the functions + args + kwargs are dilled. A submitter then submits each dilled file to the batch using subprocess. A monitor checks the status of these jobs, waits until all are finished and returns the results.\n\nFor multiprocessing no working area is needed and the submitter uses multiprocessing.\n\nFor local no submitter is needed. It just loops over the tasks.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "pysge", "package_url": "https://pypi.org/project/pysge/", "platform": "", "project_url": "https://pypi.org/project/pysge/", "project_urls": null, "release_url": "https://pypi.org/project/pysge/0.2.4/", "requires_dist": [ "future (>=0.17.1)", "tqdm (>=4.31.1)", "subprocess32 (>=3.5.4)", "lz4 (>=2.1.10)", "dill (>=0.3.0)" ], "requires_python": "", "summary": "python interface to submit functions to an SGE batch cluster", "version": "0.2.4" }, "last_serial": 5954712, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "6f1f53603e16e968f97bb15941b71f1c", "sha256": "67e0536d6f59e903ce5482991baf3b5368457342e2143a9cd20c16c6a88c43c8" }, "downloads": -1, "filename": "pysge-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "6f1f53603e16e968f97bb15941b71f1c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9474, "upload_time": "2019-06-06T14:04:41", "url": "https://files.pythonhosted.org/packages/71/7f/56761b9ea30e2626d95ab3270b25f29a9ee2f883259566225b27e228575f/pysge-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3f0d90d35b50a8267b6e73cfac285f0b", "sha256": "194a8a1fb13f09fcc799421873855966ea85e42999c667f446a2f016706e2711" }, "downloads": -1, "filename": "pysge-0.1.0.tar.gz", "has_sig": false, "md5_digest": "3f0d90d35b50a8267b6e73cfac285f0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5873, "upload_time": "2019-06-06T14:04:43", "url": "https://files.pythonhosted.org/packages/ec/2d/55792f7b81c2cf9174cf9bc26467be4f1049cba8c93ee531ff8c158f8288/pysge-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "6d0e3ca25299b9a0bc5387c9159eefe4", "sha256": "d85986344f7b6fd89d29cdd69ce56c59f0c36bef0a6b4ba535e25a2d84828e4c" }, "downloads": -1, "filename": "pysge-0.1.1-py3-none-any.whl", "has_sig": false, "md5_digest": "6d0e3ca25299b9a0bc5387c9159eefe4", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9408, "upload_time": "2019-06-19T16:28:28", "url": "https://files.pythonhosted.org/packages/50/b7/169669e761f83606a849b71e5bc35bc4c43073af023c634c39ac7acd386e/pysge-0.1.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "be76664101d2a6daaa35b8c0abbfeb1a", "sha256": "b6353d9d2d237f5d05f7ec2df07c07d79748a05efe0f96a9f208675ba9c949ab" }, "downloads": -1, "filename": "pysge-0.1.1.tar.gz", "has_sig": false, "md5_digest": "be76664101d2a6daaa35b8c0abbfeb1a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5822, "upload_time": "2019-06-19T16:28:30", "url": "https://files.pythonhosted.org/packages/62/be/a7464b97c5ce938fb22c89e1c1d63fda79b554ae12ef37ce810334662771/pysge-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "a56ff0c9c312aef210802a0a56c6062b", "sha256": "a5c5eada444c0827addd3eea6cd63d2cf0cc0939e80f49d1f9c08f4f9bde51b1" }, "downloads": -1, "filename": "pysge-0.1.2-py3-none-any.whl", "has_sig": false, "md5_digest": "a56ff0c9c312aef210802a0a56c6062b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9422, "upload_time": "2019-07-27T10:43:53", "url": "https://files.pythonhosted.org/packages/d2/94/eca062df36fdf06afc93bd4b8d56eb7f4b83a4f4e1b3efc99baeaef4f8c7/pysge-0.1.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0002cf2e5506e8eeb1688c7b68574455", "sha256": "ac9229ce010c2fcf35a5a3a6715c446f3455fe7137b2ef05ab12feedb4ee46aa" }, "downloads": -1, "filename": "pysge-0.1.2.tar.gz", "has_sig": false, "md5_digest": "0002cf2e5506e8eeb1688c7b68574455", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5841, "upload_time": "2019-07-27T10:43:56", "url": "https://files.pythonhosted.org/packages/35/ee/5a72304f6c5d940d62986a594a0dc4a575f93e9213f6bc988b202df9e8b8/pysge-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "db15ebd1313856fb3fbb53f290c0ebb1", "sha256": "57f0500b7f6b48dc5d244408108176b350978ae6f8bea987e70a026be70ccce7" }, "downloads": -1, "filename": "pysge-0.1.3-py3-none-any.whl", "has_sig": false, "md5_digest": "db15ebd1313856fb3fbb53f290c0ebb1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9030, "upload_time": "2019-07-27T11:46:56", "url": "https://files.pythonhosted.org/packages/1c/f2/110d6f7c92315208c196aec202f2011a8f3c3d5a5f7f901f56eb37421538/pysge-0.1.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "374ddb9bb19901b3613e8c87d3fecd28", "sha256": "4304f863c512033c0f978f14eee327f9b027aa4120b12877e65a461419c47db3" }, "downloads": -1, "filename": "pysge-0.1.3.tar.gz", "has_sig": false, "md5_digest": "374ddb9bb19901b3613e8c87d3fecd28", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5690, "upload_time": "2019-07-27T11:46:58", "url": "https://files.pythonhosted.org/packages/05/55/07af4dee2809b0eda52964bebd7df03cf6305fdee1620f51a64724072b2f/pysge-0.1.3.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "ec65de88eddf7d54e261571a9f305b1c", "sha256": "2993742b1071c2e8bc7d477b1524be45b53084e186a53a290df204c78678df6b" }, "downloads": -1, "filename": "pysge-0.1.4-py3-none-any.whl", "has_sig": false, "md5_digest": "ec65de88eddf7d54e261571a9f305b1c", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9111, "upload_time": "2019-08-14T09:12:39", "url": "https://files.pythonhosted.org/packages/48/93/9422a8831b2f879b16d24f1a0adc81370ae999d269366ddbcbea1752fd34/pysge-0.1.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e0edea2cf7cc5ce8781cb6f017e57d08", "sha256": "bf399cf040a39dad6bcd118a512940f6b291b7ce7f036f55d5ee162866ddb203" }, "downloads": -1, "filename": "pysge-0.1.4.tar.gz", "has_sig": false, "md5_digest": "e0edea2cf7cc5ce8781cb6f017e57d08", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5736, "upload_time": "2019-08-14T09:12:40", "url": "https://files.pythonhosted.org/packages/84/1f/94a371b832867b076b41a07ffbcb234a95e31198691758961ae1015fa5db/pysge-0.1.4.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "9c7d064bcacb664b6b04e8c2badc3b4d", "sha256": "9ea886577f9332ad29413330616a0521ebc362eccd003a7bccbe08c2a2350ff0" }, "downloads": -1, "filename": "pysge-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "9c7d064bcacb664b6b04e8c2badc3b4d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9116, "upload_time": "2019-08-22T12:35:53", "url": "https://files.pythonhosted.org/packages/0f/46/75dbe18c3233d4093a4ad33ef216d2522eceb010c1830ab5c720b3f78278/pysge-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "01ca45ed0d3d691403e2a79765830a57", "sha256": "d7cf66f57e8412dd65cff0c20e0f4985d749d0ce69b924218b1ef0dff1707212" }, "downloads": -1, "filename": "pysge-0.2.0.tar.gz", "has_sig": false, "md5_digest": "01ca45ed0d3d691403e2a79765830a57", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5818, "upload_time": "2019-08-22T12:35:55", "url": "https://files.pythonhosted.org/packages/d2/b6/c5dce0bf3e6f287f5dfe72f59aeb74ba2358f446e2f20e1e7f0ebfd08209/pysge-0.2.0.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "5888e7c718966fc962b4b8f88f82df82", "sha256": "85b69d86208211f72fe73ebe9955b77c4f00fb950e308de11afe367ff6169c6a" }, "downloads": -1, "filename": "pysge-0.2.3-py3-none-any.whl", "has_sig": false, "md5_digest": "5888e7c718966fc962b4b8f88f82df82", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9979, "upload_time": "2019-08-22T13:46:55", "url": "https://files.pythonhosted.org/packages/75/02/f2b39965640f11123e87174aa7ca334386e06f82684e02e2b4f1185d64b1/pysge-0.2.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7a9cede6e18c6e4c3a56641da162416b", "sha256": "9383c50ee844c55a807300347f0da16c01a77e4bc5469b94a40fca1af830890d" }, "downloads": -1, "filename": "pysge-0.2.3.tar.gz", "has_sig": false, "md5_digest": "7a9cede6e18c6e4c3a56641da162416b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6601, "upload_time": "2019-08-22T13:46:56", "url": "https://files.pythonhosted.org/packages/26/3f/3d9e6d5885946ce1da4a834261d630797e1dd79e6bbf291c07350998eb0e/pysge-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "af0bb6445f5f8461880d89f2771d313e", "sha256": "a8ce76d76bbe98f82b7d9e3e24dfabe77c6ef70993be34ba5afeea856b3b88ef" }, "downloads": -1, "filename": "pysge-0.2.4-py3-none-any.whl", "has_sig": false, "md5_digest": "af0bb6445f5f8461880d89f2771d313e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 10705, "upload_time": "2019-10-10T13:17:16", "url": "https://files.pythonhosted.org/packages/c9/28/66515d254d476ae04d0a6f1c47d94380a8c2816adc82848dcb69b01cbba1/pysge-0.2.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "979e6c0b2e6b8c0f5cbdca7ef80e60e2", "sha256": "991d831583e9df088a9e5e1422b7b47e773b9a8734d1c007aea8c9fcdc9c569d" }, "downloads": -1, "filename": "pysge-0.2.4.tar.gz", "has_sig": false, "md5_digest": "979e6c0b2e6b8c0f5cbdca7ef80e60e2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7736, "upload_time": "2019-10-10T13:17:19", "url": "https://files.pythonhosted.org/packages/0c/84/f2e83727aacc684bbf9ee8b3f5bb5633c54bf4ffac292139e5630cc6ebb4/pysge-0.2.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "af0bb6445f5f8461880d89f2771d313e", "sha256": "a8ce76d76bbe98f82b7d9e3e24dfabe77c6ef70993be34ba5afeea856b3b88ef" }, "downloads": -1, "filename": "pysge-0.2.4-py3-none-any.whl", "has_sig": false, "md5_digest": "af0bb6445f5f8461880d89f2771d313e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 10705, "upload_time": "2019-10-10T13:17:16", "url": "https://files.pythonhosted.org/packages/c9/28/66515d254d476ae04d0a6f1c47d94380a8c2816adc82848dcb69b01cbba1/pysge-0.2.4-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "979e6c0b2e6b8c0f5cbdca7ef80e60e2", "sha256": "991d831583e9df088a9e5e1422b7b47e773b9a8734d1c007aea8c9fcdc9c569d" }, "downloads": -1, "filename": "pysge-0.2.4.tar.gz", "has_sig": false, "md5_digest": "979e6c0b2e6b8c0f5cbdca7ef80e60e2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7736, "upload_time": "2019-10-10T13:17:19", "url": "https://files.pythonhosted.org/packages/0c/84/f2e83727aacc684bbf9ee8b3f5bb5633c54bf4ffac292139e5630cc6ebb4/pysge-0.2.4.tar.gz" } ] }