{ "info": { "author": "Tyler Gannon", "author_email": "tyler.gannon@interxinc.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3" ], "description": "# remote_sge: Remotely submit jobs to GridEngine\n\n* Remotely submit jobs to a GridEngine cluster.\n* Retrieve job results.\n* Automated transfer of jobs from one cluster to another.\n\nRemote SGE is a tool for transferring queued jobs to a remote SGE cluster,\nas well as for submitting jobs to SGE from a machine other than the cluster's\nmaster host.\n\n* [See here](https://bitbucket.org/tyler_gannon/remote_sge/wiki/Configuration%20Common%20Issues) for common configuration problems.\n\n# Usage\n\n```bash\n# Holds jobs with ids in the given range, collects all the required data, and\n# sends them to the configured remote.\nremote_sge shuttle 12300-12399\n\n# Later...\n# Downloads results from any remote jobs that have finished, and unpacks the\n# results in the working directory of the local jobs.\n# Finally, terminates (deletes) the original jobs from the local queue.\nremote_sge collect\n```\n\n# Installation\n\nThe installation utility targets Centos 6.x and Amazon Linux 1 (cfncluster).\n\n## Install Base Packages (Client AND Server)\n\n```bash\n# Feel free to change PY_VERSION to the latest, or to pypy or your favorite\n# version available within pyenv.\ncurl -o install.sh \\\n -L https://gist.githubusercontent.com/tylergannon/edf45c882b9459ac4c13cfcf0f581d1d/raw \\\n && chmod +x install.sh\n./install.sh \n\n# Restart your shell, you can log out and log in again, or else:\n$SHELL\n```\n\nThis will:\n\n* Install YUM packages required for pyenv.\n* Install latest version of pyenv.\n* Install Python 3.6.4 and create a virtualenv called `remote_sge`, based on that version.\n\n## Install the Remote SGE package. (Client AND Server)\n\nYou can also type `pyenv activate remote_sge` and then `pip` and other executables will be added\nto your path.\n\nFirst, download the latest wheel from https://bitbucket.org/tyler_gannon/remote_sge\n\n```bash\n# Activate your virtualenv:\npyenv activate remote_sge\n# Install the package:\npip install /path/to/remote-sge-xxx.whl\n```\n\n## Install the Remote SGE Server Components.\n\n```bash\n#\n# Activate your virtualenv if not already active:\npyenv activate remote_sge\n# Type remote_sge install_server -h for a full list of command-line options.\n# The below assumes that vim is installed and you know how to use it.\nremote_sge install_server -i\n```\n\nThe installer will ask you to edit a number of configuration files. Make changes as needed,\nsave and quit the editor, and the file will be copied into place.\n\n\nHere are the particular settings to pay attention to:\n\n### Main Config File:\n\n* Client Section:\n * You can safely remove this section from the server component; won't be used.\n* Server Section:\n * `work_root` should reflect the location where each job's working directory should run.\n Each job will be given a randomly-named working directory within this location.\n * `completed_files_root` should reflect the location where job results will be placed.\n * `nginx_listen` can be a port number, or an `ip_address:port` assignment.\n* Command Environment Section:\n * A list of environment variables. Use this section to configure locations of any resources\n your jobs might use, which are located outside of the job's working directory.\n\n## Test the function of the server component:\n\n```bash\n# Change the location of the config dir, if you changed the default for config files.\nremote_sge test_server ~/.config/remote_sge -P 443 -h localhost\n```\n\nIf the server is configured correctly and running, you'll see `It works!`, and you should continue.\n\n## Configure the client component:\n\nThe server install will have placed the required client certificates in a tgz file at\n`~/remote_sge_client_certs.tgz`.\n\n```bash\n# Copy the certificate files to the client machine\nssh \nscp me@server_machine:~/remote_sge_client_certs.tgz .\n\n# Do the base install (see Install Base Packages above):\ncurl -L https://goo.gl/CszXDP | PY_VERSION=3.6.4 bash\n\n# Restart your shell:\n$SHELL\n\npyenv activate remote_sge\n# Ensure that the wheel file has been downloaded to this machine, and then:\npip install /path/to/remote-sge-xxx.whl\n\nremote_sge install_client -i -c /path/to/remote_sge_client_certs.tgz\n```\n\nYou'll be prompted to correct the hostname and port number of your server instance.\nThe other settings can likely be left alone.\n\n## Test client connectivity\n\nJust to make sure that you have connectivity to the server:\n\n```bash\nremote_sge test_server ~/.config/remote_sge -P 443 -h mysge.mycompany.com\n```\n\n# Set up for development:\n\n\n``pip install -e .[testing]``\n\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/tylergannon/remote_sge", "keywords": "", "license": "Apache2", "maintainer": "", "maintainer_email": "", "name": "remote-sge", "package_url": "https://pypi.org/project/remote-sge/", "platform": "", "project_url": "https://pypi.org/project/remote-sge/", "project_urls": { "Homepage": "https://github.com/tylergannon/remote_sge" }, "release_url": "https://pypi.org/project/remote-sge/0.2.8/", "requires_dist": [ "requests (==2.18.4)", "filelock (>=2.0)", "falcon (>=1.4.0rc2)", "gunicorn (>=19.7.1)", "configobj (>=5.0.6)", "pytest (==3.6.0); extra == 'testing'", "pytest-mock; extra == 'testing'", "pytest-bdd; extra == 'testing'", "pytest-timeout; extra == 'testing'", "twine; extra == 'testing'", "vcrpy; extra == 'testing'", "pylint; extra == 'testing'", "setuptools; extra == 'testing'", "wheel; extra == 'testing'", "requests-unixsocket; extra == 'testing'" ], "requires_python": "", "summary": "Utilities for remote submission and collection of GridEngine jobs.", "version": "0.2.8" }, "last_serial": 4267326, "releases": { "0.1.20": [ { "comment_text": "", "digests": { "md5": "62d72aca539d1e40958ab429e1263960", "sha256": "abcd2aa8b97fb1df9c1273f0cd34c0b30383f05afd740e22b0e518219d0d4dfa" }, "downloads": -1, "filename": "remote_sge-0.1.20-py3-none-any.whl", "has_sig": false, "md5_digest": "62d72aca539d1e40958ab429e1263960", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 37947, "upload_time": "2018-01-12T21:15:23", "url": "https://files.pythonhosted.org/packages/06/b6/51bf492a411c3b4dc37050e00dc4ae622d0b91d0ae685e8c051409adb5e0/remote_sge-0.1.20-py3-none-any.whl" } ], "0.1.21": [ { "comment_text": "", "digests": { "md5": "8e42b847db18445d8a1eff1859a69677", "sha256": "d5535b2946f40a05ffe13dccd477892497b62624cc3557b00a34a1c7ca0137dc" }, "downloads": -1, "filename": "remote_sge-0.1.21-py3-none-any.whl", "has_sig": false, "md5_digest": "8e42b847db18445d8a1eff1859a69677", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 37966, "upload_time": "2018-01-12T21:21:26", "url": "https://files.pythonhosted.org/packages/24/aa/67b376a71dbb0133d19831ac8b1b34bf3bff70e571de32b90cfb8f6a5bd0/remote_sge-0.1.21-py3-none-any.whl" } ], "0.1.22": [ { "comment_text": "", "digests": { "md5": "7e7ee3a13435d1d07e25ab3b018bf805", "sha256": "3fade5e478b49cd935a1d2290758b8cc74844927f40a3841ae9f741c3c9067fb" }, "downloads": -1, "filename": "remote_sge-0.1.22-py3-none-any.whl", "has_sig": false, "md5_digest": "7e7ee3a13435d1d07e25ab3b018bf805", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 37958, "upload_time": "2018-01-12T21:28:57", "url": "https://files.pythonhosted.org/packages/fa/59/a573296495b2130cfd9b16b0c59eb1e43eeb3ceb3c0b06a64007f5f45a87/remote_sge-0.1.22-py3-none-any.whl" } ], "0.1.23": [ { "comment_text": "", "digests": { "md5": "98d66c74a9123028af6c9c87adfcb8be", "sha256": "9ac5ecbe731183ae78f63e061baa1bf7ad9dc1a8482e4da48c04e44ca354af4a" }, "downloads": -1, "filename": "remote_sge-0.1.23-py3-none-any.whl", "has_sig": false, "md5_digest": "98d66c74a9123028af6c9c87adfcb8be", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 37959, "upload_time": "2018-01-12T21:33:50", "url": "https://files.pythonhosted.org/packages/1b/b8/53abc781aa34b67d41d92bce184f24d3f86843c692d95014434bc5e73255/remote_sge-0.1.23-py3-none-any.whl" } ], "0.2.7": [ { "comment_text": "", "digests": { "md5": "f8ca9a38a101e1b03dbf32b0b1638346", "sha256": "38319c4ddee46adb8e7b56cf9ad2f4ab6b3bbe30a41ba45654b70ad919b2723a" }, "downloads": -1, "filename": "remote_sge-0.2.7-py3-none-any.whl", "has_sig": false, "md5_digest": "f8ca9a38a101e1b03dbf32b0b1638346", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 50432, "upload_time": "2018-09-13T03:20:29", "url": "https://files.pythonhosted.org/packages/9b/b0/0c1c5fe2fdc9cce70b1b7bf8e276ca2903f013fd1b95a858f92e61c17b31/remote_sge-0.2.7-py3-none-any.whl" } ], "0.2.8": [ { "comment_text": "", "digests": { "md5": "f9279e4b3b266d3e7d17e043460864e7", "sha256": "d74b665887c04a8e6b4eb8049cfb862dfa70dd05768133ee5c178b126485ae60" }, "downloads": -1, "filename": "remote_sge-0.2.8-py3-none-any.whl", "has_sig": false, "md5_digest": "f9279e4b3b266d3e7d17e043460864e7", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 50467, "upload_time": "2018-09-13T03:47:01", "url": "https://files.pythonhosted.org/packages/de/a5/236a3bd5709ef24111607a1376dc8bd57bb0789267a1f04943470b917977/remote_sge-0.2.8-py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f9279e4b3b266d3e7d17e043460864e7", "sha256": "d74b665887c04a8e6b4eb8049cfb862dfa70dd05768133ee5c178b126485ae60" }, "downloads": -1, "filename": "remote_sge-0.2.8-py3-none-any.whl", "has_sig": false, "md5_digest": "f9279e4b3b266d3e7d17e043460864e7", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 50467, "upload_time": "2018-09-13T03:47:01", "url": "https://files.pythonhosted.org/packages/de/a5/236a3bd5709ef24111607a1376dc8bd57bb0789267a1f04943470b917977/remote_sge-0.2.8-py3-none-any.whl" } ] }