{ "info": { "author": "Cloudmesh Team", "author_email": "laszewski@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: OpenStack", "Framework :: Flask", "Intended Audience :: Developers", "Intended Audience :: Education", "Intended Audience :: Science/Research", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 2.7", "Topic :: Database", "Topic :: Scientific/Engineering", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: System :: Boot", "Topic :: System :: Clustering", "Topic :: System :: Distributed Computing", "Topic :: System :: Systems Administration" ], "description": "Cloudmesh PBS\n======================================================================\n\nCloudmesh PBS provides an easy mechanism to interface with queuing\nsystems. It is based on cloudmesh version 2 that uses separate packages\ninstead of one big cloudmesh package. The packages are named\ncloudmesh_*, where * is a placeholder for the package names.\n\nThe advantage of cloudmesh_pbs is that it can start pbs jobs on remote\nmachines while using some simple templates. These jobs are entered in\na local database and their status on the remote machines can be\nmonitored. At this time we provide a simple API, but will soon add\nalso a command interface as well as a secure rest interface.\n\nProject requirements:\n----------------------------------------------------------------------\n\n* cloudmesh_base\n \nInstalation (pending)\n----------------------------------------------------------------------\n\nThe easiest way to install cloudmesh PBS is with pip. We recommend\nthat you do it in a virtual environment. Once you have created and\nactivated a virtualenv you can install cloudmesh_pbs with the\nfollowing commands::\n\n pip install cmd3\n pip install cloudmesh_base\n pip install cloudmesh_database (not yet needed in this release)\n pip install cloudmesh_pbs\n \nGithub repository\n----------------------------------------------------------------------\n\nThe source code can be found at:\n\n* https://github.com/cloudmesh/pbs\n\nUsage\n----------------------------------------------------------------------\n\nService Specification\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nWhen dealing with remote services we often need to customize\ninterfaces and access. Instead of completely reinventing a\nspecification file, we are leveraging first the ssh config file for\nthe remote login to the servers that allow us to execute pbs\ncommands. Second we have defined a simple yaml file that allows us to\nset up some service specific items. At this time it supports the\nspecification of jobs submitted through various supercomputers that\nare either managed individually through queues, through groups of\nqueues that are managed for multiple machines in a single management\nnode.\n\nSSH Config\n~~~~~~~~~~~~~~~~~~~~~~~~\n\nWe assume that you have set up all machine in ssh config that you like\nto access with a simple keyword. For example you like to access the\nmachine cluster.example.com. We also assume you have the username\nalbert on that machine. In this case we assume you have set up a\nsimple ssh config such as::\n\n Host cluster\n Hostname cluster.example.com\n User albert\n\nNaturally once you place your public key in the authorized_hosts files\non the cluster, you will be able to log into the machine with::\n\n ssh cluster\n\nNaturally, you can try commands such as::\n\n ssh cluster uname -a\n\nYou should be able to also verify if you can execute the command qstat\nwith::\n\n ssh cluster qstat\n\nIf this all works you can specify a yaml file for cloudmesh_pbs. We\nhave included a sample yaml file in the etc directory of the source\ncode that you can modify accordingly. If we use the example you will\nhave::\n\n meta:\n yaml_version: 2.1\n kind: pbs\n cloudmesh:\n pbs:\n cluster:\n manager: cluster\n scripts: ~/qsub/india\n queues:\n - batch\n - long\n\nThis file is places in the directory ~/.cloudmesh\n\nThe important part of the file is in the cloudmesh - pbs portion. Here\nthe name of the machine that we used in .ssh/config is used,\ne.g. cluster. The manager is specified to also be the machine\ncluster. This is the machine on which the qsub and qstat commands are\nexecuted for this machine. If the management node is different it can\nbe specified here. The scripts attribute specifies where the pbs\nscripts are placed on the remote machine before they are submitted.\nTo add specific queues you simply can include them as a list in the\nqueues attribute\n\n.. note:: queue management will be enhanced\n\nAPI\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThe API to interface with the queues is straight forward and\ndocumented in more details here::\n\n TBD\n\nA simple example will show you how to submit a job and check upon its\nstatus. First we define a default host::\n\n host = \"india\" \n\nNext we declare the pbs instance that we use to interact with the\nvarious systems. Upon creation it reads the ssh config file and the\ncloudmesh yaml file::\n\n pbs = PBS(deploy=True)\n\n Next we find the manager of the host that we use to copy and to\n initiate the pbs commands on::\n \n manager = pbs.manager(host)\n\nlet us create on that host the directory ~/scripts/test::\n \n xmkdir(manager, \"~/scripts/test\")\n\nNow we need to create a pbs job script. For this we use a template that\nwe have in the etc directory::\n\n script_template = pbs.read_script(\"etc/job.pbs\")\n\nthe template contains the ability to replace the script with some real\ncommands. Let us use the uname command::\n \n script = \"\"\"\n uname -a\n \"\"\"\n\nAlso we want to give the job a unique id. For that we maintain in pbs\nan internal variable that will be increased every time we submit a\njob. We do it here with the incr command::\n\n pbs.jobid_incr()\n jobname = \"job-\" + pbs.jobid_get()\n job_script = pbs.create_script(jobname, script, script_template)\n\nLet us now submit the job to the given host::\n\n r = pbs.qsub(jobname, host, script, template=script_template)\n\nit will return the information of the job. Optionally one can define\nan output format (see the API) such as a dict or a yaml\nrepresentation. To optain the PBS variable list as a dict we can use:: \n\n d = pbs.variable_list(r)\n\n\nStatus of the job\n~~~~~~~~~~~~~~~~~~~~~~~~\n\nThe status of a job can be obtained with::\n\n r = jobstatus(self, host, jobid)\n\nIt will not only include the status, but also the environment\nvariables the job is executed in. \n \n\nTermination of the Job\n~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTBD\n\nListing of all jobs\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTBD\n\nPersistent Database\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTBD\n\nCloudmesh integration\n~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTBD\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.org/cloudmesh/pbs", "keywords": null, "license": "UNKNOWN", "maintainer": null, "maintainer_email": null, "name": "cloudmesh_pbs", "package_url": "https://pypi.org/project/cloudmesh_pbs/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/cloudmesh_pbs/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.org/cloudmesh/pbs" }, "release_url": "https://pypi.org/project/cloudmesh_pbs/2.2.6/", "requires_dist": null, "requires_python": null, "summary": "A simple pbs queue management framework for multiple supercomputers", "version": "2.2.6" }, "last_serial": 1473832, "releases": { "2.2.0": [], "2.2.1.post0": [ { "comment_text": "", "digests": { "md5": "3b1d8ad1fb6809c79ca920a814f50181", "sha256": "13bdbc3a532289f494158803054b0b15c12feb9e3b345c31bc5e25a732f2978b" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.1.post0.tar.bz2", "has_sig": false, "md5_digest": "3b1d8ad1fb6809c79ca920a814f50181", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8670, "upload_time": "2015-03-05T18:49:42", "url": "https://files.pythonhosted.org/packages/ef/2d/cf7f2f3d7eb881d73d77eb02a4d1f0379c4cc1286cb0eeb6e0645f4aff1e/cloudmesh_pbs-2.2.1.post0.tar.bz2" }, { "comment_text": "", "digests": { "md5": "f81be611b5de86fc64bfd8acf4331fcf", "sha256": "9dda301a53042158585590ec171ebb2a17f7137b0ef7e44ef629aade1e5c74ff" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.1.post0.zip", "has_sig": false, "md5_digest": "f81be611b5de86fc64bfd8acf4331fcf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13902, "upload_time": "2015-03-05T18:49:45", "url": "https://files.pythonhosted.org/packages/54/f7/a4d02fc3c2db52c70a3c1f1ea7771a5dcf7b6c1e8da52dab0537418c05b6/cloudmesh_pbs-2.2.1.post0.zip" } ], "2.2.2": [ { "comment_text": "", "digests": { "md5": "fc15a4ff9701bc34f6677a0e5c2dc2b6", "sha256": "d874d531721632c1064c6ff0b9a56559982ce9b33722685f36b65fc8fe551650" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.2.tar.bz2", "has_sig": false, "md5_digest": "fc15a4ff9701bc34f6677a0e5c2dc2b6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8639, "upload_time": "2015-03-05T18:51:33", "url": "https://files.pythonhosted.org/packages/50/f7/b3d8c8803a05e087f46e71c3cba3721cb5c3c9fcdba99c24190ea9e662c0/cloudmesh_pbs-2.2.2.tar.bz2" }, { "comment_text": "", "digests": { "md5": "ec6af2ad0a3a8ea63ba1ec65466109c0", "sha256": "374a606d31b004f6012c9bb870cd0cb1be1666dc7ae4611b048cb5e207b3ef8d" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.2.zip", "has_sig": false, "md5_digest": "ec6af2ad0a3a8ea63ba1ec65466109c0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13669, "upload_time": "2015-03-05T18:51:36", "url": "https://files.pythonhosted.org/packages/a5/21/777ea9885fd47f6b1e110eb92c7ab799e9eb36fae07e1f5d5c0afcc35568/cloudmesh_pbs-2.2.2.zip" } ], "2.2.3": [ { "comment_text": "", "digests": { "md5": "3f5dfde8f1ebf279b978fcb63d1ea46f", "sha256": "4bfdcdd2b2488ff0b1323935227a00abf3b72ac286e0991a877a184761dbf165" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.3.tar.bz2", "has_sig": false, "md5_digest": "3f5dfde8f1ebf279b978fcb63d1ea46f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9325, "upload_time": "2015-03-06T08:28:39", "url": "https://files.pythonhosted.org/packages/3f/e3/1e447888324749b49e440229efcc111d39e2bbce3ab11722cf66e1881fce/cloudmesh_pbs-2.2.3.tar.bz2" }, { "comment_text": "", "digests": { "md5": "4fc13831bfd1e4d765d8dcf55f42d92f", "sha256": "7212938c940f1e497236a398ee09e7da10df193c88e2df10aef71ca98dfcdd51" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.3.zip", "has_sig": false, "md5_digest": "4fc13831bfd1e4d765d8dcf55f42d92f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14807, "upload_time": "2015-03-06T08:28:41", "url": "https://files.pythonhosted.org/packages/15/6b/e2f57f0cf88cd33e99762677a970491b36ed8fbe303179d826ea8a37c239/cloudmesh_pbs-2.2.3.zip" } ], "2.2.4": [ { "comment_text": "", "digests": { "md5": "b6c685380ff5281d8a621576d313773f", "sha256": "4770993793b63567593f12a262e4755871437a2bbc0e08a5721fbcdd7b9518bd" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.4.tar.bz2", "has_sig": false, "md5_digest": "b6c685380ff5281d8a621576d313773f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8858, "upload_time": "2015-03-23T16:00:17", "url": "https://files.pythonhosted.org/packages/9d/75/e3c9551aaa768afbebea8832edc1943e99ef042412e97fd4e86480e0969a/cloudmesh_pbs-2.2.4.tar.bz2" }, { "comment_text": "", "digests": { "md5": "ac74cff3b702c8afbe0c47a6404f505f", "sha256": "88dcb11f05420499397ed959b70a7fd60e74f5d863807ecf69159abf463a583b" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.4.zip", "has_sig": false, "md5_digest": "ac74cff3b702c8afbe0c47a6404f505f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13596, "upload_time": "2015-03-23T16:00:22", "url": "https://files.pythonhosted.org/packages/e1/0c/390a556407a43c78583a22eab3776aed1952e7b87e90fb3f8f706d3d5a34/cloudmesh_pbs-2.2.4.zip" } ], "2.2.6": [ { "comment_text": "", "digests": { "md5": "1dc5099b78b4eb308bf33a01399220ad", "sha256": "2baf24cff2ef78512b7bd6190e6d8022f2e69341878624c28f3d3b3d7c49b101" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.6.tar.bz2", "has_sig": false, "md5_digest": "1dc5099b78b4eb308bf33a01399220ad", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8810, "upload_time": "2015-03-23T16:10:29", "url": "https://files.pythonhosted.org/packages/a3/2e/5c5333901f8bffb3c186f5ba8bb3a4951df38d3fa1297bef088d519f8197/cloudmesh_pbs-2.2.6.tar.bz2" }, { "comment_text": "", "digests": { "md5": "6e5856e5ccb35ed6243d61b17c0b4672", "sha256": "b97f8025b3c298fd494d503133ab042584041f85af3f2b1cb9efe1ddf1ba34a2" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.6.zip", "has_sig": false, "md5_digest": "6e5856e5ccb35ed6243d61b17c0b4672", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13510, "upload_time": "2015-03-23T16:10:32", "url": "https://files.pythonhosted.org/packages/be/b4/18cc8ac657708b64bf5810333ea48741780445a927808d09ec1fb91b52f9/cloudmesh_pbs-2.2.6.zip" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1dc5099b78b4eb308bf33a01399220ad", "sha256": "2baf24cff2ef78512b7bd6190e6d8022f2e69341878624c28f3d3b3d7c49b101" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.6.tar.bz2", "has_sig": false, "md5_digest": "1dc5099b78b4eb308bf33a01399220ad", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8810, "upload_time": "2015-03-23T16:10:29", "url": "https://files.pythonhosted.org/packages/a3/2e/5c5333901f8bffb3c186f5ba8bb3a4951df38d3fa1297bef088d519f8197/cloudmesh_pbs-2.2.6.tar.bz2" }, { "comment_text": "", "digests": { "md5": "6e5856e5ccb35ed6243d61b17c0b4672", "sha256": "b97f8025b3c298fd494d503133ab042584041f85af3f2b1cb9efe1ddf1ba34a2" }, "downloads": -1, "filename": "cloudmesh_pbs-2.2.6.zip", "has_sig": false, "md5_digest": "6e5856e5ccb35ed6243d61b17c0b4672", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13510, "upload_time": "2015-03-23T16:10:32", "url": "https://files.pythonhosted.org/packages/be/b4/18cc8ac657708b64bf5810333ea48741780445a927808d09ec1fb91b52f9/cloudmesh_pbs-2.2.6.zip" } ] }