{ "info": { "author": "Acrisel Team", "author_email": "support@acrisel.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Other Environment", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 3", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: System :: Distributed Computing" ], "description": "=======\nsshpipe\n=======\n\n---------------------------------------------------\nSSH tools to manage and channel data to remote host\n---------------------------------------------------\n\n.. contents:: Table of Contents\n :depth: 2\n\nOverview\n========\n\n .. _Eventor: https://github.com/Acrisel/eventor\n .. _Sequent: https://github.com/Acrisel/sequent\n \n *sshpipe* was build as part of Eventor_ and Sequent_ to allow distributed processing using *SSH*. Usually, network based system are using ports to communicate between server and clients. However, in development environment, there may be many developers in need to individual port assignments. The management of such operation can become overwhelming.\n \n With SSH tunneling, each developer can manage its own environment. Using virtualenvs and SSH keys, developer can manage himself connections between servers and clients they are working on. \n \n If you have comments or insights, please don't hesitate to contact us at support@acrisel.com\n\nsshconfig\n=========\n\t\n sshconfig is used to read SSH configuration file and give access to information stored there. It can be used also to save SSH configuration.\n \nLoads and dumps\n---------------\n \n *loads()*, *load()* methods are used to read SSH configuration from string stream or file respectively into *SSHConfig* object. \n *dumps()*, *dump()* methods are used to write *SSHConfig* object to string stream or file respectively.\n \nSSHConfig Class\n---------------\n\n SSHConfig class holds SSH configuration as read by *load()* or *loads()*. It can then be stored back into SSH configuration file with sshconfig's *dump()* and *dumps(). *SSHConfig* provides *get()* method to retrieve SSH settings.\n\n Future functionality:\n \n 1. validation of configuration.\n #. manipulation of configuration (e.g., add key, change flags, etc.)\n \nSSHPipe\n=======\n\n SSHPipe class is used to initiate an SSH channel to a process running in remote host. SSHPipe is initiated with the command for the agent process. It would then start the agent (commonly an object of *SSHPipeClient* or of a class inheriting from it.) \n \n Once agent is started, SSHPipe provides methods to send the agent work assignments. When agent is done or fails, it would communicate back to SSHPipe object. The method *response()* can be used to retrieve that response.\n \nExample\n-------\n\n This example shows a SSHPipe creation from one host to another with. The service in the remote host will accept string message sent via the pipe and would store them into a file.\n \n *sshremotehandlerusage.py*, below, initiates \n\n .. code:: python\n :number-lines:\n \n import os\n import multiprocessing as mp\n from sshpipe import SSHPipe\n\n def run():\n agent_dir = '/var/acrisel/sand/acris/sshpipe/sshpipe/sshpipe_examples' \n agentpy = os.path.join(agent_dir, \"sshremotehandler.py\")\n host = 'ubuntud01_eventor' # SSH config host name of remote server.\n \n sshagent = SSHPipe(host, agentpy)\n sshagent.start()\n \n if not sshagent.is_alive():\n print(\"Agent not alive\", sshagent.response())\n exit(1)\n \n sshagent.send(\"This is life.\\n\")\n sshagent.send(\"This is also life.\\n\")\n sshagent.send(\"This is yet another life.\\n\")\n sshagent.send(\"That is all, life.\\n\")\n sshagent.send(\"TERM\")\n \n if not sshagent.is_alive():\n print(sshagent.response())\n exit()\n \n response = sshagent.close()\n if response:\n exitcode, stdout, stderr = response\n print('Response: ', response)\n \n if __name__ == '__main__':\n mp.set_start_method('spawn')\n run()\n \n The remote agent *sshremotehandler.py* is would run through SHHPipe and would loop awaiting input on its *stdin* stream. \n \n .. code:: python\n :number-lines:\n \n import logging\n from sshpipe import SSHPipeHandler\n\n module_logger = logging.getLogger(__file__)\n\n class MySSHPipeHandler(SSHPipeHandler):\n \n def __init__(self, *args, **kwargs):\n super(MySSHPipeHandler, self).__init__(*args, **kwargs)\n self.file = None\n \n def atstart(self, received):\n file = \"{}{}\".format(__file__, \".remote.log\")\n self.module_logger.debug(\"Opening file: {}.\".format(file))\n self.file = open(file, 'w')\n \n def atexit(self, received):\n if self.file is not None:\n self.file.close()\n super(MySSHPipeHandler, self).atexit(received)\n \n def handle(self, received):\n self.file.write(str(received)) \n \n if __name__ == '__main__':\n client = MySSHPipeHandler()\n client.service_loop()\n \n The handler overrides the four methods of *SSHPipeHandler*. *__init__()* defines an instance member *file*, *atstart()* opens file to which records would be written, *atexit()* closes the file, and *handle()* writes received record to file.\n \nExample Explanation\n-------------------\n\nLets say we run *sshremotehandlerusage.py* program on some server, ubuntud20\n \nClasses and Methods\n-------------------\n\n .. code:: python\n \n SSHPipe(host, command, name=None, user=None, term_message='TERM', config=None, encoding='utf8', callback=None, logger=None)\n \n SSHPipe establishes connection to remote *host* and runs *command*. *host* can be ip address, hostname, or SSH host name.\n *name* associates and id to the pipe. If *user* is provided, it will use for the SSH connectivity. term_message, is \n \nSSHPipeClient\n=============\n\n\n\n\t\nexample\n-------\n\n .. code-block:: python\n\t\n import logging\n\t\n # create logger\n logger = logging.getLogger('simple_example')\n logger.setLevel(logging.DEBUG)\n\t\n # create console handler and set level to debug\n ch = logging.TimedRotatingFileHandler()\n ch.setLevel(logging.DEBUG)\n\t\n # create formatter\n formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')\n\t\n # add formatter to ch\n ch.setFormatter(formatter)\n\t\n # add ch to logger\n logger.addHandler(ch)\n\t\n # 'application' code\n logger.debug('debug message')\n logger.info('info message')\n logger.warn('warn message')\n logger.error('error message')\n logger.critical('critical message')\t\n\nMpLogger and LevelBasedFormatter\n================================\n\n Multiprocessor logger using QueueListener and QueueHandler\n It uses TimedSizedRotatingHandler as its logging handler\n\n It also uses acris provided LevelBasedFormatter which facilitate message formats\n based on record level. LevelBasedFormatter inherent from logging.Formatter and\n can be used as such in customized logging handlers. \n\t\nexample\n-------\n\nWithin main process\n~~~~~~~~~~~~~~~~~~~\n\n .. code-block:: python\n\t\n import time\n import random\n import logging\n from acris import MpLogger\n import os\n import multiprocessing as mp\n\n def subproc(limit=1, logger_info=None):\n logger=MpLogger.get_logger(logger_info, name=\"acrilog.subproc\", )\n \t\tfor i in range(limit):\n sleep_time=3/random.randint(1,10)\n time.sleep(sleep_time)\n logger.info(\"proc [%s]: %s/%s - sleep %4.4ssec\" % (os.getpid(), i, limit, sleep_time))\n\n level_formats={logging.DEBUG:\"[ %(asctime)s ][ %(levelname)s ][ %(message)s ][ %(module)s.%(funcName)s(%(lineno)d) ]\",\n 'default': \"[ %(asctime)s ][ %(levelname)s ][ %(message)s ]\",\n }\n \n mplogger=MpLogger(logging_level=logging.DEBUG, level_formats=level_formats, datefmt='%Y-%m-%d,%H:%M:%S.%f')\n logger=mplogger.start(name='main_process')\n\n logger.debug(\"starting sub processes\")\n procs=list()\n for limit in [1, 1]:\n proc=mp.Process(target=subproc, args=(limit, mplogger.logger_info(),))\n procs.append(proc)\n proc.start()\n \n for proc in procs:\n if proc:\n proc.join()\n \n logger.debug(\"sub processes completed\")\n\n mplogger.stop()\t\n \n \nExample output\n--------------\n\n .. code-block:: python\n\n [ 2016-12-19,11:39:44.953189 ][ DEBUG ][ starting sub processes ][ mplogger.(45) ]\n [ 2016-12-19,11:39:45.258794 ][ INFO ][ proc [932]: 0/1 - sleep 0.3sec ]\n [ 2016-12-19,11:39:45.707914 ][ INFO ][ proc [931]: 0/1 - sleep 0.75sec ]\n [ 2016-12-19,11:39:45.710487 ][ DEBUG ][ sub processes completed ][ mplogger.(56) ]\n \nClarification of parameters\n===========================\n\nname\n----\n\n**name** identifies the base name for logger. Note the this parameter is available in both MpLogger init method and in its start method.\n\nMpLogger init's **name** argument is used for consolidated logger when **consolidate** is set. It is also used for private logger of the main process, if one not provided when calling *start()* method. \n\nproecess_key\n------------\n\n**process_key** defines one or more logger record field that would be part of the file name of the log. In case it is used, logger will have a file per records' process key. This will be in addition for a consolidated log, if **consolidate** is set. \n\nBy default, MpLogger uses **name** as the process key. If something else is provided, e.g., **processName**, it will be concatenated to **name** as postfix. \n\nfile_prefix and file_suffix\n---------------------------\n\nAllows to distinguish among sets of logs of different runs by setting one (or both) of **file_prefix** and **file_suffix**. Usually, the use of PID and granular datetime as prefix or suffix would create unique set of logs.\n\nfile_mode\n---------\n\n**file_mode** let program define how logs will be opened. In default, logs are open in append mode. Hense, history is collected and file a rolled overnight and by size. \n\nconsolidate\n----------- \n\n**consolidate**, when set, will create consolidated log from all processing logs.\nIf **consolidated** is set and *start()* is called without **name**, consolidation will be done into the main process.\n\nkwargs\n------\n\n**kwargs** are named arguments that will passed to FileHandler. This include:\n | file_mode='a', for RotatingFileHandler\n | maxBytes=0, for RotatingFileHandler\n | backupCount=0, for RotatingFileHandler and TimedRotatingFileHandler\n | encoding='ascii', for RotatingFileHandler and TimedRotatingFileHandler\n | delay=False, for TimedRotatingFileHandler\n | when='h', for TimedRotatingFileHandler\n | interval=1, TimedRotatingFileHandler\n | utc=False, TimedRotatingFileHandler\n | atTime=None, for TimedRotatingFileHandler\n \n \nChange History\n==============\n \n \nNext Steps\n==========\n\n 1. Acknowledgment from handler that SSH pipe was established.\n #. SSHMultiPipe to allow management of multiple pipe from a single point.", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/Acrisel/sshpipe", "keywords": "library logger multiprocessing", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "sshpipe", "package_url": "https://pypi.org/project/sshpipe/", "platform": "", "project_url": "https://pypi.org/project/sshpipe/", "project_urls": { "Homepage": "https://github.com/Acrisel/sshpipe" }, "release_url": "https://pypi.org/project/sshpipe/0.5.2/", "requires_dist": null, "requires_python": "", "summary": "sshpipe provide tools to manage ssh channel to remote hosts.", "version": "0.5.2" }, "last_serial": 3507895, "releases": { "0.5.0": [ { "comment_text": "", "digests": { "md5": "0e5d4339d049d3bc2c618f1527fcaad6", "sha256": "33c933481738d312da0eeb5fa35afb5e76220b3a0bad4421f46b2253c8a615eb" }, "downloads": -1, "filename": "sshpipe-0.5.0.tar.gz", "has_sig": true, "md5_digest": "0e5d4339d049d3bc2c618f1527fcaad6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 29523, "upload_time": "2018-01-01T15:44:06", "url": "https://files.pythonhosted.org/packages/26/07/420cc59db80e91652a9e0f688c8c258489a4d71cdb0c2c594fa366a59d37/sshpipe-0.5.0.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "e4a53241392cc0ced5d7b64a03217abd", "sha256": "57523edd21480e419278319280e94e4b32febf433b2ef2f2f969a95d4356dd60" }, "downloads": -1, "filename": "sshpipe-0.5.1.tar.gz", "has_sig": true, "md5_digest": "e4a53241392cc0ced5d7b64a03217abd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 30369, "upload_time": "2018-01-21T00:21:37", "url": "https://files.pythonhosted.org/packages/ad/5e/d29238085419487fee5523f6c8ffb8829e309db25ac97fd5a136bf26a32a/sshpipe-0.5.1.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "8176ae3c1532cdb4673b9234812ffc66", "sha256": "aaac0d90706494bf45dc3b386b89b9eb53a40648c2740a8453688a4a3edd68fb" }, "downloads": -1, "filename": "sshpipe-0.5.2.tar.gz", "has_sig": true, "md5_digest": "8176ae3c1532cdb4673b9234812ffc66", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 30407, "upload_time": "2018-01-21T00:40:58", "url": "https://files.pythonhosted.org/packages/0e/c1/b4b90a9307b69be314a6a48a8fc010c9ae95bd9e262558b47ce156fb7f89/sshpipe-0.5.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "8176ae3c1532cdb4673b9234812ffc66", "sha256": "aaac0d90706494bf45dc3b386b89b9eb53a40648c2740a8453688a4a3edd68fb" }, "downloads": -1, "filename": "sshpipe-0.5.2.tar.gz", "has_sig": true, "md5_digest": "8176ae3c1532cdb4673b9234812ffc66", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 30407, "upload_time": "2018-01-21T00:40:58", "url": "https://files.pythonhosted.org/packages/0e/c1/b4b90a9307b69be314a6a48a8fc010c9ae95bd9e262558b47ce156fb7f89/sshpipe-0.5.2.tar.gz" } ] }