{ "info": { "author": "Yannick Martin - INRIA IBIS", "author_email": "janiick@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Environment :: Web Environment", "Intended Audience :: Developers", "Intended Audience :: Science/Research", "Intended Audience :: System Administrators", "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)", "Natural Language :: English", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Topic :: Scientific/Engineering", "Topic :: Software Development", "Topic :: System :: Hardware :: Symmetric Multi-processing", "Topic :: Utilities" ], "description": "pyLoadBalancer\n==============\n\nLicence\n~~~~~~~\n\npyLoadBalancer has been developped by IBIS team (INRIA Grenoble\nRh\u00f4ne-Alpes, France) and is distributed under the GPL licence.\n\nThis program is free software: you can redistribute it and/or modify it\nunder the terms of the GNU General Public License as published by the\nFree Software Foundation, either version 3 of the License, or (at your\noption) any later version. This program is distributed in the hope that\nit will be useful, but WITHOUT ANY WARRANTY; without even the implied\nwarranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details. You should have received a\ncopy of the GNU General Public License along with this program. If not,\nsee http://www.gnu.org/licenses/.\n\n--------------\n\nInstall\n~~~~~~~\n\nTo install the pyLoadBalancer package, simply run :\n\n::\n\n pip install pyLoadBalancer\n\n--------------\n\nGetting Started\n~~~~~~~~~~~~~~~\n\nThe pyLoadBalancer Package contains 5 main objects: 1. The\n``Load Balancer``, that distribute tasks 2. The ``Client``, that send\ntasks 3. The ``Worker``, that receive tasks 4. The ``Health Check``,\nthat check the status of the Load Balancer *(optional)* 5. The\n``Monitor``, a small web server that displays information on the Load\nBalancer *(optional)*\n\nThe following scheme explains the relationship:\n\n::\n\n\n +--------------+\n | | info +----------+\n | Health Check | <----------> | |\n | | | Worker 1 |\n +----------+ tasks +------^-------+ +---> | |\n | Client 1 +----+ | | +----------+\n +----------+ | |info task\n | +--------v----------+ | +----------+\n +----------+ | | | | | |\n | Client 2 +-------->+ Load Balancer | <-------> | Worker 2 |\n +----------+ | | | | | |\n | +--------^----------+ | +----------+\n +----------+ | | |\n | Client 3 +----+ | +----------+\n +----------+ +------v-------+ +---> | |\n | Monitor | | Worker 3 |\n ... +--------------+ | |\n +----------+\n ...\n\nAll the messages between objects are exchanged in json format.\n\nParameters\n~~~~~~~~~~\n\npyLoadBalancer has to be configured in order to allow the communication\nbetween the different objects (and also between different computers).\n\nThe default parameters are stored in ``parameters.json`` file.\n\nLoad Balancer parameters\n''''''''''''''''''''''''\n\n- ``\"SOCKET_TIMEOUT\" : 500`` *Timeout for communication in ms (increase\n this values if computers are not inside the same local network)*\n- ``\"LB_IP\": \"127.0.0.1\"`` *IP of the LoadBalancer computer as visible\n for other objects of the Load Balancer, 127.0.0.1 means local\n computer*\n- ``\"LB_WKPULLPORT\": 5699`` *Port for communication with the workers*\n- ``\"LB_HCREPPORT\" : 5999`` *Port for communication with the healt\n check*\n- ``\"LB_CLIENTPULLPORT\" : 5799`` *Port for communication with clients*\n- ``\"LB_QUEING_MAXPERUSER\" : 1000`` *Maximum number of queuing task for\n a given user*\n\nWorkers parameters\n''''''''''''''''''\n\n- ``\"WKHUB_IP\" : \"127.0.0.1\"`` *IP of the Worker computer as visible\n for the Load Balancer, 127.0.0.1 means local computer*\n- ``\"WKHUB_LBPORT\" : 8300`` *Port for communication with the Load\n Balancer*\n- ``\"WKHUB_HCPORT\" : 8301`` *Port for communication with the Health\n Check*\n- ``\"WKHUB_WORKERS\" : []`` *List of workers*\n\nThe list of workers is defined with the following template:\n\n::\n\n {\"nWorkers\" : Number of workers, \"minP\" : minimum prority, \"maxP\": minimum prority, \"processP\" : process prority}\n\nTo start 12 workers, with 3 different load balancer priorities, and one\nlow process priority, use:\n\n.. code:: json\n\n [ {\"nWorkers\" : 4, \"minP\" : 0, \"maxP\": 9, \"processP\" : 10},\n {\"nWorkers\" : 4, \"minP\" : 10, \"maxP\": 99, \"processP\" : 0},\n {\"nWorkers\" : 4, \"minP\" : 100, \"maxP\": 1000, \"processP\" : 0}\n ]\n\nMonitor parameters\n''''''''''''''''''\n\n- ``\"MONITOR_IP\": \"127.0.0.1\"`` *IP of the Monitor computer as visible\n for the user that wants to monitor the load balancer, 127.0.0.1 means\n local computer*\n- ``\"MONITOR_PORT\" : 9000,`` *Port used to access the monitor from a\n web browser*\n\n--------------\n\nStarting the Load Balancer core\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTo start the LoadBalancer on one computer, run the following command :\n\n.. code:: python\n\n from pyLoadBalancer import startAll\n startAll()\n # A parameter file other than the default one can be given\n # by specifying startAll('parameters.json')\n\nWhere ``parameters.json`` is a parameter file as defined in the previous\nsection.\n\nThe ``startAll()`` function starts the core of the Load Balancer: - The\n``Load Balancer``, that distribute tasks - The ``Helth Check``, that\ncheck the status of the Load Balancer *(optional)* - The ``Monitor``, a\nsmall web server that displays information on the Load Balancer\n*(optional)*\n\nThere is still no client to ask for a job and no workers to do it, the\nHealthCheck should therefore print the following warning :\n\n``HC - WARNING : LB HAS NO WORKERS.``\n\nIf you get another error message, please check check your firewall and\nantivirus settings that may block the communication between the Load\nBalancer objects.\n\nThe LoadBalancer can be monitored by typing the monitor address\n(http://localhost:9000 by default) in a web browser.\n\nNote that it is possible to execute the core objects on separate\ncomputers. See advanced uses.\n\n--------------\n\nStarting Workers\n~~~~~~~~~~~~~~~~\n\nNow the LoadBalancer core is running, workers have to be run. Workers\nare configured to connect automatically to the Load Balancer (with the\nhelp of the parameter file).\n\nWorkers can be created using the following syntax :\n\n.. code:: python\n\n from pyLoadBalancer import WorkerHub\n WKHub = WorkerHub()\n # A parameter file other than the default one can be given\n # by specifying WorkerHub('parameters.json')\n\nThe workers have to learn what tasks they will perform and how to\nperform them. This can be done by adding a task like:\n\n.. code:: python\n\n WKHub.addTask('HELLO', helloFunction)\n\nIn this exemple, when the worker receive a task named ``HELLO``, it will\ncall the ``helloFunction``.\n\nA task function should be in the following form:\n\n.. code:: python\n\n def helloFunction(**kwargs):\n print('WORKER GOT TASK', kwargs)\n result = 'Hello %s %s !' % (kwargs.get('task').get('firstName'), kwargs.get('task').get('lastName'))\n #do something with task\n return result\n\nkwargs arguments will contains the task sent by the Client. It can be\naccessed by the worker task function using \\`kwargs.get('task')\\`\\`\n\nThen, when all tasks are defined, start the worker using :\n\n.. code:: python\n\n WKHub.startWKHUB()\n\nWhen the workers are started, you should see the Load Balancer console\ndisplaying massages like ``LB - ADDING WORKER (worker_c0b28c1f)``\n\n--------------\n\nStarting Client\n~~~~~~~~~~~~~~~\n\nNow we have a pretty consistent Load Balancer with active workers. Let's\nexecute the Client side that will send task to the Load Balancer. Use\nthe following syntax:\n\n.. code:: python\n\n from pyLoadBalancer import Client\n CL = Client()\n # A parameter file other than the default one can be given\n # by specifying Client('parameters.json')\n\nThen create the task you want to send. It simply is a python dictionary\nthat corresponds the parameters of the task function to be done :\n\n.. code:: python\n\n task = {'firstName': 'John', 'lastName': 'Doe'}\n\n| The dictionnary must be JSON serializable, because sockets are using\n JSON format to communicate between each-others.\n| When the task is to be sent, send it using:\n\n.. code:: python\n\n taskid = CL.sendTask('HELLO', task, userid='username').get('taskid')\n\nThe Load Balancer will return a task unique id, that can be used to\nasynchronously retrieved the status of the task:\n\n.. code:: python\n\n taskinfo = CL.getTask(taskid)\n progress = taskinfo.get('progress')\n result = taskinfo.get('result')\n\nThe returned ``progress`` can take the following values:\n\n- ``None`` *taskid is not correct or task result in an error*\n- ``0`` *task is queing*\n- ``100`` *task is done*\n\nWhen the task is done, the ``result`` is return by the Load Balancer\n\nPlease note that the Load Balancer automatically remove tasks whose\nresult has been collected by the Client within 60 seconds.\n\nA waiting (or processing) task can also be canceled by using the\nfollowing command:\n\n.. code:: python\n\n CL.cancelTask(taskid)\n\n--------------\n\nMonitoring\n~~~~~~~~~~\n\nA monitoring interface is available to easilly monitor the Load Balancer\nstatus (queing task, statistics ...)\n\nIt can be access by a web browser at the monitor address\n(http://localhost:9000 by default).\n\n--------------\n\nAdvanced usages\n~~~~~~~~~~~~~~~\n\nUser priority\n'''''''''''''\n\nThe username sent by clients have an influence on the priority of the\nqueuing tasks.\n\nWhen a worker is available, the first task from the user that have the\nlowest number of occupied worker will be processed.\n\nTask priority\n'''''''''''''\n\nA task is sent by default with a zero priority. This priority can be\nchanged by setting the\n\n.. code:: python\n\n taskid = CL.sendTask('HELLO', task, userid='username', priority=100)\n\nThis task will only be processed by a worker that follows minP \u2265 100 and\na maxP \u2264 100.\n\nThis behavior allows to keep available workers for high priority tasks.\n\nRun on clusters of computers\n''''''''''''''''''''''''''''\n\npyLoadBalancer is designed to be run in clusters of computers.\n\nEvery objects of pyLoadBalancer can be run by a different computer (one\ncomputer can run the Load Balancer core, and few other computers can run\neach one a hub of workers).\n\nConfigure a ``parameters.json`` file in each computers in order to\nassign the correct IPs and ports, and be sure to open corresponding\nports.\n", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/ibis-inria/pyLoadBalancer/archive/0.2.tar.gz", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ibis-inria/pyLoadBalancer", "keywords": "load-balancer", "license": "GPL", "maintainer": "", "maintainer_email": "", "name": "pyLoadBalancer", "package_url": "https://pypi.org/project/pyLoadBalancer/", "platform": "", "project_url": "https://pypi.org/project/pyLoadBalancer/", "project_urls": { "Download": "https://github.com/ibis-inria/pyLoadBalancer/archive/0.2.tar.gz", "Homepage": "https://github.com/ibis-inria/pyLoadBalancer" }, "release_url": "https://pypi.org/project/pyLoadBalancer/0.2.0/", "requires_dist": null, "requires_python": "", "summary": "A load-balancer package for Python", "version": "0.2.0" }, "last_serial": 2848641, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "aa7f99f2fc9ded95c254f8530614f875", "sha256": "a065ab86a1765b12b632f7275f28423aefa171e84f7a73ad3b55359bccc73b27" }, "downloads": -1, "filename": "pyLoadBalancer-0.1.0.tar.gz", "has_sig": false, "md5_digest": "aa7f99f2fc9ded95c254f8530614f875", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 358186, "upload_time": "2017-05-03T09:38:59", "url": "https://files.pythonhosted.org/packages/5e/7c/e1787d6572a5d7564ea14346cec2eff356fef0787fcf01fc731b6dc53d79/pyLoadBalancer-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "4c7ddc2b4ab015d73dd490c2b01c2b65", "sha256": "20164d6297bc599b3775e7a2086a4360cb616f91cbd094fcfcb77209261a9400" }, "downloads": -1, "filename": "pyLoadBalancer-0.2.0.tar.gz", "has_sig": false, "md5_digest": "4c7ddc2b4ab015d73dd490c2b01c2b65", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 358187, "upload_time": "2017-05-03T09:47:41", "url": "https://files.pythonhosted.org/packages/58/e3/d19db4b414b8daa4911e356091180abc107d2a8fd0a45a1bf22d8abd30cf/pyLoadBalancer-0.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "4c7ddc2b4ab015d73dd490c2b01c2b65", "sha256": "20164d6297bc599b3775e7a2086a4360cb616f91cbd094fcfcb77209261a9400" }, "downloads": -1, "filename": "pyLoadBalancer-0.2.0.tar.gz", "has_sig": false, "md5_digest": "4c7ddc2b4ab015d73dd490c2b01c2b65", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 358187, "upload_time": "2017-05-03T09:47:41", "url": "https://files.pythonhosted.org/packages/58/e3/d19db4b414b8daa4911e356091180abc107d2a8fd0a45a1bf22d8abd30cf/pyLoadBalancer-0.2.0.tar.gz" } ] }