{ "info": { "author": "Benoit Chesneau", "author_email": "benoitc@e-engura.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Other Environment", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: MacOS :: MacOS X", "Operating System :: POSIX", "Programming Language :: Python", "Topic :: Internet", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Utilities" ], "description": "pistil\n------\n\n\nSimple multiprocessing toolkit. This is based on the `Gunicorn `_ multiprocessing engine. \n\nThis library allows you to supervise multiple type of workers and chain\nsupervisors. Gracefull, reload, signaling between workers is handled. \n\n\nSimple Arbiter launching one worker::\n\n from pistil.arbiter import Arbiter\n from pistil.worker import Worker\n\n class MyWorker(Worker):\n\n def handle(self):\n print \"hello from worker n\u00b0%s\" % self.pid\n\n if __name__ == \"__main__\":\n conf = {}\n specs = [(MyWorker, 30, \"worker\", {}, \"test\")]\n a = Arbiter(conf, specs)\n a.run()\n\nThe same with different with the Pool arbiter. This time we send the\nsame worker on 3 os processes::\n \n from pistil.pool import PoolArbiter\n from pistil.worker import Worker\n\n class MyWorker(Worker):\n\n def handle(self):\n print \"hello from worker n\u00b0%s\" % self.pid\n\n if __name__ == \"__main__\":\n conf = {\"num_workers\": 3 }\n spec = (MyWorker, 30, \"worker\", {}, \"test\",)\n a = PoolArbiter(conf, spec)\n a.run()\n\nA common use for that pattern is a tcp server tjhat share the same\nsocket between them. For that purpose pistil provides the TcpArbiter and\nTcpSyncWorker and the GeventTcpWorker to use with gevent.\n\nPistil allows you to mix diffrent kind of workers in an arbiter::\n\n from pistil.arbiter import Arbiter\n from pistil.worker import Worker\n\n class MyWorker(Worker):\n\n def handle(self): \n print \"hello worker 1 from %s\" % self.name\n\n class MyWorker2(Worker):\n\n def handle(self):\n print \"hello worker 2 from %s\" % self.name\n\n\n if __name__ == '__main__':\n conf = {}\n\n specs = [\n (MyWorker, 30, \"worker\", {}, \"w1\"),\n (MyWorker2, 30, \"worker\", {}, \"w2\"),\n (MyWorker2, 30, \"kill\", {}, \"w3\")\n ]\n # launchh the arbiter\n arbiter = Arbiter(conf, specs)\n arbiter.run()\n\nYou can also chain arbiters:: \n\n import time\n import urllib2\n\n from pistil.arbiter import Arbiter\n from pistil.worker import Worker\n from pistil.tcp.sync_worker import TcpSyncWorker\n from pistil.tcp.arbiter import TcpArbiter\n\n from http_parser.http import HttpStream\n from http_parser.reader import SocketReader\n\n class MyTcpWorker(TcpSyncWorker):\n\n def handle(self, sock, addr):\n p = HttpStream(SocketReader(sock))\n path = p.path()\n data = \"welcome wold\"\n sock.send(\"\".join([\"HTTP/1.1 200 OK\\r\\n\", \n \"Content-Type: text/html\\r\\n\",\n \"Content-Length:\" + str(len(data)) + \"\\r\\n\",\n \"Connection: close\\r\\n\\r\\n\",\n data]))\n\n\n class UrlWorker(Worker):\n\n def handle(self):\n f = urllib2.urlopen(\"http://localhost:5000\")\n print f.read()\n\n class MyPoolArbiter(TcpArbiter):\n\n def on_init(self, conf):\n TcpArbiter.on_init(self, conf)\n # we return a spec\n return (MyTcpWorker, 30, \"worker\", {}, \"http_welcome\",)\n\n\n if __name__ == '__main__':\n conf = {\"num_workers\": 3, \"address\": (\"127.0.0.1\", 5000)}\n\n specs = [\n (MyPoolArbiter, 30, \"supervisor\", {}, \"tcp_pool\"),\n (UrlWorker, 30, \"worker\", {}, \"grabber\")\n ]\n\n arbiter = Arbiter(conf, specs)\n arbiter.run()\n\n\nThis examplelaunch a web server with 3 workers on port 5000 and another\nworker fetching the welcome page hosted by this server::\n\n\n $ python examples/multiworker2.py \n\n 2011-08-08 00:05:42 [13195] [DEBUG] Arbiter master booted on 13195\n 2011-08-08 00:05:42 [13196] [INFO] Booting grabber (worker) with pid: 13196\n ici\n 2011-08-08 00:05:42 [13197] [INFO] Booting pool (supervisor) with pid: 13197\n 2011-08-08 00:05:42 [13197] [DEBUG] Arbiter pool booted on 13197\n 2011-08-08 00:05:42 [13197] [INFO] Listening at: http://127.0.0.1:5000 (13197)\n 2011-08-08 00:05:42 [13198] [INFO] Booting worker (worker) with pid: 13198\n 2011-08-08 00:05:42 [13199] [INFO] Booting worker (worker) with pid: 13199\n welcome world\n welcome world\n\n\nMore documentation is comming. See also examples in the examples/\nfolder.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/meebo/pistil", "keywords": null, "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "pistil", "package_url": "https://pypi.org/project/pistil/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/pistil/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/meebo/pistil" }, "release_url": "https://pypi.org/project/pistil/0.2.0/", "requires_dist": null, "requires_python": null, "summary": "Multiprocessing toolkit", "version": "0.2.0" }, "last_serial": 796296, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "d3bec4985715899fc385d91aa9ae9157", "sha256": "7ba107adf5f958fd75a1a08673b84e5d59dc852232e858da5b47d11340763b89" }, "downloads": -1, "filename": "pistil-0.1.0.tar.gz", "has_sig": false, "md5_digest": "d3bec4985715899fc385d91aa9ae9157", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 50500, "upload_time": "2011-08-07T10:14:30", "url": "https://files.pythonhosted.org/packages/e4/85/57fde02353dbbac78bac0f12ee142f9b1b2716b35ca37bc6fdf29b58369c/pistil-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "0fab1474346c4b378447737b2e547d77", "sha256": "8a46d23c4762fff3a1de41f89905e9abfc14fd8bf70ae90cb64a1cd0c4cf69a3" }, "downloads": -1, "filename": "pistil-0.2.0.tar.gz", "has_sig": false, "md5_digest": "0fab1474346c4b378447737b2e547d77", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 51780, "upload_time": "2011-09-18T11:44:14", "url": "https://files.pythonhosted.org/packages/93/58/3380d4e9ad48a4dca483884b074792c6a851f5f94b78935905fc7249e61c/pistil-0.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "0fab1474346c4b378447737b2e547d77", "sha256": "8a46d23c4762fff3a1de41f89905e9abfc14fd8bf70ae90cb64a1cd0c4cf69a3" }, "downloads": -1, "filename": "pistil-0.2.0.tar.gz", "has_sig": false, "md5_digest": "0fab1474346c4b378447737b2e547d77", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 51780, "upload_time": "2011-09-18T11:44:14", "url": "https://files.pythonhosted.org/packages/93/58/3380d4e9ad48a4dca483884b074792c6a851f5f94b78935905fc7249e61c/pistil-0.2.0.tar.gz" } ] }