{ "info": { "author": "Patrick Lucas", "author_email": "plucas@yelp.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Topic :: Software Development :: Libraries", "Topic :: System :: Distributed Computing" ], "description": "Pyleus\n======\n\nPyleus is a Python 2.6+ framework for developing and launching `Apache Storm`_ topologies.\n\nPlease visit our `documentation`_.\n\n=============== ================\n master develop\n=============== ================\n|master-status| |develop-status|\n=============== ================\n\n.. |master-status| image:: https://travis-ci.org/Yelp/pyleus.svg?branch=master\n :target: https://travis-ci.org/Yelp/pyleus\n\n.. |develop-status| image:: https://travis-ci.org/Yelp/pyleus.svg?branch=develop\n :target: https://travis-ci.org/Yelp/pyleus\n\nAbout\n-----\n\nPyleus is a framework for building Apache Storm topologies in idiomatic Python.\n\nWith Pyleus you can:\n\n* define a topology with a simple YAML file\n\n* have dependency management with a ``requirements.txt`` file\n\n* run faster thanks to Pyleus\u2019 `MessagePack`_ based serializer\n\n* pass options to your components directly from the YAML file\n\n* use the Kafka spout built into Storm with only a YAML change\n\nInstall\n-------\n\nFrom PyPI:\n\n.. code-block:: shell\n\n $ pip install pyleus\n\n**Note:**\n\nYou do **NOT** need to install pyleus on your Storm cluster. That\u2019s cool, isn't it?\n\nHowever, if you are going to use ``system_site_packages: true`` in your config file, you should be aware that the environment of your Storm nodes needs to match the one on the machine used for building the topology. This means you actually **have to install** pyleus on your Storm cluster in this case.\n\nTry it out!\n-----------\n\n.. code-block:: shell\n\n $ git clone https://github.com/Yelp/pyleus.git\n $ pyleus build pyleus/examples/exclamation_topology/pyleus_topology.yaml\n $ pyleus local exclamation_topology.jar\n\nOr, submit to a Storm cluster with:\n\n.. code-block:: shell\n\n $ pyleus submit -n NIMBUS_HOST exclamation_topology.jar\n\nThe `examples`_ directory contains several annotated Pyleus topologies that try to cover as many Pyleus features as possible.\n\nPyleus command line interface\n-----------------------------\n\n* Build a topology:\n\n .. code-block:: shell\n\n $ pyleus build /path/to/pyleus_topology.yaml\n\n* Run a topology locally:\n\n .. code-block:: shell\n\n $ pyleus local /path/to/topology.jar\n\n* Submit a topology to a Storm cluster:\n\n .. code-block:: shell\n\n $ pyleus submit [-n NIMBUS_HOST] /path/to/topology.jar\n\n* List all topologies running on a Storm cluster:\n\n .. code-block:: shell\n\n $ pyleus list [-n NIMBUS_HOST]\n\n* Kill a topology running on a Storm cluster:\n\n .. code-block:: shell\n\n $ pyleus kill [-n NIMBUS_HOST] TOPOLOGY_NAME\n\nTry ``pyleus -h`` for a list of all the available commands or ``pyleus CMD -h`` for any command-specific help.\n\nWrite your first topology\n-------------------------\n\nPlease refer to the `documentation`_ for a more detailed tutorial.\n\nOrganize your files\n^^^^^^^^^^^^^^^^^^^\n\nThis is an example of the directory tree of a simple topology:\n\n.. code-block:: none\n\n my_first_topology/\n |-- my_first_topology/\n | |-- __init__.py\n | |-- dummy_bolt.py\n | |-- dummy_spout.py\n |-- pyleus_topology.yaml\n |-- requirements.txt\n\nDefine the topology layout\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nA simple ``pyleus_topology.yaml`` should look like the following:\n\n.. code-block:: yaml\n\n name: my_first_topology\n\n topology:\n\n - spout:\n name: my-first-spout\n module: my_first_topology.dummy_spout\n \n - bolt:\n name: my-first-bolt\n module: my_first_topology.dummy_bolt\n groupings:\n - shuffle_grouping: my-first-spout\n\nThis defines a topology where a single bolt subscribes to the output stream of a single spout. As simple as it is.\n\nWrite your first spout\n^^^^^^^^^^^^^^^^^^^^^^\n\nThis is the code implementing ``dummy_spout.py``:\n\n.. code-block:: python\n\n from pyleus.storm import Spout\n\n\n class DummySpout(Spout):\n\n OUTPUT_FIELDS = ['sentence', 'name']\n\n def next_tuple(self):\n self.emit((\"This is a sentence.\", \"spout\",))\n\n\n if __name__ == '__main__':\n DummySpout().run()\n\nWrite your first bolt\n^^^^^^^^^^^^^^^^^^^^^\n\nLet's now look at ``dummy_bolt.py``:\n\n.. code-block:: python\n\n from pyleus.storm import SimpleBolt\n\n\n class DummyBolt(SimpleBolt):\n\n OUTPUT_FIELDS = ['sentence']\n\n def process_tuple(self, tup):\n sentence, name = tup.values\n new_sentence = \"{0} says, \\\"{1}\\\"\".format(name, sentence)\n self.emit((new_sentence,), anchors=[tup])\n\n\n if __name__ == '__main__':\n DummyBolt().run()\n\nRun your topology\n^^^^^^^^^^^^^^^^^\n\nRun the topology on your local machine for debugging:\n\n.. code-block:: shell\n\n pyleus build my_first_topology/pyleus_topology.yaml\n pyleus local --debug my_first_topology.jar\n\nWhen you are done, hit ``C-C``.\n\nConfiguration File\n^^^^^^^^^^^^^^^^^^\n\nYou can set default values for many configuration options by placing a ``.pyleus.conf`` file in your home directory:\n\n.. code-block:: none\n\n [storm]\n nimbus_host: 10.11.12.13\n jvm_opts: -Djava.io.tmpdir=/home/myuser/tmp\n\n [build]\n pypi_index_url: http://pypi.ninjacorp.com/simple/\n\nReference\n---------\n* `Apache Storm Documentation`_\n\nLicense\n-------\n\nPyleus is licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0\n\n\n.. _Apache Storm: https://storm.apache.org/\n.. _Apache Storm Documentation: https://storm.apache.org/documentation/Home.html\n.. _MessagePack: http://msgpack.org/\n.. _documentation: http://pyleus.org/\n.. _examples: https://github.com/Yelp/pyleus/tree/master/examples", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://pyleus.org", "keywords": null, "license": "UNKNOWN", "maintainer": null, "maintainer_email": null, "name": "pyleus", "package_url": "https://pypi.org/project/pyleus/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/pyleus/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://pyleus.org" }, "release_url": "https://pypi.org/project/pyleus/0.3.0/", "requires_dist": null, "requires_python": null, "summary": "Standard library and deployment tools for using Python with Storm", "version": "0.3.0" }, "last_serial": 1580386, "releases": { "0.2": [ { "comment_text": "", "digests": { "md5": "4115dbdca9fed8b26b7e84a53ef58009", "sha256": "457892034f74155379f403eef98d2ef9df74bd8683c616e4d6051879f8e52da0" }, "downloads": -1, "filename": "pyleus-0.2.tar.gz", "has_sig": false, "md5_digest": "4115dbdca9fed8b26b7e84a53ef58009", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17546600, "upload_time": "2014-10-15T18:29:35", "url": "https://files.pythonhosted.org/packages/1a/eb/1877923be55baaa0c6eb71529a0f0b098adddd1f317bacd94bef9b23ecf6/pyleus-0.2.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "575e83b0808485f7fcc84cd19b7eced5", "sha256": "ac5cec99a7d42fd9060dc4f2f41bb2eaa03e0ecd01429ee4eff46bceb97b1de6" }, "downloads": -1, "filename": "pyleus-0.2.1.tar.gz", "has_sig": false, "md5_digest": "575e83b0808485f7fcc84cd19b7eced5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17544732, "upload_time": "2014-10-18T01:11:57", "url": "https://files.pythonhosted.org/packages/7f/eb/e818dea7407ab8418442e6d5409ee3faa605b57d6e62239513ee6a792826/pyleus-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "a8635fa78c30166444e2c14949d812b3", "sha256": "bd44a9dc8662a5b84f55c459742b293fc583c79cfbee7da7111bf7a65a49d237" }, "downloads": -1, "filename": "pyleus-0.2.2.tar.gz", "has_sig": false, "md5_digest": "a8635fa78c30166444e2c14949d812b3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17544975, "upload_time": "2014-10-24T21:44:07", "url": "https://files.pythonhosted.org/packages/72/a4/11b0a1f15eb736febac8ad9d8d781b24fc2e2a6bf34f2ba2a9d09fa86883/pyleus-0.2.2.tar.gz" } ], "0.2.3": [ { "comment_text": "", "digests": { "md5": "6e213c5cec37b3c31c776e208407a0e9", "sha256": "51db885a36df15375c3635f68dcf076132a7416d7fc1e7c68081753943a1fc4b" }, "downloads": -1, "filename": "pyleus-0.2.3.tar.gz", "has_sig": false, "md5_digest": "6e213c5cec37b3c31c776e208407a0e9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17546153, "upload_time": "2014-12-08T19:46:34", "url": "https://files.pythonhosted.org/packages/f1/dd/9cf46dd10b04650d3744569624bfe588e85a2b297b0e7c2d820e0392584c/pyleus-0.2.3.tar.gz" } ], "0.2.4": [ { "comment_text": "", "digests": { "md5": "b1634deaa4db35fc5489e6a9099643dd", "sha256": "9125a02030da7780267765d095b1d727b57b5ac7c10255bdd76a76aeda46e109" }, "downloads": -1, "filename": "pyleus-0.2.4.tar.gz", "has_sig": false, "md5_digest": "b1634deaa4db35fc5489e6a9099643dd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17546271, "upload_time": "2014-12-10T22:57:13", "url": "https://files.pythonhosted.org/packages/1b/ea/f8c2fdf68d871c89780cf155a0a2a085bdc14f76d94621ca2eb0bca74fcd/pyleus-0.2.4.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "68ccfb6839e7f11154fc9b25e6c4f94f", "sha256": "16542c5eb4cf4174ee46148d6cf89b7cff911f79de9ccc9c32bc0e07e4c30794" }, "downloads": -1, "filename": "pyleus-0.3.0.tar.gz", "has_sig": false, "md5_digest": "68ccfb6839e7f11154fc9b25e6c4f94f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17978016, "upload_time": "2015-06-05T19:34:58", "url": "https://files.pythonhosted.org/packages/39/27/516fb9ee0ebb3e607620275d44af6ebf79fab3482b09c8605530a98819c8/pyleus-0.3.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "68ccfb6839e7f11154fc9b25e6c4f94f", "sha256": "16542c5eb4cf4174ee46148d6cf89b7cff911f79de9ccc9c32bc0e07e4c30794" }, "downloads": -1, "filename": "pyleus-0.3.0.tar.gz", "has_sig": false, "md5_digest": "68ccfb6839e7f11154fc9b25e6c4f94f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17978016, "upload_time": "2015-06-05T19:34:58", "url": "https://files.pythonhosted.org/packages/39/27/516fb9ee0ebb3e607620275d44af6ebf79fab3482b09c8605530a98819c8/pyleus-0.3.0.tar.gz" } ] }