{ "info": { "author": "System73 Engineering Team", "author_email": "opensource@system73.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 3.6" ], "description": "# tamarco-kafka\n\n[![Build Status](https://travis-ci.com/System73/tamarco-kafka.svg?branch=master)](https://travis-ci.com/System73/tamarco-kafka)\n[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=System73_tamarco-kafka&metric=coverage)](https://sonarcloud.io/dashboard?id=System73_tamarco-kafka)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=System73_tamarco-kafka&metric=alert_status)](https://sonarcloud.io/dashboard?id=System73_tamarco-kafka)\n\nKafka resource for Tamarco microservice framework. It runs a confluent-kafka client in a thread.\n\nThis repository is a plugin for Tamarco, for more information go to [Tamarco main repository](https://github.com/System73/tamarco).\n\n## Settings\n\nThis resource depends on the following configuration:\n\n```yaml\n system:\n resources:\n kafka:\n bootstrap_servers: kafka:9092\n```\n\nThe bootstrap servers are the address of the members of a kafka cluster separated by coma.\n\n## Input and outputs\n\nThe inputs and outputs need to be declared in the resource.\n\n### Input\n\nThe input can be used with two different patterns, as decorator and as async stream.\n\nThis resource only supports balanced consumer groups with auto commit.\n\n#### Async stream\n\nThis usage case uses the input as asynchronous iterator to consume the metric stream.\n\n```python3\nclass MyMicroservice(Microservice):\n name = \"input_example\"\n\n metrics_input = KafkaInput(topic='metrics', codec=JsonCodec)\n kafka = KafkaResource(inputs=[metrics_input])\n\n @task\n async def metrics_consumer(self):\n async for metric in self.metrics_input:\n self.logger.info(f'Consumed message from metrics topic: {metric}')\n```\n\n### Decorator\n\nThis usage case declares a function as handler of the messages, and the resource is going to open automatically a\ncoroutine to consume each message. \n\n```python3\nclass MyMicroservice(Microservice):\n name = \"input_example\"\n\n kafka = KafkaResource(inputs=[metrics_input])\n\n @KafkaInput(resource=kafka, topic='metrics', codec=JsonCodec)\n async def metrics_handler(self, message):\n self.logger.info(f'Consumed message from metrics topic: {message}')\n```\n\n## Output\n\nIt is a Kafka producer very simple to use.\n\n```python3\nclass MyMicroservice(Microservice):\n name = \"output_example\"\n metrics_output = KafkaOutput(topic='metrics', codec=JsonCodec)\n kafka = KafkaResource(outputs=[metrics_output])\n\n @task_timer(interval=1000, autostart=True)\n async def metrics_producer(self):\n metrics_message = {'metrics': {'cat': 'MEOW'}}\n await self.metrics_output.push(metrics_message)\n self.logger.info(f'Produced message {metrics_message} to metrics topic')\n```\n\n## How to run the examples\n\nTo run them you just need to launch the docker-compose, install the requirements and run it.\n\n```python3\npip install -r examples/requirements.txt\ndocker-compose up -d\npython examples/microservice_stream_input.py\n```", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/System73/tamarco-kafka", "keywords": "tamarco,kafka,microservices", "license": "", "maintainer": "", "maintainer_email": "", "name": "tamarco-kafka", "package_url": "https://pypi.org/project/tamarco-kafka/", "platform": "", "project_url": "https://pypi.org/project/tamarco-kafka/", "project_urls": { "Homepage": "https://github.com/System73/tamarco-kafka" }, "release_url": "https://pypi.org/project/tamarco-kafka/0.1.0/", "requires_dist": null, "requires_python": "", "summary": "Kafka resource for Tamarco microservice framework.", "version": "0.1.0" }, "last_serial": 5890409, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "9ecc49d4d1abc30bef0f5a60e9eaafa4", "sha256": "4f8bdef609aeff86dfbb47c3ebb8d26ed14744b622f466ead0cdf8c42e0c26db" }, "downloads": -1, "filename": "tamarco-kafka-0.1.0.tar.gz", "has_sig": false, "md5_digest": "9ecc49d4d1abc30bef0f5a60e9eaafa4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15458, "upload_time": "2019-09-26T12:36:36", "url": "https://files.pythonhosted.org/packages/0a/86/3dd8fdca104969e84cd4427a16ae9c8daf4bc4ef1a58e037b4c1bc1d2aa7/tamarco-kafka-0.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "9ecc49d4d1abc30bef0f5a60e9eaafa4", "sha256": "4f8bdef609aeff86dfbb47c3ebb8d26ed14744b622f466ead0cdf8c42e0c26db" }, "downloads": -1, "filename": "tamarco-kafka-0.1.0.tar.gz", "has_sig": false, "md5_digest": "9ecc49d4d1abc30bef0f5a60e9eaafa4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15458, "upload_time": "2019-09-26T12:36:36", "url": "https://files.pythonhosted.org/packages/0a/86/3dd8fdca104969e84cd4427a16ae9c8daf4bc4ef1a58e037b4c1bc1d2aa7/tamarco-kafka-0.1.0.tar.gz" } ] }