{
    "info": {
        "author": "Sergei Beilin",
        "author_email": "saabeilin@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "License :: OSI Approved :: Apache Software License",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "# kafkian \n\n[![Build Status](https://travis-ci.org/saabeilin/kafkian.svg?branch=master)](https://travis-ci.org/saabeilin/kafkian)\n[![codecov](https://codecov.io/gh/saabeilin/kafkian/branch/master/graph/badge.svg)](https://codecov.io/gh/saabeilin/kafkian) \n[![PyPI](https://img.shields.io/pypi/v/kafkian.svg)](https://pypi.org/project/kafkian/)\n\n*kafkian* is a opinionated a high-level consumer and producer on top of \n[confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python)/librdkafka \nand partially inspired by [confluent_kafka_helpers](https://github.com/fyndiq/confluent_kafka_helpers). \nIt is intended for use primarily in CQRS/EventSourced systems when usage is mostly\nlimited to producing and consuming encoded messages.\n\n*kafkian* partially mimics Kafka JAVA API, partially is more pythonic, partially just like the maintainer likes it.\n\nInstead of configuring all the things via properties, most of the things \nare planned to be configured explicitely and, wneh possible, via dependency\ninjection for easier testing. The configuration dictionaries for both producer\nand consumer are passed-through directly to underlying confluent producer and \nconsumer, hidden behind a facade.\n\nThe library provides a base serializer and deserializer classes, as well as \ntheir specialized Avro subclasses, `AvroSerializer` and `AvroDeserializer`. \nThis allows having, say, a plain string key and and avro-encoded message, \nor vice versa. Quite often an avro-encoded string is used as a key, for \nthis purpose we provide `AvroStringKeySerializer`.\n\nUnlike the Confluent library, we support supplying the specific Avro schema\ntogether with the message, just like the Kafka JAVA API. Schemas could be\nautomatically registered with schema registry, also we provide three\n`SubjectNameStrategy`, again compatible with Kafka JAVA API.\n\n## Usage\n### Producing messages\n\n#### 1. Initialize the producer\n\n```python\nfrom kafkian import Producer\nfrom kafkian.serde.serialization import AvroSerializer, AvroStringKeySerializer, SubjectNameStrategy\n\nproducer = Producer(\n    {\n        'bootstrap.servers': config.KAFKA_BOOTSTRAP_SERVERS,\n    },\n    key_serializer=AvroStringKeySerializer(schema_registry_url=config.SCHEMA_REGISTRY_URL),\n    value_serializer=AvroSerializer(schema_registry_url=config.SCHEMA_REGISTRY_URL,\n                                    subject_name_strategy=SubjectNameStrategy.RecordNameStrategy)\n)\n\n```\n\n#### 2. Define your message schema(s)\n\n```python\nfrom confluent_kafka import avro\nfrom kafkian.serde.avroserdebase import AvroRecord\n\n\nvalue_schema_str = \"\"\"\n{\n   \"namespace\": \"auth.users\",\n   \"name\": \"UserCreated\",\n   \"type\": \"record\",\n   \"fields\" : [\n     {\n       \"name\" : \"uuid\",\n       \"type\" : \"string\"\n     },     \n     {\n       \"name\" : \"name\",\n       \"type\" : \"string\"\n     },\n     {\n        \"name\": \"timestamp\",\n        \"type\": {\n            \"type\": \"long\",\n            \"logicalType\": \"timestamp-millis\"\n        }\n     }\n   ]\n}\n\"\"\"\n\n\nclass UserCreated(AvroRecord):\n    _schema = avro.loads(value_schema_str)\n\n```\n\n#### 3. Produce the message\n\n```python\n\nproducer.produce(\n    \"auth.users.events\",\n    user.uuid,\n    UserCreated({\n        \"uuid\": user.uuid,\n        \"name\": user.name,\n        \"timestamp\": int(user.timestamp.timestamp() * 1000)\n    }),\n    sync=True\n)\n```\n\n### Consuming messages\n\n#### 1. Initialize the consumer\n\n```python\nCONSUMER_CONFIG = {\n    'bootstrap.servers': config.KAFKA_BOOTSTRAP_SERVERS,\n    'default.topic.config': {\n        'auto.offset.reset': 'latest',\n    },\n    'group.id': 'notifications'\n}\n\nconsumer = Consumer(\n    CONSUMER_CONFIG,\n    topics=[\"auth.users.events\"],\n    key_deserializer=AvroDeserializer(schema_registry_url=config.SCHEMA_REGISTRY_URL),\n    value_deserializer=AvroDeserializer(schema_registry_url=config.SCHEMA_REGISTRY_URL),\n)\n```\n\n#### 2. Consume the messages via the generator\n\n```python\n\nfor message in consumer:\n    handle_message(message)\n    consumer.commit()\n```\n\nHere, `message` is an instance of `Message` class, that wraps the original \nmessage exposed by the confluent-kafka-python, and you can access \nthe decoded key and value via `.key` and `.value` properties respectively.\n\nNotice that deserialization will happen on first access of the properties,\nso you can properly handle deserialization errors (log it, send to DLQ, etc)\n\nBoth key and value are wrapped in a dynamically-generated class,\nthat has the full name same as the corresponding Avro schema full name.\nIn the example above, the value would have class named `auth.users.UserCreated`.\n\nAvro schemas for the consumed message key and value are accessible via `.schema` property.\n\nIn addition, `topic`, `partition`, `offset`, `timestamp`, `headers` properties\nare available.\n\n## Contributing\nThis library is, as stated, quite opinionated, however, I'm open to suggestions.\nWrite your questions and suggestions as issues here on github!\n\n#### Running tests\nBoth unit and system tests are provided. \n\nTo run unit-tests, install the requirements and just run \n```bash\npy.test tests/unit/\n``` \n\nTo run system tests, a Kafka cluster together with a schema registry is \nrequired. A Docker compose file is provided, just run \n```bash\ndocker-compose up\n```\nand once the cluster is up and running, run system tests via \n```bash\npy.test tests/system/\n```\n\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/saabeilin/kafkian",
        "keywords": "kafka",
        "license": "Apache 2.0",
        "maintainer": "",
        "maintainer_email": "",
        "name": "kafkian",
        "package_url": "https://pypi.org/project/kafkian/",
        "platform": "",
        "project_url": "https://pypi.org/project/kafkian/",
        "project_urls": {
            "Homepage": "https://github.com/saabeilin/kafkian"
        },
        "release_url": "https://pypi.org/project/kafkian/0.13.0/",
        "requires_dist": [
            "confluent-kafka (>=1.0.0)",
            "fastavro (>=0.18.0)",
            "avro-python3 (>=1.8.2)"
        ],
        "requires_python": "",
        "summary": "Opinionated Kafka Python client on top of Confluent python library",
        "version": "0.13.0"
    },
    "last_serial": 5864664,
    "releases": {
        "0.10.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2ade23be0ea66d8629a76797b623c6d6",
                    "sha256": "4740ac474f290ddbd88aee9ecaee2be8b73ca1315d0acdc67cf22f48107c0b7a"
                },
                "downloads": -1,
                "filename": "kafkian-0.10.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2ade23be0ea66d8629a76797b623c6d6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 17998,
                "upload_time": "2019-01-05T22:20:08",
                "url": "https://files.pythonhosted.org/packages/9f/e4/93d2d1aa85a94be5b3cd2e40fb108083c3504c08afe4fc5572c873b3a328/kafkian-0.10.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "05e893f62fb4eeb37990a9b84b6118ab",
                    "sha256": "275e7c818945d8f53f3d1bba740ba589a22a565cf94816afb0e4dc341923fc02"
                },
                "downloads": -1,
                "filename": "kafkian-0.10.0.tar.gz",
                "has_sig": false,
                "md5_digest": "05e893f62fb4eeb37990a9b84b6118ab",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10857,
                "upload_time": "2019-01-05T22:20:10",
                "url": "https://files.pythonhosted.org/packages/6e/e4/4a7b170499cd5b69f439a79739b75e37f11b0d4919cd703a1c225db30186/kafkian-0.10.0.tar.gz"
            }
        ],
        "0.13.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b5cd0a01d560892a182fb2f4fe6751c4",
                    "sha256": "0e80b2287d8da04318b829d962423e08898377b0dc9690d0133afb62f9037bd2"
                },
                "downloads": -1,
                "filename": "kafkian-0.13.0.linux-x86_64.tar.gz",
                "has_sig": false,
                "md5_digest": "b5cd0a01d560892a182fb2f4fe6751c4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 28851,
                "upload_time": "2019-09-20T23:55:50",
                "url": "https://files.pythonhosted.org/packages/9e/05/0357400cbaf0b7f4cdb9c8d282e2c08e62c1a5dbc84bdc36e439f04cadfd/kafkian-0.13.0.linux-x86_64.tar.gz"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "5773e8fc31514e5145345f1cadc0afa0",
                    "sha256": "0fa09e60f95c1bebe7095914a2ff0873adc857c3b5649f17607cce04074fae93"
                },
                "downloads": -1,
                "filename": "kafkian-0.13.0-py3.7.egg",
                "has_sig": false,
                "md5_digest": "5773e8fc31514e5145345f1cadc0afa0",
                "packagetype": "bdist_egg",
                "python_version": "3.7",
                "requires_python": null,
                "size": 51552,
                "upload_time": "2019-09-20T23:55:52",
                "url": "https://files.pythonhosted.org/packages/ae/37/c697c8dd18e1ec32e8d4080788ca8ff876e83555ce2afa9cc874a82a2e58/kafkian-0.13.0-py3.7.egg"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8f3d4040819d48b631a673ad249d5492",
                    "sha256": "6469c088208f5d103fc7950ac701003f75f9e1b432270509ffbc7cfa67a2afc1"
                },
                "downloads": -1,
                "filename": "kafkian-0.13.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "8f3d4040819d48b631a673ad249d5492",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 26277,
                "upload_time": "2019-09-20T23:55:48",
                "url": "https://files.pythonhosted.org/packages/08/5f/dd80d5eca659601b13e1a0b241b2d787e0cb3ec57b6391f55ab2f2c64a00/kafkian-0.13.0-py3-none-any.whl"
            }
        ],
        "0.7.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dcd3e63947355aef9b2a6bd17bcb685f",
                    "sha256": "b87c31c98ee5438573d5cc74343069d2cc889c17e8643f8257f1234cd0c93cd8"
                },
                "downloads": -1,
                "filename": "kafkian-0.7.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "dcd3e63947355aef9b2a6bd17bcb685f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 10134,
                "upload_time": "2018-09-23T10:31:18",
                "url": "https://files.pythonhosted.org/packages/9e/f3/a8b94b9df0bed1f6bee0f8917cef2b3676515dc6e0876e7199fa505610a3/kafkian-0.7.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3333e09797a03da6819ecc7e5b1f7ba8",
                    "sha256": "8904c02f86e860ac98210f1a3a356bf32d221bf37624d9e22b8579f1709d94e2"
                },
                "downloads": -1,
                "filename": "kafkian-0.7.0.tar.gz",
                "has_sig": false,
                "md5_digest": "3333e09797a03da6819ecc7e5b1f7ba8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 5828,
                "upload_time": "2018-09-23T10:31:20",
                "url": "https://files.pythonhosted.org/packages/ad/7a/8d2e7329688837a0c17057e2799cc9031a4d39ab693f4e56aec8696fd329/kafkian-0.7.0.tar.gz"
            }
        ],
        "0.7.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad5509dca16710ba1233f7690a3ce8e7",
                    "sha256": "c633fe94ca84aa9d76075e8980d6955472ad8d7514381ce6ad516f7c5d84d142"
                },
                "downloads": -1,
                "filename": "kafkian-0.7.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ad5509dca16710ba1233f7690a3ce8e7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12163,
                "upload_time": "2018-09-24T18:13:44",
                "url": "https://files.pythonhosted.org/packages/4c/50/10d89ca437261a5bf2c820b83c348b1ee7cec0330cfa762ee2bc55208ba6/kafkian-0.7.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a1ea18eaafedd408242e588046de60d",
                    "sha256": "bcfd4b6317b403fe85fdcacf5163c3040a398baad39d2b49a40262b9565996e0"
                },
                "downloads": -1,
                "filename": "kafkian-0.7.1.tar.gz",
                "has_sig": false,
                "md5_digest": "7a1ea18eaafedd408242e588046de60d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8229,
                "upload_time": "2018-09-24T18:13:46",
                "url": "https://files.pythonhosted.org/packages/bf/47/1401c133ba28d9c0f800e34dcb81e50345d6518fdf1769be5d4e60cfbf16/kafkian-0.7.1.tar.gz"
            }
        ],
        "0.7.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "04f15959acbf610057f1025e5d0f89ff",
                    "sha256": "daed38b9627e5650bba34ff04622a0211ff2e7e0de391aa57ff26293ba30e15a"
                },
                "downloads": -1,
                "filename": "kafkian-0.7.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "04f15959acbf610057f1025e5d0f89ff",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12737,
                "upload_time": "2018-09-24T20:05:47",
                "url": "https://files.pythonhosted.org/packages/55/ca/c00fea087a435339381d9107db82985efd98ed0bd54174d90b76a0c8814b/kafkian-0.7.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "58ebd9ff045818d9aa19845f3b976965",
                    "sha256": "93f379cb693602b9ef1c8eb56748f90ef23e71789db73184166e9200b79efafd"
                },
                "downloads": -1,
                "filename": "kafkian-0.7.2.tar.gz",
                "has_sig": false,
                "md5_digest": "58ebd9ff045818d9aa19845f3b976965",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8420,
                "upload_time": "2018-09-24T20:05:49",
                "url": "https://files.pythonhosted.org/packages/4d/04/1a40862fc3b503bc6bcff9061f2476ed1fc11dfb2056e35de04f94f05853/kafkian-0.7.2.tar.gz"
            }
        ],
        "0.8.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e3ca0e470770355ccf7ddf63715adbf6",
                    "sha256": "af4921f0a7d86b89e8aa94b17e014b01497c84986dbe74bd37de74e98be867cd"
                },
                "downloads": -1,
                "filename": "kafkian-0.8.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e3ca0e470770355ccf7ddf63715adbf6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 14398,
                "upload_time": "2018-11-21T21:22:57",
                "url": "https://files.pythonhosted.org/packages/a9/08/5f90836510e2a4bf0a506c9cfca232c180f15e6542e6840c0be3fbb2fe11/kafkian-0.8.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8c724ab9237bcd4284be1d0e07fe0264",
                    "sha256": "c5f4d7ba8ad400fbbdf91df748ec5027e616e00bbd2c98143ffe554c0feb3dc8"
                },
                "downloads": -1,
                "filename": "kafkian-0.8.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8c724ab9237bcd4284be1d0e07fe0264",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9490,
                "upload_time": "2018-11-21T21:22:59",
                "url": "https://files.pythonhosted.org/packages/f1/6f/5f2d949ffd9b69b11fbae4b701f31d6928e4962c4e15db7561e2f1873c0e/kafkian-0.8.0.tar.gz"
            }
        ],
        "0.9.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "75310868a2b3905167b00dcc4e9d8c98",
                    "sha256": "83689936050d697b3ccafa9d719e5a4847a1a636bc7d347cf46941244e81eb16"
                },
                "downloads": -1,
                "filename": "kafkian-0.9.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "75310868a2b3905167b00dcc4e9d8c98",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 16432,
                "upload_time": "2018-12-06T21:17:18",
                "url": "https://files.pythonhosted.org/packages/53/f5/99110402badb0b5e2892493a1f2eedaa0209666c943c8f74abe712869e03/kafkian-0.9.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "d7f17dc948ed7cadd8f5b39a9e5c32e6",
                    "sha256": "ce9fe02e7816fb2fb16bfdaf46fdcf29d964f20687c227f642fb85a477fe65ad"
                },
                "downloads": -1,
                "filename": "kafkian-0.9.0.tar.gz",
                "has_sig": false,
                "md5_digest": "d7f17dc948ed7cadd8f5b39a9e5c32e6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10098,
                "upload_time": "2018-12-06T21:17:21",
                "url": "https://files.pythonhosted.org/packages/3c/54/a401b946748e54596b3942f47500043d97d7dc1d3f5e225c3aa25fd51ae4/kafkian-0.9.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "b5cd0a01d560892a182fb2f4fe6751c4",
                "sha256": "0e80b2287d8da04318b829d962423e08898377b0dc9690d0133afb62f9037bd2"
            },
            "downloads": -1,
            "filename": "kafkian-0.13.0.linux-x86_64.tar.gz",
            "has_sig": false,
            "md5_digest": "b5cd0a01d560892a182fb2f4fe6751c4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 28851,
            "upload_time": "2019-09-20T23:55:50",
            "url": "https://files.pythonhosted.org/packages/9e/05/0357400cbaf0b7f4cdb9c8d282e2c08e62c1a5dbc84bdc36e439f04cadfd/kafkian-0.13.0.linux-x86_64.tar.gz"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "5773e8fc31514e5145345f1cadc0afa0",
                "sha256": "0fa09e60f95c1bebe7095914a2ff0873adc857c3b5649f17607cce04074fae93"
            },
            "downloads": -1,
            "filename": "kafkian-0.13.0-py3.7.egg",
            "has_sig": false,
            "md5_digest": "5773e8fc31514e5145345f1cadc0afa0",
            "packagetype": "bdist_egg",
            "python_version": "3.7",
            "requires_python": null,
            "size": 51552,
            "upload_time": "2019-09-20T23:55:52",
            "url": "https://files.pythonhosted.org/packages/ae/37/c697c8dd18e1ec32e8d4080788ca8ff876e83555ce2afa9cc874a82a2e58/kafkian-0.13.0-py3.7.egg"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "8f3d4040819d48b631a673ad249d5492",
                "sha256": "6469c088208f5d103fc7950ac701003f75f9e1b432270509ffbc7cfa67a2afc1"
            },
            "downloads": -1,
            "filename": "kafkian-0.13.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "8f3d4040819d48b631a673ad249d5492",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 26277,
            "upload_time": "2019-09-20T23:55:48",
            "url": "https://files.pythonhosted.org/packages/08/5f/dd80d5eca659601b13e1a0b241b2d787e0cb3ec57b6391f55ab2f2c64a00/kafkian-0.13.0-py3-none-any.whl"
        }
    ]
}