{ "info": { "author": "Leonardo de Campos Almeida", "author_email": "leocalm@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "[![Build Status](https://travis-ci.org/leocalm/avro_validator.svg?branch=master)](https://travis-ci.org/leocalm/avro_validator)\n[![Coverage Status](https://coveralls.io/repos/github/leocalm/avro_validator/badge.svg?branch=master)](https://coveralls.io/github/leocalm/avro_validator?branch=master)\n[![Documentation Status](https://readthedocs.org/projects/avro-validator/badge/?version=latest)](https://avro-validator.readthedocs.io/en/latest/?badge=latest)\n[![PyPI version](https://badge.fury.io/py/avro-validator.svg)](https://badge.fury.io/py/avro-validator)\n\n# Avro Validator\nA pure python avro schema validator.\n\nThe default avro library for Python provide validation of data against the schema, the problem is that the output of \nthis validation doesn't provide information about the error. \nAll you get is the `the datum is not an example of the schema` error message.\n\nWhen working with bigger avro schemas, sometimes is not easy to visually find the field that has an issue.\n\nThis library provide clearer exceptions when validating data against the avro schema, in order to be easier to \nidentify the field that is not compliant with the schema and the problem with that field.\n\n## Installing\nInstall using pip:\n```bash\n$ pip install -U avro_validator\n```\n\n## Validating data against Avro schema\nThe validator can be used as a console application. It receives a schema file, and a data file, validating the data\nand returning the error message in case of failure.\n\nThe avro_validator can also be used as a library in python code.\n\n### Console usage\nIn order to validate the `data_to_validate.json` file against the `schema.avsc` using the `avro_validator` callable, just type:\n```bash\n$ avro_validator schema.avsc data_to_valdate.json\nOK\n```\nSince the data is valid according to the schema, the return message is `OK`.\n\n#### Error validating the data\nIf the data is not valid, the program returns an error message:\n```bash\n$ avro_validator schema.avsc data_to_valdate.json\nError validating value for field [data,my_boolean_value]: The value [123] is not from one of the following types: [[NullType, BooleanType]]\n```\nThis message indicates that the field `my_boolean_value` inside the `data` dictionary has value `123`, which is not \ncompatible with the `bool` type.\n\n#### Command usage\nIt is possible to get information about usage of the `avro_validator` using the help:\n```bash\n$ avro_valdidator -h\n```\n\n### Library usage\n#### Using schema file\nWhen using the avr_validator as a library, it is possible to pass the schema as a file:\n```python\nfrom avro_validator.schema import Schema\n\nschema_file = 'schema.avsc'\n\nschema = Schema(schema_file)\nparsed_schema = schema.parse()\n\ndata_to_validate = {\n 'name': 'My Name'\n}\n\nparsed_schema.validate(data_to_validate)\n```\nIn this example, if the `data_to_validate` is valid according to the schema, then the\n `parsed_schema.validate(data_to_validate)` call will return `True`.\n\n#### Using a dict as schema\nIt is also possible to provide the schema as a json string:\n```python\nimport json\nfrom avro_validator.schema import Schema\n\nschema = json.dumps({\n 'name': 'test schema',\n 'type': 'record',\n 'doc': 'schema for testing avro_validator',\n 'fields': [\n {\n 'name': 'name',\n 'type': 'string'\n }\n ]\n})\n\nschema = Schema(schema)\nparsed_schema = schema.parse()\n\ndata_to_validate = {\n 'name': 'My Name'\n}\n\nparsed_schema.validate(data_to_validate)\n```\nIn this example, the `parsed_schema.validate(data_to_validate)` call will return `True`, since the data is valid according to the schema.\n\n#### Invalid data\nIf the data is not valid, the `parsed_schema.validate` will raise a `ValueError`, with the message containing the error description.\n```python\nimport json\nfrom avro_validator.schema import Schema\n\nschema = json.dumps({\n 'name': 'test schema',\n 'type': 'record',\n 'doc': 'schema for testing avro_validator',\n 'fields': [\n {\n 'name': 'name',\n 'type': 'string',\n 'doc': 'Field that stores the name'\n }\n ]\n})\n\nschema = Schema(schema)\nparsed_schema = schema.parse()\n\ndata_to_validate = {\n 'my_name': 'My Name'\n}\n\nparsed_schema.validate(data_to_validate)\n```\nThe schema defined expects only one field, named `name`, but the data contains only the field `name_2`, \nmaking it invalid according to the schema. In this case, the `validate` method will return the following error:\n```\nTraceback (most recent call last):\n File \"/Users/leonardo.almeida/.pyenv/versions/avro_validator_venv/lib/python3.7/site-packages/IPython/core/interactiveshell.py\", line 3326, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"\", line 23, in \n parsed_schema.validate(data_to_validate)\n File \"/opt/dwh/avro_validator/avro_validator/avro_types.py\", line 563, in validate\n raise ValueError(f'The fields from value [{value}] differs from the fields '\nValueError: The fields from value [{'my_name': 'My Name'}] differs from the fields of the record type [{'name': RecordTypeField }]\n```\nThe message detailed enough to enable the developer to pinpoint the error in the data.\n\n#### Invalid schema\nIf the schema is not valid according to avro specifications, the `parse` method will also return a `ValueError`.\n```python\nimport json\nfrom avro_validator.schema import Schema\n\nschema = json.dumps({\n 'name': 'test schema',\n 'type': 'record',\n 'doc': 'schema for testing avro_validator',\n 'fields': [\n {\n 'name': 'name',\n 'type': 'invalid_type',\n 'doc': 'Field that stores the name'\n }\n ]\n})\n\nschema = Schema(schema)\nparsed_schema = schema.parse()\n```\nSince the schema tries to define the `name` field as `invalid_type`, the schema declaration is invalid, \nthus the following exception will be raised:\n```\nTraceback (most recent call last):\n File \"/Users/leonardo.almeida/.pyenv/versions/avro_validator_venv/lib/python3.7/site-packages/IPython/core/interactiveshell.py\", line 3326, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"\", line 18, in \n parsed_schema = schema.parse()\n File \"/opt/dwh/avro_validator/avro_validator/schema.py\", line 28, in parse\n return RecordType.build(schema)\n File \"/opt/dwh/avro_validator/avro_validator/avro_types.py\", line 588, in build\n record_type.__fields = {field['name']: RecordTypeField.build(field) for field in json_repr['fields']}\n File \"/opt/dwh/avro_validator/avro_validator/avro_types.py\", line 588, in \n record_type.__fields = {field['name']: RecordTypeField.build(field) for field in json_repr['fields']}\n File \"/opt/dwh/avro_validator/avro_validator/avro_types.py\", line 419, in build\n field.__type = cls.__build_field_type(json_repr)\n File \"/opt/dwh/avro_validator/avro_validator/avro_types.py\", line 401, in __build_field_type\n raise ValueError(f'Error parsing the field [{fields}]: {actual_error}')\nValueError: Error parsing the field [name]: The type [invalid_type] is not recognized by Avro\n```\nThe message is clearly indicating that the the `invalid_type` is not recognized by avro.", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/leocalm/avro_validator", "keywords": "avro schema", "license": "", "maintainer": "", "maintainer_email": "", "name": "avro-validator", "package_url": "https://pypi.org/project/avro-validator/", "platform": "", "project_url": "https://pypi.org/project/avro-validator/", "project_urls": { "Bug Reports": "https://github.com/leocalm/avro_validator/issues", "Documentation": "https://avro-validator.readthedocs.io/", "Homepage": "https://github.com/leocalm/avro_validator", "Repository": "https://github.com/leocalm/avro_validator" }, "release_url": "https://pypi.org/project/avro-validator/1.0.9/", "requires_dist": null, "requires_python": ">=3.6", "summary": "Pure python avro schema validator", "version": "1.0.9", "yanked": false, "yanked_reason": null }, "last_serial": 7796127, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "d6ad93578f003499c4c201617848cbc3", "sha256": "18ed5709d48e1497182cb61f6962a4bc4562e641b3eed736e2ed8001b8c59d7e" }, "downloads": -1, "filename": "avro_validator-1.0.0-py3.7.egg", "has_sig": false, "md5_digest": "d6ad93578f003499c4c201617848cbc3", "packagetype": "bdist_egg", "python_version": "3.7", "requires_python": ">=3.6", "size": 22892, "upload_time": "2019-10-22T21:21:37", "upload_time_iso_8601": "2019-10-22T21:21:37.853401Z", "url": "https://files.pythonhosted.org/packages/d0/2a/d452921dfca8bb50dfe5fc79d4959f255b8867c9fa06f61447f1f88c8098/avro_validator-1.0.0-py3.7.egg", "yanked": false, "yanked_reason": null }, { "comment_text": "", "digests": { "md5": "b383cd9c7cc1727e5749c8251199894a", "sha256": "e15f2a3d25899840f668ffc24acfd4b42bfbaca0cb4def7561f97a66c621f221" }, "downloads": -1, "filename": "avro_validator-1.0.0.tar.gz", "has_sig": false, "md5_digest": "b383cd9c7cc1727e5749c8251199894a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 12359, "upload_time": "2019-10-22T21:21:39", "upload_time_iso_8601": "2019-10-22T21:21:39.916516Z", "url": "https://files.pythonhosted.org/packages/79/06/53c53ef2388d8068d1ee7d10d87994e2c9bf6ec0da97933b46c511afadf3/avro_validator-1.0.0.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "b0cd2df2ae127798690deaf032f233e4", "sha256": "e8f220fe86daea5255c4750591776c9d8848eadbdf40e3a17e0e920160db20aa" }, "downloads": -1, "filename": "avro_validator-1.0.1.tar.gz", "has_sig": false, "md5_digest": "b0cd2df2ae127798690deaf032f233e4", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.5", "size": 12481, "upload_time": "2019-10-22T21:59:47", "upload_time_iso_8601": "2019-10-22T21:59:47.940469Z", "url": "https://files.pythonhosted.org/packages/6d/8d/a01e9bd5625f9c0be6159a72bfea55da9139793f66c5fdf04c84ec5b0769/avro_validator-1.0.1.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "7280ae0ab3ad029a9e6a12f998624e8d", "sha256": "d38b77d583234b84ab9583fbd781b69c14ff0e551e099946e37cc61bedc19f59" }, "downloads": -1, "filename": "avro_validator-1.0.2.tar.gz", "has_sig": false, "md5_digest": "7280ae0ab3ad029a9e6a12f998624e8d", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 12476, "upload_time": "2019-10-22T22:06:39", "upload_time_iso_8601": "2019-10-22T22:06:39.631590Z", "url": "https://files.pythonhosted.org/packages/c2/94/ce8ba42bab1c49d64173b69acb19a57a37b2490afeb2b19a2f8010637de7/avro_validator-1.0.2.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "d8b224430119d3ddd6df1d3525a7dc9e", "sha256": "45369a55410310bc44dee3d0b6749b86c37ba28adf6d406d1b3508635956d54f" }, "downloads": -1, "filename": "avro_validator-1.0.3.tar.gz", "has_sig": false, "md5_digest": "d8b224430119d3ddd6df1d3525a7dc9e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 12766, "upload_time": "2019-10-29T07:55:19", "upload_time_iso_8601": "2019-10-29T07:55:19.405188Z", "url": "https://files.pythonhosted.org/packages/5b/88/324e2bbe7b1e9d7c570809753daddc90dcb0577de9398c8fc5b224be2732/avro_validator-1.0.3.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "3ed12f6f0ceb997cfa69b236d8e7b25d", "sha256": "b6e680c8ee66781e8b22f24dae1c285e1985990362917ae0ea9ba16371bd63b9" }, "downloads": -1, "filename": "avro_validator-1.0.4.tar.gz", "has_sig": false, "md5_digest": "3ed12f6f0ceb997cfa69b236d8e7b25d", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 10888, "upload_time": "2019-10-31T10:51:53", "upload_time_iso_8601": "2019-10-31T10:51:53.401424Z", "url": "https://files.pythonhosted.org/packages/d3/02/49b81d7dce3f4154f39b8fd0cae40d6f10433f9de639fa4a418f60eb4848/avro_validator-1.0.4.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "ffe57d564e5e2bca769cd0baa2cd8675", "sha256": "40780546dd7f87941bc1e20b5701f10e20b9352c6e698776d1bf277060997539" }, "downloads": -1, "filename": "avro_validator-1.0.5.tar.gz", "has_sig": false, "md5_digest": "ffe57d564e5e2bca769cd0baa2cd8675", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 10903, "upload_time": "2020-01-30T12:14:31", "upload_time_iso_8601": "2020-01-30T12:14:31.733357Z", "url": "https://files.pythonhosted.org/packages/88/11/eda773482c89d02dfe10371f007fc96c293083e734a3c8f11195b5909b60/avro_validator-1.0.5.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.6": [ { "comment_text": "", "digests": { "md5": "c82fd0468962fdd79654a14fc3df0d9d", "sha256": "c9a5d230c42f5b54e4724dd08040f613a01629dc6f3a9d14394cacb8fdac32ad" }, "downloads": -1, "filename": "avro_validator-1.0.6.tar.gz", "has_sig": false, "md5_digest": "c82fd0468962fdd79654a14fc3df0d9d", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 13214, "upload_time": "2020-01-30T19:44:16", "upload_time_iso_8601": "2020-01-30T19:44:16.793342Z", "url": "https://files.pythonhosted.org/packages/ac/28/d5e05032bac658725e807198c2069dee65ecdcaef79664e1f09ffe525d47/avro_validator-1.0.6.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.7": [ { "comment_text": "", "digests": { "md5": "2a1e08784e2e1100aefd52b5d134bb4a", "sha256": "c3db3ca7930aeed72211445c901118d7bd4e52d6597ce2dc9373468ad5757941" }, "downloads": -1, "filename": "avro_validator-1.0.7.tar.gz", "has_sig": false, "md5_digest": "2a1e08784e2e1100aefd52b5d134bb4a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 10923, "upload_time": "2020-07-24T15:49:00", "upload_time_iso_8601": "2020-07-24T15:49:00.083470Z", "url": "https://files.pythonhosted.org/packages/9c/6b/4a17c3ef14734b83f343ec49e7e90eac661751981513bb013f1c7b6d8f29/avro_validator-1.0.7.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.8": [ { "comment_text": "", "digests": { "md5": "298f38da069cbd186d06f24a8701ba60", "sha256": "f4911dc1f353ad172b8bfb6134d34af7eb6fcc8048ecfffb2d2a15c54706fb8f" }, "downloads": -1, "filename": "avro_validator-1.0.8.tar.gz", "has_sig": false, "md5_digest": "298f38da069cbd186d06f24a8701ba60", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 11020, "upload_time": "2020-07-24T16:17:57", "upload_time_iso_8601": "2020-07-24T16:17:57.837191Z", "url": "https://files.pythonhosted.org/packages/36/b2/c01cc2f264937b117dafaffbb75d862dff67870e6c8b9d65b148b1be9393/avro_validator-1.0.8.tar.gz", "yanked": false, "yanked_reason": null } ], "1.0.9": [ { "comment_text": "", "digests": { "md5": "d306d056c56db9530a40ba0127141609", "sha256": "00a439bbbdc10be95b67ab6dcb6008e90243e0c886c10d735d14c8c004467364" }, "downloads": -1, "filename": "avro_validator-1.0.9.tar.gz", "has_sig": false, "md5_digest": "d306d056c56db9530a40ba0127141609", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 11305, "upload_time": "2020-07-27T11:33:30", "upload_time_iso_8601": "2020-07-27T11:33:30.763212Z", "url": "https://files.pythonhosted.org/packages/06/76/0ba3e21857d9506884e6a5a3e240f249d6f498504c5b4a055c51d6382b07/avro_validator-1.0.9.tar.gz", "yanked": false, "yanked_reason": null } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d306d056c56db9530a40ba0127141609", "sha256": "00a439bbbdc10be95b67ab6dcb6008e90243e0c886c10d735d14c8c004467364" }, "downloads": -1, "filename": "avro_validator-1.0.9.tar.gz", "has_sig": false, "md5_digest": "d306d056c56db9530a40ba0127141609", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6", "size": 11305, "upload_time": "2020-07-27T11:33:30", "upload_time_iso_8601": "2020-07-27T11:33:30.763212Z", "url": "https://files.pythonhosted.org/packages/06/76/0ba3e21857d9506884e6a5a3e240f249d6f498504c5b4a055c51d6382b07/avro_validator-1.0.9.tar.gz", "yanked": false, "yanked_reason": null } ], "vulnerabilities": [] }