{ "info": { "author": "Quantlane", "author_email": "code@quantlane.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3 :: Only" ], "description": "======\nCharon\n======\n\nCharon makes data serialization and deserialization simple and secure.\n\nCharon was inspired by the `Camel project `_,\nbut unlike Camel it does not force a particular serialization format. Charon offers a simple interface\nfor defining functions that convert between complex Python objects and primitives that can be\nserialized into a format of your choice.\n\nIn other words, this is not a tool that takes an object and serializes it to JSON.\nIt is a tool that takes an object and using a *user-defined serialization function* it converts it\nto basic Python types which are then serializable to JSON, YAML, msgpack etc.\n\nThis project also contains (de)serializers for some standard types like ``datetime.datetime``,\n`set`` and ``frozenset``.\n\nRegistries can be tested (see `Writing tests`_). This heavily relies on the ``pytest`` module and allows the user\nto:\n\n * test implemented dumpers and loaders while keeping track of implemented versions (see `Version tests`_);\n * test if all dumpers and loaders have their tests (see `Metatests <#metatests-tests-testing-the-existence-of-loader-dumper-tests>`_);\n * write tests easily by simply parametrizing a predefined generalized test case (see `Generic tests`_).\n\n\nUsage\n=====\n\nDefine dumpers and loaders using decorators with a ``charon.CodecRegistry`` instance\nand then group multiple codec registries together using ``charon.Codec``.\n\nYou can serialize and deserialize objects for which dumpers and loaders were defined by calling\nthe ``dump`` and ``load`` methods of ``Codec``.\n\nWhenever a codec serializes an object, it checks what the serialization function returned and if it finds an object\nthat is not a basic Python type, it tries to serialize it again. This kind of recurrent behavior can be time-consuming\n(and even dangerous for circular references).\n\n.. note::\n\n If there are multiple registries able to serialize the same object\n with same version, the first one found from the *end* of the registries list is picked and used.\n\n----------------\nCreating dumpers\n----------------\n\nCreating a dumper is simple. First you have to create a registry in which this dumper will be stored.\nThis is done by:\n\n::\n\n import charon\n\n registry = charon.CodecRegistry()\n\n\nThen you just have to decorate a function which will receive an instance to serialize.\nThe class of the instance to serialize is given to the decorator as the first argument.\nThe second argument is the *version* which allows versioning of your class and its serialized structure.\nThis approach allows you to restore serialized objects that were created using an older structure\n(field names may change, variables may come and go etc.).\n\nThe last parameter ``class_hash`` is optional and is used for version testing (see `Version tests`_).\n\nThe function that you decorate should accept one argument - the object instance to be serialized.\nWe serialize it into a dictionary containing base values from which we can restore this object.\n\n.. _timedelta_dumper:\n\n::\n\n @registry.dumper(datetime.timedelta, version = 1, class_hash = 'dadf350239d3779d72d9c933ab52db1b')\n def _dump_timedelta(obj):\n return {'days': obj.days, 'seconds': obj.seconds, 'microseconds': obj.microseconds}\n\n\n\n----------------\nCreating loaders\n----------------\n\nCreating a loader is also simple. Again you have to have a registry which will store this loader (we use the one defined\nin the previous section), and again you have to decorate a function which will receive data,\nbut this time these data are those we created using the dumper function.\n\nThe loader decorator is similar to the dumper one. The first argument is the class which we should deserialize\n(which we will instantiate from the data received). The second argument, version, is used as mentioned above for\nversioning of the class implementation. It allows developers to change the structure of their classes and still be able\nto restore previously serialized objects into this newer structure.\n\nThe function which you decorate should accept one parameter: the data previously returned by the serialization\nfunction (see `_dump_timedelta`).\nFrom the received dictionary we recreate an object using its constructor (if applicable) or\nsetting its internal variables directly (yes, this may be appropriate when deserializing data).\n\n::\n\n @registry.loader(datetime.timedelta, version = 1, class_hash = 'dadf350239d3779d72d9c933ab52db1b')\n def _load_timedelta(data):\n return datetime.timedelta(days = data['days'], seconds = data['seconds'], microseconds = data['microseconds'])\n\n\n-------------------------\nUsing loaders and dumpers\n-------------------------\n\nAfter we have defined all our classes which will be serialized we can use ``charon.Codec`` to serialize\nand deserialize objects.\n\n.. code:: python\n\n >>> import datetime\n >>> import charon, charon.extensions\n >>> codec = charon.Codec([charon.extensions.STANDARD_REGISTRY])\n >>> delta = datetime.timedelta(seconds = 42)\n >>> encoded = codec.dump(delta)\n >>> print(encoded)\n {'!meta': {'dtype': 'timedelta', 'version': 2}, 'params': [0, 42, 0]}\n >>> loaded = codec.load(encoded)\n >>> print(delta)\n 0:00:42\n >>> print(loaded)\n 0:00:42\n >>> print(delta == loaded)\n True\n\nWriting tests\n=============\n\n\n.. note:: These tests use ``pytest`` module.\n\nThis section describes options that Charon offers for testing loaders and dumpers.\nThese tests are meant to help with keeping all loaders and dumpers tested and\nup to date with class structure.\n\nThere is also a test function that should represent basic test structure for testing\na serialization/deserialization pipeline.\n\n.. _pytest.mark.parametrize: https://docs.pytest.org/en/latest/parametrize.html\n.. _pytest_generate_tests: https://docs.pytest.org/en/latest/parametrize.html#pytest-generate-tests\n\n.. _generic_tests:\n\n\n-------------\nGeneric tests\n-------------\n\nCharon contains a generic test definition ``test_serialization_pipeline``.\nThis test is a generalized test case consisting of object serialization, deserialization and comparison.\n\nThe original object and the deserialized object are both tested if they match the class for which the test is used.\nThis is a sanity check to prevent serializating instances of one class and getting instances of another class\nafter deserialization.\n\nThe original and the deserialized objects are then compared against each other using the ``vars`` function\nif possible, otherwise the standard equality operator (``__eq__``) is used.\n\nUsage\n-----\n\nFirst you have to import the appropriate test function. This is best done in ``conftest.py`` because we will use it later.\nYou also have to define a ``serializer`` fixture to use with this test.\n\n::\n\n import pytest\n\n import charon\n from charon.testing.generic import test_serialization_pipeline\n\n\n @pytest.fixture\n def serializer():\n return charon.Codec([registry])\n\n\nThen you have to define parameters.\nYou can rename the test function to some convenient name and then use a wrapper to call it.\nBut it is better to parametrize it directly using the pytest marker `pytest.mark.parametrize`_:\n\n::\n\n pytest.mark.parametrize([(ExampleClass, ExampleClass('Ahoy'))])(test_serialization_pipeline)\n\n\n\nAnother way to parametrize it is by using the `pytest_generate_tests`_ function of pytest.\n\n::\n\n def pytest_generate_tests(metafunc):\n '''\n Generates test cases for simple deserialization and serialization.\n Test cases are generated by functions with ```generate_``` prefix\n '''\n if metafunc.function.__name__ == 'test_serialization_pipeline':\n metafunc.parametrize('cls, original_obj', [(ExampleClass, ExampleClass('Ahoy'))])\n\n.. _Metatests:\n\n--------------------------------------------------------------\nMetatests - Tests testing the existence of loader/dumper tests\n--------------------------------------------------------------\n\nCharon contains tests for testing whenever all dumpers and loaders have tests. This aproach is called metatesting.\n\nThese tests are convenient when you want to make sure that all of your dumpers and loaders have tests.\n\n\nUsage\n-----\n\n\nTo use this metatest you have to mark your dumper and loader tests with ``pytest.mark``.\n\n.. code:: python\n\n @pytest.mark.charon(cls = ExampleClass, dumper_test = True, loader_test = False)\n def test_example_dumper(self):\n pass\n\nAs you can see you use keywords to set up the mark. The ``cls`` keyword specifies the class for which this test works,\n``dumper_test`` specifies if this is a dumper test and obviously ``loader_test`` specifies whenever this is a loader test.\n\nThis way you mark all of your tests. Then you just have to import metatests from the ``charon.testing.metatest`` package.\n\n.. code:: python\n\n from charon.testing.metatest import (\n test_charon_dumper_tests,\n test_charon_loader_tests,\n scope_charon_tests\n )\n\n pytest.fixture(scope = 'module')(scope_charon_tests)\n\n\n\nAnd that's all: from this point forth all loader and dumper methods would have to be properly tested.\n\n.. note:: This only tests the existence of tests. It does not test the version for which those teste were written. To test the version for which the tests were written see `Version tests`_.\n\n.. note::\n\n pytest.fixture(scope = 'module')(scope_charon_tests)\n\n We create this fixture in the module scope and in the session scope because that could create false positive cases.\n Example:\n You have have two registries in different codecs which serialize and deserialize the same class differently.\n One of these codecs have tests for this class and the others don't. If you use a session fixture in this case\n you will get a false positive check, because the fixture will return tests which are defined for one of the codecs\n but not for the other. Metatests do not check to which codec (implementation) is a test bound.\n\n\n.. _version_tests:\n\n-------------\nVersion tests\n-------------\n\nIn large projects it is sometimes difficult to keep track of changes in classes, and to keep their serialization up to date.\nFor example, in a project with multiple people colaborating on it changes in one class can be made separatwly by multiple\ndevelopers, and one of them may forget to update the particular dumpers and loaders and increment their version.\n\nTo prevent this, Charon has an option to create a hash of class implementation (hash of the AST) and annotate\na dumper / loader with it. Charon also includes tests from ``charon.testing.ast_hash`` called\n``test_dumpers_version``\nand ``test_loaders_version``.\n\nUsage\n-----\n\nTo use this feature first we have to create a hash of the current implementation.\nThis can be done using ``charon_ast_hash`` script provided by this package.\nSee: ``Generating a hash``\n\nWhen we havea hash of the class implementation we add a keyword to the standard decorator for loader / dumper.\n\n.. code:: python\n\n @registry.dumper(Object, version = 1, class_hash = 'd2498176fad81ad017d1b0875eeeeb1b')\n def _load_object_v1(_):\n pass\n\nThis way we pass the hash of the class implementation to the registry.\n\n.. note::\n\n The hash is kept only for the latest version of a dumper / loader because\n we cannot check these versions with older implementations of a particular class.\n\nThese are all the changes in dumper and loader implementations. Next you have to import the test methods\n`charon.testing.ast_hash.test_dumpers_version`` and ``charon.testing.ast_hash.test_loaders_version``\ninto your test file and pass it an instance of ``charon.Codec``, containing the registries you want to test.\n\n.. code:: python\n\n from charon.testing.ast_hash import test_dumpers_version, test_loaders_version\n\n @pytest.fixture\n def serializer():\n return charon.Codec([my_registry])\n\n\nFrom this point on, whenever you run pytest, all your loaders and dumpers which have ``class_hash`` defined will be\nchecked against the hash of the current class implementation.\n\n\n.. _generating-class-hash:\n\n---------------------------------------------\nGenerating a hash of a class implementation\n---------------------------------------------\n\nTo generate the hash of the AST (Abstract Syntax Tree) of a class you can use script provided with Charon\ncalled ``charon_ast_hash``. This script takes a list of classes to be hashed as an argument list.\n\n.. note::\n\n This command internally uses the ``inspect`` module to get the source code which is then parsed by the ``ast`` module.\n Methods and classes from ``__builtins__`` and from compiled libraries cannot be hashed.\n\n\nExample usage\n--------------\n.. code:: bash\n\n $ charon_ast_hash datetime.datetime datetime.time\n datetime.datetime: 4927808ca19f2a1494719baa11024a7d\n datetime.time: c36b819f18698ee9143ecd92e3788c66\n\nStandard Registry\n=================\n\nThe Charon package comes with an implementation for some Python types that are built in or in the standard library:\n\n* ``decimal.Decimal``\n* ``set``\n* ``frozenset``\n* ``datetime.datetime``\n* ``datetime.date``\n* ``datetime.time``\n* ``datetime.timedelta``\n\nThis registry can be used by simply creating your ``charon.codec`` with an additional registry\n``charon.extensions.STANDARD_REGISTRY``.\n\n\n\n-------------\nExample usage\n-------------\n\nBasic usage is pretty simple. You just have to create a ``charon.codec`` object with an additional codec registry\n``charon.extensions.STANDARD_REGISTRY``,\npreferably at the begining of the list (in case you would want to override standard implementations of dumpers / loaders).\n\n.. code:: python\n\n >>> import charon, charon.extensions\n >>> import decimal\n >>> codec = charon.Codec([charon.extensions.STANDARD_REGISTRY])\n >>> number = decimal.Decimal('4.5')\n >>> print(number)\n 4.5\n >>> serialized = codec.dump(number)\n >>> print(serialized)\n {'!meta': {'dtype': 'Decimal', 'version': 1}, 'params': '4.5'}\n >>> loaded = codec.load(serialized)\n >>> print(loaded)\n 4.5", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/qntln/charon", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "ql-charon", "package_url": "https://pypi.org/project/ql-charon/", "platform": "", "project_url": "https://pypi.org/project/ql-charon/", "project_urls": { "Homepage": "https://github.com/qntln/charon" }, "release_url": "https://pypi.org/project/ql-charon/2.1.2/", "requires_dist": null, "requires_python": "", "summary": "Serialization library as a replacement for pickle", "version": "2.1.2" }, "last_serial": 3657641, "releases": { "2.1.2": [ { "comment_text": "", "digests": { "md5": "cf4d2150a777588f95dd9105c8d747db", "sha256": "725c96da544f340baef31c0dbb0051f1a62cd77cfe881bf2d3634b30e0b4722a" }, "downloads": -1, "filename": "ql-charon-2.1.2.tar.gz", "has_sig": true, "md5_digest": "cf4d2150a777588f95dd9105c8d747db", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 138434, "upload_time": "2018-03-10T19:11:57", "url": "https://files.pythonhosted.org/packages/c3/8d/3869eb4a08ffeea813f251a044b4fba02d07e76f7bd561ea811028dbfdd8/ql-charon-2.1.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "cf4d2150a777588f95dd9105c8d747db", "sha256": "725c96da544f340baef31c0dbb0051f1a62cd77cfe881bf2d3634b30e0b4722a" }, "downloads": -1, "filename": "ql-charon-2.1.2.tar.gz", "has_sig": true, "md5_digest": "cf4d2150a777588f95dd9105c8d747db", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 138434, "upload_time": "2018-03-10T19:11:57", "url": "https://files.pythonhosted.org/packages/c3/8d/3869eb4a08ffeea813f251a044b4fba02d07e76f7bd561ea811028dbfdd8/ql-charon-2.1.2.tar.gz" } ] }