{ "info": { "author": "saaj", "author_email": "mail@saaj.me", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Software Development :: Libraries" ], "description": ".. image:: https://bitbucket-badges.atlassian.io/badge/saaj/hermes.svg?ref=default\n :target: https://bitbucket.org/saaj/hermes/addon/pipelines/home\n.. image:: https://codecov.io/bitbucket/saaj/hermes/branch/default/graph/badge.svg\n :target: https://codecov.io/bitbucket/saaj/hermes/branch/default\n.. image:: https://badge.fury.io/py/HermesCache.png\n :target: https://pypi.python.org/pypi/HermesCache\n\n***********\nHermesCache\n***********\n\nHermes is a Python caching library. The requirements it was designed to fulfil:\n\n* Tag-based cache invalidation\n* Dogpile effect prevention\n* Thread-safety\n* Straightforward design\n* Simple, at the same time, flexible decorator as end-user API\n* Interface for implementing multiple backends\n\nImplemented backends: ``redis``, ``memcached``, ``dict`` (no expiry).\n\n\nInstall\n=======\n\n.. sourcecode::\n\n pip install HermesCache\n\nMake sure desired backend client library is installed. See below.\n\n\nUsage\n=====\n\nThe following demonstrates all end-user API.\n\n.. sourcecode:: python\n\n import hermes.backend.redis\n\n cache = hermes.Hermes(hermes.backend.redis.Backend, ttl = 600, host = 'localhost', db = 1)\n\n\n @cache\n def foo(a, b):\n return a * b\n\n class Example:\n\n @cache(tags = ('math', 'power'), ttl = 1200)\n def bar(self, a, b):\n return a ** b\n\n @cache(tags = ('math', 'avg'), key = lambda fn, *args, **kwargs: 'avg:{0}:{1}'.format(*args))\n def baz(self, a, b):\n return (a + b) / 2.0\n\n\n print(foo(2, 333))\n\n example = Example()\n print(example.bar(2, 10))\n print(example.baz(2, 10))\n\n foo.invalidate(2, 333)\n example.bar.invalidate(2, 10)\n example.baz.invalidate(2, 10)\n\n cache.clean(['math']) # invalidate entries tagged 'math'\n cache.clean() # flush cache\n\n.. note::\n The API encourages import-time instantiation of ``Hermes`` to allow end-user\n to decorate its functions. The instantiation has no side effects. Underlying\n backend server connections are lazy.\n\nFor advanced examples look in\n`test suite `_.\n\n\nTagging cache entries\n=====================\n\nFirst let's look how basic caching works.\n\n.. sourcecode:: python\n\n import hermes.backend.dict\n\n cache = hermes.Hermes(hermes.backend.dict.Backend)\n\n @cache\n def foo(a, b):\n return a * b\n\n foo(2, 2)\n foo(2, 4)\n\n print(cache.backend.dump())\n # {\n # 'cache:entry:foo:515d5cb1a98de31d': 8,\n # 'cache:entry:foo:a1c97600eac6febb': 4\n # \u2193\n # argument hash\n # }\n\nBasically we have a key-value storage with O(1) complexity for ``set``, ``get`` and ``delete``.\nThis means that the speed of operation is constant and irrelevant of number of items already\nstored. When a callable (function or method) is cached, the key is calculated per invocation from\ncallable itself and passed arguments. Callable's return value is saved to the key. Next invocation\nwe can use the value from cache.\n\n *\"There are only two hard problems in Computer Science: cache invalidation and naming things.\"* \u2014\n Phil Karlton\n\nSo it comes in a complex application. There's a case that certain group of methods operate the same\ndata and it's impractical to invalidate individual entries. In particular, it often happens when\nmethod returns complex values, spanning multiple entities. Cache tagging makes it possible to mark\nthis group of method results with a tag and invalidate them all at once.\n\nHere's `article\n`_ by Eric\nFlorenzano which explains the idea. Let's look the code.\n\n.. sourcecode:: python\n\n import hermes.backend.dict\n\n cache = hermes.Hermes(hermes.backend.dict.Backend)\n\n @cache(tags = ('tag1', 'tag2'))\n def foo(a, b):\n return a * b\n\n foo(2, 2)\n\n print(cache.backend.dump())\n # {\n # 'cache:tag:tag1': '0674536f9eb4eb19',\n # 'cache:tag:tag2': 'db22b5ab2e504895',\n # 'cache:entry:foo:a1c97600eac6febb:c1da510b3d42bad6': 4\n # \u2193\n # tag hash\n # }\n\nWhen we want to tag a cache entry, first we need to create the tag entries. Each tag is represented\nby its own entry. Value of tag entry is set to random value each time tag is created. Once all tags\nvalues exist, they are joined and hashed. Tag hash is added to cache entry key.\n\nOnce we want to invalidate tagged entries we just need to remove the tag entry. Without any of tag\nvalues tag hash was created with, it is impossible to construct the entry key so the tagged cache\nentries become inaccessible thus invalidated. As usually a feature built on-top of another feature\nadds complexity.\n\nPerformance. All operations become O(n) where *n* is number of entry tags. However since we can\nrarely need more than a few dozens of tags, practically it is still O(1). Tag entry operations are\nbatched so the implications on number of network operations go as follow:\n\n* ``set`` \u2013 3x backend calls (``get + 2 * set``) in worst case. Average is expected to be 2x when\n all used tag entries are created.\n* ``get`` \u2013 2x backend calls.\n* ``delete`` \u2013 2x backend calls.\n\nMemory overhead consists of tag entries and stale cache entries. Demonstrated below.\n\n.. sourcecode:: python\n\n import hermes.backend.dict\n\n cache = hermes.Hermes(hermes.backend.dict.Backend)\n\n @cache(tags = ('tag1', 'tag2'))\n def foo(a, b):\n return a * b\n\n foo(2, 2)\n\n print(cache.backend.dump())\n # {\n # 'cache:tag:tag1': '047820ac777abe8a',\n # 'cache:tag:tag2': '126365ec7175e851',\n # 'cache:entry:foo:a1c97600eac6febb:5cae80f5e7d58329': 4\n # }\n\n cache.clean(['tag1'])\n foo(2, 2)\n\n print(cache.backend.dump())\n # {\n # 'cache:tag:tag1': '66336fec212def16', \u2190 recreated tag entry\n # 'cache:tag:tag2': '126365ec7175e851',\n # 'cache:entry:foo:a1c97600eac6febb:8e7e24cf70c1f0ab': 4,\n # 'cache:entry:foo:a1c97600eac6febb:5cae80f5e7d58329': 4 \u2190 garbage\n # }\n\nSo the TTLs should be chosen elaborately. With Redis backend it's also recommended\nto set `maxmemory-policy `_ to ``volatile-lru``.\n\n\nBackend and client library\n==========================\n\nSupported dependencies are listed in\n`tox.ini `_\nof the package. The way they put together is on project's\n`drone.io CI page `_.\n\n\nRedis\n-----\n``hermes.backend.redis`` depends on `redis `_. Optionally\n`hiredis `_ can be installed in addition to boost protocol\nparsing. However *hiredis* gives significant advantage on big bulk operations and in\ncontext of the package adds about 10%.\n\nMemcached\n---------\n``hermes.backend.memcached`` depends either on pure-python\n`python-memcached `_\n(`python3-memcached `_ as prior one is still\nbroken on Python 3 with binary data) or on, *libmemcached* wrapper,\n`pylibmc `_. *pylibmc* gives about 50% improvement.\n\nDict\n----\n``hermes.backend.dict`` is neither complete backend nor it is designed for any distributed use.\nOriginal purpose was a development need and in fact it's just a wrapper on Python ``dict``. It\ndoesn't implement entry expiry and any memory limiting. Though it can be used in limited cases\nwhere cached entry size is a priori small and actual state is maintained only with manual\ninvalidation.\n\n\nPerformance\n-----------\n\nHere are some clues about performance of backends and client libraries. It wasn't an intension\nto provide some statistically significant performance estimation. These are just results from\none of CI builds.\n\n.. image:: https://goo.gl/ZT0phq\n\n.. image:: https://goo.gl/ZYCSGi\n\n\nReviewed implementations\n========================\n\nBefore I wrote the library I looked through the Cheese Shop for one that fits my needs.\nUnfortunately there was none, however some matched partially and were the inspiration in\ncertain aspects:\n\n`cache `_\n\nPro:\n * clean end-user API\n * straightforward design\nCon:\n * no auto cache key calculation\n * no dogpile effect prevention\n * no cache entry tagging\n * fail with instance methods\n\n`dogpile.cache `_\n\nPro:\n * mature\n * very well documented\n * prevents dogpile effect\nCon:\n * no cache entry tagging\n * complicated code-base\n * not concise end-user API\n\n`cache-tagging `_\n\nPro:\n * cache entry tagging\nCon:\n * designed for the news website scaffolding framework\n * thus bloat is all around\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://bitbucket.org/saaj/hermes", "keywords": "python cache tagging redis memcached", "license": "LGPL-2.1+", "maintainer": "", "maintainer_email": "", "name": "HermesCache", "package_url": "https://pypi.org/project/HermesCache/", "platform": "Any", "project_url": "https://pypi.org/project/HermesCache/", "project_urls": { "Homepage": "https://bitbucket.org/saaj/hermes" }, "release_url": "https://pypi.org/project/HermesCache/0.7.2/", "requires_dist": null, "requires_python": "", "summary": "Python caching library with tag-based invalidation and dogpile effect prevention", "version": "0.7.2" }, "last_serial": 3475024, "releases": { "0.1.5": [ { "comment_text": "", "digests": { "md5": "d97d4dfd44b4fad142e9172640472148", "sha256": "49db074ba12d01839ef7cdcc84aaf51f4037eb943bb53bdb9f0025d6bfe84af3" }, "downloads": -1, "filename": "HermesCache-0.1.5.zip", "has_sig": false, "md5_digest": "d97d4dfd44b4fad142e9172640472148", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22211, "upload_time": "2013-07-19T12:23:28", "url": "https://files.pythonhosted.org/packages/43/e4/9e42bb2539bd9d7dc700640d0d3800601d493da6c4811ff15762bcccaa2a/HermesCache-0.1.5.zip" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "70f605c69c07bcb06f60f48063dcda89", "sha256": "30912310120fd83217fb079c5c263debd9124cdc4982328c1b957472372aee5d" }, "downloads": -1, "filename": "HermesCache-0.2.0.tar.gz", "has_sig": false, "md5_digest": "70f605c69c07bcb06f60f48063dcda89", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14769, "upload_time": "2013-10-31T13:30:48", "url": "https://files.pythonhosted.org/packages/6e/36/e9fbce0e1ea63ef06971aea305d6124959b9a9c56868a1863ec9eeafb1d2/HermesCache-0.2.0.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "de6517396a9b6f26dfa067e26f30383b", "sha256": "5c6aa1aeaedcb6b2e8fe718e70892e1cff4ac8e2b6b8570e2cf4013b85dc8598" }, "downloads": -1, "filename": "HermesCache-0.3.0.tar.gz", "has_sig": false, "md5_digest": "de6517396a9b6f26dfa067e26f30383b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15066, "upload_time": "2014-03-31T10:23:46", "url": "https://files.pythonhosted.org/packages/7f/dd/ef5811d5e334dc80199e1575010988ab4572c56ffe0a6f6e519f89a5891d/HermesCache-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "8eea6ce637e689acdf4e22be33acf437", "sha256": "dfc02bef9416f0f849140f99423b0249523bbef2723e3595c0862a444fa64ebf" }, "downloads": -1, "filename": "HermesCache-0.4.0.tar.gz", "has_sig": false, "md5_digest": "8eea6ce637e689acdf4e22be33acf437", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15633, "upload_time": "2014-04-10T10:00:42", "url": "https://files.pythonhosted.org/packages/d2/67/9ac4580190ed8c7202011f4184a0b8a918459da4a4fb2f63a4c3bc967d95/HermesCache-0.4.0.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "6ea5c8e1d4c8c8a5631d75c56af7a5e1", "sha256": "e88655abd2baa85762050d2a021cb5627a18c4fd6f8f5692ab1d02ff7b1d47b9" }, "downloads": -1, "filename": "HermesCache-0.5.0.tar.gz", "has_sig": false, "md5_digest": "6ea5c8e1d4c8c8a5631d75c56af7a5e1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20072, "upload_time": "2014-07-25T12:49:35", "url": "https://files.pythonhosted.org/packages/25/5c/c9915f66d46492cbc5f4f124e51fc1b90faef362f6b4b9b070e6e377a331/HermesCache-0.5.0.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "fb08cfca1122eb8b3eda75baefc375a5", "sha256": "952e6d198fef7d2af3520514e86de59887b75a181b5d5cdaa6db1c2c9f9ae249" }, "downloads": -1, "filename": "HermesCache-0.5.1.tar.gz", "has_sig": false, "md5_digest": "fb08cfca1122eb8b3eda75baefc375a5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20063, "upload_time": "2015-03-13T11:10:27", "url": "https://files.pythonhosted.org/packages/6f/64/f93468f49cd4188a24eddaecd27b815814c6981de7ea904a91f8f07f453a/HermesCache-0.5.1.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "0436d324a756e8357e5f7b47ac3b1b6e", "sha256": "9f3bfbcc7f48d1d83e1a26becff77227895a4681eb6e49a34500e15c111f2589" }, "downloads": -1, "filename": "HermesCache-0.5.2.tar.gz", "has_sig": false, "md5_digest": "0436d324a756e8357e5f7b47ac3b1b6e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20373, "upload_time": "2015-06-09T18:23:48", "url": "https://files.pythonhosted.org/packages/9c/cf/e78f74632c98d44f37818efe01e7d0d0b748de85f95dad123fad86ff638f/HermesCache-0.5.2.tar.gz" } ], "0.6.0": [ { "comment_text": "", "digests": { "md5": "74372c0b205c4e2df8105d14ac400167", "sha256": "67af66d6bdca30f71fb1fed165b1312359a6ebcabf9a6c31ba82970dcef9b668" }, "downloads": -1, "filename": "HermesCache-0.6.0.tar.gz", "has_sig": false, "md5_digest": "74372c0b205c4e2df8105d14ac400167", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19900, "upload_time": "2015-12-06T16:51:03", "url": "https://files.pythonhosted.org/packages/2a/d8/f1cdfe0235422efbdd80b89de7b804c74f6d1632e4cea33ae3eaa71f085d/HermesCache-0.6.0.tar.gz" } ], "0.6.1": [ { "comment_text": "", "digests": { "md5": "72c353d38f1875fd5e052047dcbfa729", "sha256": "5b0934c09c3736f653ebda9290fe752b33b37d798b0842f029df00f719e2b3b1" }, "downloads": -1, "filename": "HermesCache-0.6.1.tar.gz", "has_sig": false, "md5_digest": "72c353d38f1875fd5e052047dcbfa729", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18332, "upload_time": "2017-02-18T21:10:50", "url": "https://files.pythonhosted.org/packages/1c/06/6dd8511e302d45d703f0aabc142d8043f51f30524fa753acc0686dd49f34/HermesCache-0.6.1.tar.gz" } ], "0.7.1": [ { "comment_text": "", "digests": { "md5": "64f664b1c3fc26f6a35a7b6d063536ce", "sha256": "251565139803868c2b4dc97490de00f600c97e26f18eea5f8d00d7e3c10ce350" }, "downloads": -1, "filename": "HermesCache-0.7.1.tar.gz", "has_sig": false, "md5_digest": "64f664b1c3fc26f6a35a7b6d063536ce", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18282, "upload_time": "2017-12-25T20:54:32", "url": "https://files.pythonhosted.org/packages/fe/35/afcc213a014fe9d8eb50d94c1d46b281da5c2d92f6a837f03ee5a66bceae/HermesCache-0.7.1.tar.gz" } ], "0.7.2": [ { "comment_text": "", "digests": { "md5": "dc229509f701161089266e8cad8b8a24", "sha256": "6a9e7df6079a530582b614f03b36b0b0a95210c68d36aad330b6154fbf24b9cb" }, "downloads": -1, "filename": "HermesCache-0.7.2.tar.gz", "has_sig": false, "md5_digest": "dc229509f701161089266e8cad8b8a24", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19563, "upload_time": "2018-01-09T17:49:52", "url": "https://files.pythonhosted.org/packages/1b/a5/2911150b62f8d701808219d9cd72f57ccca0fed154b4265ed02e654946f9/HermesCache-0.7.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "dc229509f701161089266e8cad8b8a24", "sha256": "6a9e7df6079a530582b614f03b36b0b0a95210c68d36aad330b6154fbf24b9cb" }, "downloads": -1, "filename": "HermesCache-0.7.2.tar.gz", "has_sig": false, "md5_digest": "dc229509f701161089266e8cad8b8a24", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19563, "upload_time": "2018-01-09T17:49:52", "url": "https://files.pythonhosted.org/packages/1b/a5/2911150b62f8d701808219d9cd72f57ccca0fed154b4265ed02e654946f9/HermesCache-0.7.2.tar.gz" } ] }