{ "info": { "author": "Mikhail Shvein", "author_email": "work_shvein_mihail@mail.ru", "bugtrack_url": null, "classifiers": [], "description": "# django-pg-hll\nProvides a django wrapper for [postgresql-hll library by CitusData](https://github.com/citusdata/postgresql-hll#the-importance-of-hashing)\n\n## Requirements\n* Python 2.7 or Python 3.4+\n* django >= 1.9\n* pytz\n* six\n* typing\n* psycopg2\n* PostgreSQL 9.4+ \n\n## Installation\nInstall via pip: \n`pip install django-pg-hll` \nor via setup.py: \n`python setup.py install`\n\n## Usage\n### Prerequisites\nInstall [postgresql-hll extension](https://github.com/citusdata/postgresql-hll#install)\n\n### Creating table with hll field\n* Add HllField to your model:\n ```python\n from django.db import models\n from django_pg_hll import HllField\n\n class MyModel(models.Model):\n hll = HllField()\n ```\n* Call [makemigrations](https://docs.djangoproject.com/en/2.1/ref/django-admin/#django-admin-makemigrations) to create a migration\n* Call [migrate](https://docs.djangoproject.com/en/2.1/ref/django-admin/#django-admin-migrate) to apply migration.\n\n### Hll values\nIn order to create and update Hll this library introduces a set of functions \n(corresponding to [postgres-hll hash functions](https://github.com/citusdata/postgresql-hll#hashing)),\n to hash values:\n```python\nfrom django_pg_hll import HllField\n\n# Empty hll\nHllEmpty()\n\n# Hash from boolean\nHllBoolean(True)\n\n# Hash from integer with different ranges\nHllSmallInt(1)\nHllInteger(65540)\nHllBigint(2147483650)\n\n# Hash from bytes sequence\nHllByteA(b'test')\n\n# Hash from text\nHllText('test')\n\n# Auto detection of type by postgres-hll\nHllAny('some data')\n```\n\nTo save a value to HllField, you can pass any of these functions as a value:\n```python\nfrom django_pg_hll import HllInteger\n\ninstance = MyModel.objects.create(hll=HllInteger(123))\ninstance.hll |= HllInteger(456)\ninstance.save()\n```\n\n#### Chaining hll values\nHll values can be chained with each other and functions like `django.db.models.F` using `|` operator. \nThe chaining result will be `django_pg_hll.values.HllSet` instance, which can be also saved to database. \nYou can also chain simple values and iterables. \nIn this case, library will try to detect appropriate hashing function, based on value. \n*Important*: Native django functions can't be used as chain start, as `|` operator is redeclared for HllValue instances. \nExample:\n```python\nfrom django_pg_hll import HllInteger\nfrom django.db.models import F\n\ninstance = MyModel.objects.create(hll=HllInteger(123))\n\n# This works\ninstance.hll |= HllInteger(456)\ninstance.hll = HllInteger(456) | F('hll')\ninstance.hll |= 789 # HllSmallInt will be used\ninstance.hll |= 100500 # HllInteger will be used\ninstance.hll |= True # HllBoolean will be used\ninstance.hll |= {1, 2, 3, 4, 5} # set. HllSmallInt will be used.\n\n# This throws exception, as F function doesn't support bitor operator\ninstance.hll = F('hll') | HllInteger(456)\n```\n\n#### Hashing seed\nYou can pass `hash_seed` optional argument to any HllValue, expecting data. \n[Look here](https://github.com/citusdata/postgresql-hll#the-importance-of-hashing) for more details about hashing.\n\n\n### Filtering QuerySet\nHllField realizes `cardinality` lookup (returning integer value) in order to make filtering easier:\n```python\nMyModel.objects.filter(hll__cardinality=3).count()\n```\n\n### Aggregate functions\nIn order to count aggregations and annotations, library provides 4 aggregate functions:\n* `django_pg_hll.aggregate.Cardinality`\n Counts cardinality of hll field\n* `django_pg_hll.aggregate.UnionAgg`\n Aggregates multiple hll fields to one hll.\n* `django_pg_hll.aggregate.UnionAggCardinality`\n Counts cardinality of hll, combined by UnionAgg function. In fact, it does `Cardinality(UnionAgg(hll))`. \n P. s. django doesn't give ability to use function inside function.\n* `django_pg_hll.aggregate.CardinalitySum`\n Counts sum of multiple rows hll cardinalities. In fact, it does `Sum(Cardinality(hll))`. \n P. s. django doesn't give ability to use function inside function.\n```python\nfrom django.db import models\nfrom django_pg_hll import HllField, HllInteger\nfrom django_pg_hll.aggregate import Cardinality, UnionAggCardinality, CardinalitySum\n\n\nclass ForeignModel(models.Model):\n pass\n\n\nclass MyModel(models.Model):\n hll = HllField()\n fk = models.ForeignKey(ForeignModel)\n\nMyModel.objects.bulk_create([\n MyModel(fk=1, hll=HllInteger(1)),\n MyModel(fk=2, hll=HllInteger(2) | HllInteger(3) | HllInteger(4)),\n MyModel(fk=3, hll=HllInteger(4))\n])\n\nMyModel.objects.annotate(card=Cardinality('hll_field')).values_list('id', 'card')\n# outputs (1, 1), (2, 3), (3, 1)\n\n# Count cardinality for hll, build by union of all rows\n# 4 element exists in rows with fk=2 and fk=3. After union it gives single result \nForeignModel.objects.annotate(card=UnionAggCardinality('testmodel__hll_field')).values_list('card', flat=True)\n# outputs [4]\n\n# Count sum of cardinalities for each row\nForeignModel.objects.annotate(card=CardinalitySum('testmodel__hll_field')).values_list('card', flat=True)\n# outputs [5]\n```\n\n### [django-pg-bulk-update](https://github.com/M1hacka/django-pg-bulk-update) integration\nThis library provides a `hll_concat` set function,\nallowing to use hll in `bulk_update` and `bulk_update_or_create` queries.\n```python\n# !!! Don't forget to import function, or django_pg_bulk_update will not find it\nfrom django_pg_hll.bulk_update import HllConcatFunction\n\nMyModel.objects.bulk_update_or_create([\n {'id': 100501, 'hll_field': HllInteger(1)},\n {'id': 100502, 'hll_field': HllInteger(2) | HllInteger(3)}\n ], set_functions={'hll_field': 'hll_concat'}\n)\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/M1hacka/django-pg-hll", "keywords": "", "license": "BSD 3-clause \"New\" or \"Revised\" License", "maintainer": "", "maintainer_email": "", "name": "django-pg-hll", "package_url": "https://pypi.org/project/django-pg-hll/", "platform": "", "project_url": "https://pypi.org/project/django-pg-hll/", "project_urls": { "Homepage": "https://github.com/M1hacka/django-pg-hll" }, "release_url": "https://pypi.org/project/django-pg-hll/1.2.0/", "requires_dist": null, "requires_python": "", "summary": "Provides a django wrapper for postgresql-hll library by CitusData", "version": "1.2.0" }, "last_serial": 5943835, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "72ee2fcd5ac401596f71163036e0164d", "sha256": "39b410b828d5932bc4e856ee4d36c84d3c8aac1ff9afd242441df84688c5e433" }, "downloads": -1, "filename": "django_pg_hll-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "72ee2fcd5ac401596f71163036e0164d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9272, "upload_time": "2019-03-25T17:37:24", "url": "https://files.pythonhosted.org/packages/90/c6/1a74959a737dda29c54451980ea2c159b03247779243ff44d824927bdaa1/django_pg_hll-1.0.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c54f15c4c798672797e87642170b7661", "sha256": "44bfdebf3778f0f3d368dcdeae92b56002dddff296071bb7cb2ec23b4fc62533" }, "downloads": -1, "filename": "django-pg-hll-1.0.0.tar.gz", "has_sig": false, "md5_digest": "c54f15c4c798672797e87642170b7661", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7691, "upload_time": "2019-03-25T17:37:27", "url": "https://files.pythonhosted.org/packages/8e/83/0ba795035f981db2cc4f3f18a46d8ea9d0bfccaca8f566f6ffff11ccaf81/django-pg-hll-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "eaa5f7bfed896aa5207af919afe1da42", "sha256": "838936adadd9066d8695dd3e7554e6fb8d0ddb9ce4bf5271fb252ffc39908379" }, "downloads": -1, "filename": "django_pg_hll-1.0.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "eaa5f7bfed896aa5207af919afe1da42", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9349, "upload_time": "2019-03-31T18:45:33", "url": "https://files.pythonhosted.org/packages/f9/48/a3b51eaeb3f2082d7a0de21ac8230713aa4605a8c48eb71155743885dd14/django_pg_hll-1.0.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0286fe679d747d3f93411babfee69127", "sha256": "b295d5089ce5d85a0cdb505d37376658d60f6181a25a602e31d6aeedc9e8247b" }, "downloads": -1, "filename": "django-pg-hll-1.0.1.tar.gz", "has_sig": false, "md5_digest": "0286fe679d747d3f93411babfee69127", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7718, "upload_time": "2019-03-31T18:45:35", "url": "https://files.pythonhosted.org/packages/ed/ba/17d88fffa970b74ac6dc4e83fd69d95375e67d8cfe08d99566d01da73b75/django-pg-hll-1.0.1.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "12e4a6f798f47a498a3b112e16c02666", "sha256": "5fdd87abd97d6adbda913847fd67e6812bb3e496a4e27478fb7743aef58a1481" }, "downloads": -1, "filename": "django_pg_hll-1.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "12e4a6f798f47a498a3b112e16c02666", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9814, "upload_time": "2019-05-08T09:42:38", "url": "https://files.pythonhosted.org/packages/4e/9a/8e5da5ef5f152089ead6e42dd80ac60514048536d7782d020fa3af96c89e/django_pg_hll-1.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "74c3d05ce917da9a5f48b9cb7486f38f", "sha256": "75e3a25ae333771538fc09fb3f6e9c15cc548895dfc1f295ac8e4aaeae292f8b" }, "downloads": -1, "filename": "django-pg-hll-1.1.0.tar.gz", "has_sig": false, "md5_digest": "74c3d05ce917da9a5f48b9cb7486f38f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8053, "upload_time": "2019-05-08T09:42:40", "url": "https://files.pythonhosted.org/packages/1a/84/99662622eca6ee6e11aa1b93c524948602d1bb8c65679d69f6eb635ec809/django-pg-hll-1.1.0.tar.gz" } ], "1.2.0": [ { "comment_text": "", "digests": { "md5": "375a325d8f8413ea138fe8d37daab182", "sha256": "232d0b5b4ea369e5fc23b0ac7ea5d0f2a738664d11ccd4d88a95efbcda04429a" }, "downloads": -1, "filename": "django_pg_hll-1.2.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "375a325d8f8413ea138fe8d37daab182", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9950, "upload_time": "2019-10-08T09:27:02", "url": "https://files.pythonhosted.org/packages/3a/49/b38de1a4b5e9c74966a9336c6ccb0b62904e34129d097ecdc2c6ad8ea41a/django_pg_hll-1.2.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d05f5db50d8b5ebd500ad6c4a5723e19", "sha256": "a78fa82185aa06058aaf3fefa5cfb063766511c789005a80b89c93c0cb9c6374" }, "downloads": -1, "filename": "django-pg-hll-1.2.0.tar.gz", "has_sig": false, "md5_digest": "d05f5db50d8b5ebd500ad6c4a5723e19", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8078, "upload_time": "2019-10-08T09:27:03", "url": "https://files.pythonhosted.org/packages/ca/65/d098baddb702beefaf92d4095bde5fe223dbfb278e81aaf49497bdffd8fe/django-pg-hll-1.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "375a325d8f8413ea138fe8d37daab182", "sha256": "232d0b5b4ea369e5fc23b0ac7ea5d0f2a738664d11ccd4d88a95efbcda04429a" }, "downloads": -1, "filename": "django_pg_hll-1.2.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "375a325d8f8413ea138fe8d37daab182", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 9950, "upload_time": "2019-10-08T09:27:02", "url": "https://files.pythonhosted.org/packages/3a/49/b38de1a4b5e9c74966a9336c6ccb0b62904e34129d097ecdc2c6ad8ea41a/django_pg_hll-1.2.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "d05f5db50d8b5ebd500ad6c4a5723e19", "sha256": "a78fa82185aa06058aaf3fefa5cfb063766511c789005a80b89c93c0cb9c6374" }, "downloads": -1, "filename": "django-pg-hll-1.2.0.tar.gz", "has_sig": false, "md5_digest": "d05f5db50d8b5ebd500ad6c4a5723e19", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8078, "upload_time": "2019-10-08T09:27:03", "url": "https://files.pythonhosted.org/packages/ca/65/d098baddb702beefaf92d4095bde5fe223dbfb278e81aaf49497bdffd8fe/django-pg-hll-1.2.0.tar.gz" } ] }