{
    "info": {
        "author": "Tumbler",
        "author_email": "zimbler@bk.ru",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "Django Prepared Queries\n=======================\n\n[![PyPI version](https://badge.fury.io/py/django_prepared_queries.svg)](https://badge.fury.io/py/django_prepared_queries)\n[![codecov](https://codecov.io/gh/rutube/django_prepared_queries/branch/master/graph/badge.svg)](https://codecov.io/gh/rutube/django_prepared_queries)\n[![Build Status](https://travis-ci.org/rutube/django_prepared_queries.svg)](https://travis-ci.org/rutube/django_prepared_queries)\n\n`django_pq` allows to cache SQL generated with Django ORM and reuse cached \nqueries with only substituting new parameters values.\n\nShort example\n-------------\n\nSome developers think that Django ORM is slow. It is true if your code looks \nlike this:\n\n```python\nfrom django.db import models\nfrom countries_field.fields import countries_isnull, countries_contains\n\n    \ndef filter_queryset(self, domains=None, **kwargs):\n    query = models.Q()\n    if domains:\n        query &= ((models.Q(allow_domains__name__in=domains) |\n                   models.Q(allow_domains__isnull=True)) &\n                  (~models.Q(deny_domains__name__in=domains) |\n                   models.Q(deny_domains__isnull=True)))\n    else:\n        query &= (models.Q(allow_domains__isnull=True) &\n                  models.Q(deny_domains__isnull=True))\n    user_agent = kwargs.pop('user_agent', None)\n    if user_agent:\n        query &= (models.Q(user_agents=user_agent) |\n                  models.Q(user_agents__isnull=True))\n    else:\n        query &= models.Q(user_agents__isnull=True)\n\n    country = kwargs.pop('country')\n    if country:\n        query &= countries_isnull() | countries_contains([country])\n    else:\n        query &= countries_isnull()\n\n    return self.get_queryset().filter(query)\n```\n\nGenerated SQL query is quite long and in our case takes up to 50% of HTTP \nrequest handling. What if we could cache generated SQL and just substitute\nactual parameters values instead of repeating heavy queryset filtering?\n\nWell, with `django_pq` you can do following.\n\n```python\n\nfrom django.db import models\nimport django_pq\n\n\n# Add caching decorator for heavy queryset constructing method\n@django_pq.substitute_lazy()\ndef filter_queryset_lazy(self, domains=None, **kwargs):\n    query = models.Q()\n    \n    # branches in decorated function must check real value instead of Lazy \n    # wrapper, because actual value this time could be False.\n    if django_pq.reveal(domains):\n        # You pass Lazy wrappers in to any lookup parameters for queryset,\n        # and these Lazy wrappers remain lazy until it's time to query the \n        # database.\n        query &= ((models.Q(allow_domains__name__in=domains) |\n                   models.Q(allow_domains__isnull=True)) &\n                  (~models.Q(deny_domains__name__in=domains) |\n                   models.Q(deny_domains__isnull=True)))\n    else:\n        query &= (models.Q(allow_domains__isnull=True) &\n                  models.Q(deny_domains__isnull=True))\n                \n    # ... \n    # \n    # modify other parts of queryset constuction with respect of lazy nature of\n    # arguments.\n    \n    return self.get_queryset().filter(query)\n        \ndef filter_queryset(self, **kwargs):\n    # wrap parameters into context manager so Lazy wrappers could get actual\n    # values when they need.\n    with django_pq.LazyContext(**kwargs):\n        queryset = self.filter_queryset_lazy(**kwargs)\n        # queryset is now RawQuerySet with Lazy wrappers in params.\n        \n        # database queries should be performed within LazyContext.\n        return queryset.first()\n        \n```\n\nThat's it - your queryset generation code is cached.\n\nRules for preparing code for caching\n------------------------------------\n\n1. Don't check Lazy wrappers for anything - use `reveal()` to check actual \nparameter values. I.e. `Lazy(None) is not None` is always true (this it not \nwhat you meant really).\n2. Don't pass Model instances as parameters. This allows Model instance method \ncalls and may lead to implicit branching that could not be detected from actual \nparameters list. Instead, pass primary key values.\n3. Don't query DB within cached method - branching could not be detected.\n4. Add all `if` expressions as new parameters to your method - it would be \nusefull for proper caching.\n5. Don't pass empty lists as parameter values. Django ORM checks it for \nemptiness and removes empty lookups from WhereNode (with respect of boolean \nalgebra rules). Pass `None` instead.\n6. Don't use any volatile values like `datetime.now() `in queryset filtering;\npass it as a parameter instead.\n7. Test your code with 100% branch coverage *before* adding caching.\n\nNormalize parameters\n--------------------\n\nTo help you to normalize parameters passed into cached function `LazyContext` \nmay call a list of callables and return normalized parameter values when \nentering context.\n\n```python\nfrom django.db import models\nfrom django_pq import LazyContext\n\n\ndef model_to_pk(kwargs):\n    for k, v in list(kwargs.items()):\n        if isinstance(v, models.Model):\n            kwargs[k] = v.pk  # Model -> Model.pk\n    return kwargs \n    \ndef empty_list_to_none(kwargs):\n    for k, v in list(kwargs.items()):\n        if isinstance(v, list) and not v:\n            kwargs[k] = None  # [] -> None\n    return kwargs\n\n\ndef filter_queryset(self, **kwargs):    \n    with LazyContext(model_to_pk, empty_list_to_none, **kwargs) as lazy_kwargs:\n        queryset = self.filter_queryset_lazy(**lazy_kwargs)\n        return queryset.first()\n\n```\n\nHow it works\n------------\n\n1. First, `substitute_lazy()` decorator wraps all parameters with Lazy wrapper,\nand with wrapper remains \"lazy\" until SQL generation is completed.\n2. Your code is called twice, with lazy wrappers as arguments and with actual \nvalues, to ensure that lazy result is identical to native queryset.\n3. If SQL and normalized parameters match, a `RawQuerySet` instance is cached\nwith Lazy wrappers as parameters.\n4. Cache key respects presence of any argument and certain constants like \n`True, False, 0, 1, None`.\n5. In \"cache hit\" situation new actual parameters values are substituted from \nLazyContext into `RawQuerySet.params` and that's result of caching.\n6. If you are doing it right, `RawQuerySet` will act almost like normal \n`QuerySet`, or (more correctly) as your Model instances iterator.",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/rutube/django_prepared_queries",
        "keywords": "",
        "license": "Beer license",
        "maintainer": "",
        "maintainer_email": "",
        "name": "django-prepared-queries",
        "package_url": "https://pypi.org/project/django-prepared-queries/",
        "platform": "",
        "project_url": "https://pypi.org/project/django-prepared-queries/",
        "project_urls": {
            "Homepage": "https://github.com/rutube/django_prepared_queries"
        },
        "release_url": "https://pypi.org/project/django-prepared-queries/0.3.1/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Caches SQL queries built with Django ORM",
        "version": "0.3.1"
    },
    "last_serial": 4688151,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9d6474860111b2794acf7b821f3a241f",
                    "sha256": "5643223c868da342763ff94e2e42fdfcd760df63608d5f4bebeafeabfe961abf"
                },
                "downloads": -1,
                "filename": "django_prepared_queries-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "9d6474860111b2794acf7b821f3a241f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8482,
                "upload_time": "2017-09-26T14:44:07",
                "url": "https://files.pythonhosted.org/packages/36/86/5461df018abb772df0b2ec992660ed91cbd9b574fcd233cb81700dbb3603/django_prepared_queries-0.1.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7ea57ee2098718b495e8dfbd9699dda0",
                    "sha256": "c3d5be7a8e568ef80d56a09dccbb5ab7f8e02cea57f364006da2671be3c3772e"
                },
                "downloads": -1,
                "filename": "django_prepared_queries-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "7ea57ee2098718b495e8dfbd9699dda0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9179,
                "upload_time": "2017-10-05T11:10:58",
                "url": "https://files.pythonhosted.org/packages/d5/af/4c546f13d5b578f1b1ee038b9f6b4e5c1a4fe31d5d7fdf75d7feeab9ae0a/django_prepared_queries-0.2.1.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8f8040d5e98d188c3b82821447edc22c",
                    "sha256": "b6872d617955c3aeb7d3895ef69ad5af2b4528e4b531f97b389f63225f902b1d"
                },
                "downloads": -1,
                "filename": "django_prepared_queries-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8f8040d5e98d188c3b82821447edc22c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8935,
                "upload_time": "2019-01-12T09:46:11",
                "url": "https://files.pythonhosted.org/packages/b9/17/d84c068dfc31435a4e161f34c92f1ad83607ef925279740a84a9a120b90b/django_prepared_queries-0.3.0.tar.gz"
            }
        ],
        "0.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "db4a469b6f37bfa9d9777e61f0e26fe4",
                    "sha256": "a618df03e26f58676b934155dfa9589614db6ca4aa7e5b95415556f8d7f3c26b"
                },
                "downloads": -1,
                "filename": "django_prepared_queries-0.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "db4a469b6f37bfa9d9777e61f0e26fe4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8877,
                "upload_time": "2019-01-12T09:51:02",
                "url": "https://files.pythonhosted.org/packages/1f/30/e620509232fcc4df664a14fafef2bdc34234dded5cc5ecdef2d50f73f710/django_prepared_queries-0.3.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "db4a469b6f37bfa9d9777e61f0e26fe4",
                "sha256": "a618df03e26f58676b934155dfa9589614db6ca4aa7e5b95415556f8d7f3c26b"
            },
            "downloads": -1,
            "filename": "django_prepared_queries-0.3.1.tar.gz",
            "has_sig": false,
            "md5_digest": "db4a469b6f37bfa9d9777e61f0e26fe4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8877,
            "upload_time": "2019-01-12T09:51:02",
            "url": "https://files.pythonhosted.org/packages/1f/30/e620509232fcc4df664a14fafef2bdc34234dded5cc5ecdef2d50f73f710/django_prepared_queries-0.3.1.tar.gz"
        }
    ]
}