{
    "info": {
        "author": "Jeremy Dunck",
        "author_email": "jdunck@votizen.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Environment :: Web Environment",
            "Framework :: Django",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Topic :: Software Development :: Libraries :: Python Modules"
        ],
        "description": "=====\npindb\n=====\n\npindb is a master/slave router toolkit for Django.  It provides database replica pinning, round-robin read from replicas, the use of unmanaged databases side by side with managed ones, and delegate routers for deciding among sets of replicated DBs.\n\nTL;DR API\n=========\n\n::\n\n    # Try to save without pinning:\n    foo = Model.objects.all()[0]\n    \n    # UnpinnedWriteException raised under strict mode, or master pinning occurs under greedy mode.         \n    foo.save()\n    \n    # read only from the master, allow writes.\n    pindb.pin(alias)\n    \n    # Initialize/end the container:\n    pindb.unpin_all()\n    \n    with unpinned_replica(alias):\n       ... # read from replicas despite pinning state\n    # or \n    queryset.using(pindb.get_replica('master-alias'))\n    \n    with master(alias):\n       ... # write to master despite pinning state\n\nJargon\n======\n\nMaster refers to a writable DB.  Replica (or slave) refers to a read-only\nDB whose data comes from master writes.  One or more master/slave database sets can help scale reads and avoid lock contention on the master.  \nTypically all reads go to  replicas until a write occurs -- then all subsequent \nreads also go to the master to avoid inconsistent reads due to replication lag.  \nPinning is time-based and round-trips between web requests via cookies.  See \n\"design notes\" below for more.\n\nInstallation\n============\n\n::\n\n    pip install pindb\n\nTL;DR: \n \n#. Set ``DATABASES_ROUTERS`` to use a pindb router\n#. set ``PINDB_ENABLED`` to False under test\n#. Define DB masters and replica sets.\n#. populate ``DATABASES`` with ``pindb.populate_replicas``.\n#. Add ``PinDbMiddleware`` to your middleware.\n#. Integrate with celery (if needed).\n#. profile for places to explicitly side-step pinning.\n\nMore explicitly:\n\nAdd ``pindb.StrictPinDbRouter`` or ``pindb.GreedyPinDbRouter`` to\n``DATABASE_ROUTERS``.\n\n``StrictPinDbRouter`` requires that pinning be declared before a write is\nattempted. The advantage is that read replicas are used as much as\npossible. The disadvantage is that your code will need many declarations to\nexplicitly allow and opt out of pinning.\n\n``GreedyPinDbRouter`` will pin to a master as soon as a write occurs. The\nadvantage is that most of your code will just work. The disadvantage is\nthat you will use the read replicas less than possible. You also might\nencounter more situations where the state of your DB changes behind your\nback: you might read from a lagged replica, then perform a write (which\npins you to the master) based on that old information.\n\n``PINDB_ENABLED`` can be used to disable pindb under \ntest.  Each TEST_MIRROR'd alias gets its own connection (and hence transaction), \nwhich is problematic under Django's `TestCase`_, where a master write will not \nbe visible under the replica's connection. \n\npindb has an extensive test suite;\ndisabling it under your own test suite is sane/recommended.\n\n.. _`TestCase`: https://docs.djangoproject.com/en/1.4/topics/testing/#testcase\n\nIf you need to manage more than 1 master/replica set, add\n``PINDB_DELEGATE_ROUTERS`` for pindb to defer to on DB set selection. This is\njust another Django `database router`_ which should return a master alias; \npindb will then choose a master or replica as appropriate for the current pinning\nstate of the returned master.\n\n.. _`database router`: https://docs.djangoproject.com/en/1.4/topics/db/multi-db/#database-routers\n\nDefine ``MASTER_DATABASES``, same schema as ``DATABASES``::\n\n    DATABASES = {\n      \"unmanaged\": {\n        # HOST0, etc.\n      }\n    }\n\n    MASTER_DATABASES = {\n      \"default\": {\n        # HOST1, etc.\n      },\n      \"some_other_master\": {\n        ...\n      }\n    }\n\nDefine ``DATABASE_SETS``, which overrides specific settings for replicas::\n\n    DATABASE_SETS = {\n      \"default\": [{HOST:HOST1}, {HOST:HOST2}, ...],\n      \"some_other_master\": [...] # zero or more replicas is fine.\n    }\n\nFinalize ``DATABASES`` with ``pindb.populate_replicas``::\n\n    DATABASES.update(populate_replicas(MASTER_DATABASES, DATABASE_SETS))\n\nAdd ``pindb.middleware.PinDbMiddleware`` to your ``MIDDLEWARE_CLASSES``\nbefore any expected database access.\n\nOptionally, throughout your codebase, if you intend to write, declare it \nas early as possible to avoid inconsistent reads off the related replicas::\n\n    pin(\"default\")\n\nThat will cause all future reads to use the master.\n\nTo use under celery, hook ``celery.signals.task_postrun`` to call\n``pindb.unpin_all``::\n\n    import pindb\n    from celery.signals import task_postrun\n\n    def end_pinning(**kwargs):\n      pindb.unpin_all()\n    task_postrun.connect(end_pinning)\n\nExceptions and avoiding them\n============================\n\nExceptions\n----------\n\n``PinDbConfigError`` may be caused by...\n\n* Your settings not including ``MASTER_DATABASES`` and ``DATABASE_SETS``\n* Your ``MASTER_DATABASES`` not including a \"default\" and populate_replicas\n  being called without passing ``unmanaged_default=True``.\n* Declaring an alias in ``MASTER_DATABASES`` which does not have a related\n  ``DATABASE_SETS`` entry\n\n``UnpinnedWriteException`` may be caused by...\n\n* ``Model.objects.create``, ``Model.save``, ``qs.update``, or ``qs.delete``\n  without previously calling ``pindb.pin`` for the master\n\n  Note that writes to unmanaged aliases (that is, ones unlisted in\n  ``MASTER_DATABASES`` and related ``DATABASE_SETS``) are allowed at any time.\n\nOverriding pinning\n------------------\n\nIf you wish to read from a replica despite having previously pinned the master,\nyou can do so with... ::\n\n    with pindb.unpinned_replica(alias):\n      # code which reads from replicas\n\nIf you wish to write to a master despite not having pinned to it, you can do so\nwith... ::\n\n    with pindb.master(alias):\n      # code which writes to the DB\n\nRequirements and design notes\n=============================\n\nWe have multiple separate masters (not necessarily sharded). Let's call a\ngrouping of a master and its replicas a \"DB set\".\n\nWe would like to have read replicas of these masters, and we would like to read\nfrom replicas as much as possible and we would like all writes to go to the\nmaster of the set. But we would also like reads to be consistent to writers.\n\nWe would like this to be possible for web request cycles but also for units of\nwork like tasks or shell scripts. So we call this unit of work the \"pinning\ncontext\".\n\nWrites to a given master should continue to read from the master to avoid\ninconsistency in the replication lag window, so there will be an API for\ndeclaring that. Declaring (or otherwise preferring) that a set master is needed\nis \"pinning\" and the group of pins for all DB sets is called the \"pinned set\".\n\nCode which plans on writing (or needs the very lastest data) should be able to\ndeclare that as early as possible to get a fully-consistent view from the\nmaster(s).\n\nIt should be a clear error if we've made a mistake in pinning (that is, writing\nafter reading from a set). The issue here is that if we allow reads (not\nknowing that a write is coming) that gives us an inconsistency window. For\nexample, a process reads from replica, gets a PK that has been deleted in\nmaster, writes to master, fails. Or gets a PK that's been mutated in master so\nthat it shouldn't have been processed, etc.\n\nCode which needs to write without pinning the whole container (e.g. a logging\ntable) should be able to side-step the pinning.\n\nWe should be able to manage the DB sets in settings with minimal repetition,\nand it should compose well with multiple settings files.\n       \nApproach\n--------\n\nWe use a threadlocal to hold the pinned set.\n\nThe database router will then respect pinned set.\n\nThe ``DATABASES`` dict in settings is \"final\" in the sense that it isn't\nstructured with any master/replica semantics. So we use an intermediate setting\nfor defining sets::\n\n    MASTER_DATABASES = {\n      'master-alias': { 'HOST':\"a\", ...normal settings },\n       ...\n    }\n    \n    DATABASE_SETS = { \n      'master-alias': [{'HOST':'someotherhost',...},], \n       # override some of the master settings\n    }\n\nAnd replica config can be finalized... ::\n\n    DATABASES = DATABASES.update(populate_replicas(MASTER_DATABASES, DATABASE_SETS))\n\n...resulting in something like... ::\n\n    DATABASES = {\n      'master-alias': { 'HOST':\"a\", ...normal settings },\n      'master-alias-1': { 'HOST':\"someotherhost\", ...merged settings,\n                          TEST_MIRROR='master-alias' },\n      ...}\n\n      \nIf no master is named \"default\", then the master of your first DB set will also\nbe aliased to \"default\". You should use\n``django.utils.datastructures.SortedDict`` to make that stable.\n\nIf you have multiple database sets, you will also want to compose pinning with \nselection of the appropriate set.  For this, there is one additional setting: \n``DATABASE_ROUTER_DELEGATE``. It has the same interface as a normal \n``DATABASE_ROUTER``, but ``db_for_read`` and ``db_for_write`` must return only \nmaster aliases. Then an appropriate master or replica will be chosen for that DB \nset.\n\nMore concretely, suppose you have 2 different masters, and each of them has a read slave.  Your delegate router (as it existed before use of pindb) likely chooses which master based on app semantics.  Keep doing that.  Then pindb's router will select a read slave from the DB set whose master your existing (now delegate) router chose.\n\nThe strict router will throw an error if ``db_for_write`` is called without\ndeclaring that it's OK. The correct approach is to pin the DB you intend to do\nwrites to *before you read* from a replica.\n\nTo explictly prefer a read replica despite pinning, use either... ::\n\n    with pindb.unpinned_replica('master-alias'):\n           ...\n\n...or the ``.using`` method of a queryset.\n\nIf you would like to explicitly use a replica, ``pindb.get_replica()`` will\nreturn a replica alias.\n\nPinning a set lasts the duration of a pinning context: once pinned, you should not\nunpin a DB. If you want to write to a DB without pinning the container, you can\nuse queryset's ``.using`` method, which bypasses ``db_for_write``. Careful with\nthis axe.\n\nTo declare a pin... ::\n\n    pindb.pin('master-alias')\n\nTODO: Use signed cookies if available (dj 1.4+) for web pinning context.\n\nCoverage\n========\n\nTo see coverage of ``pindb``:\n\n    $ PYTHONPATH=.:$PYTHONPATH coverage run setup.py test\n    $ coverage html\n\nExample configuration\n=====================\n\n::\n\n    MASTER_DATABASES = {\n        'default': {\n            'NAME': 'db1',\n            'ENGINE': DB_ENGINE,\n            'USER': '...',\n            'PASSWORD': '...',\n            'HOST': '10.0.1.0',\n            'PORT': 3306,\n            'OPTIONS': DB_OPTIONS\n        },\n        'api': {\n            'NAME': 'db2',\n            'ENGINE': DB_ENGINE,\n            'USER': '...',\n            'PASSWORD': '...',\n            'HOST': '10.0.2.0',\n            'PORT': 3306,\n            'OPTIONS': DB_OPTIONS\n        },\n    }\n    \n    DATABASE_SETS = {\n        \"default\": [{'HOST': '10.0.1.1'},{'HOST': '10.0.1.2'}],\n        \"api\": [{'HOST': '10.0.2.1'}]\n    }\n    \n    DATABASES = {...}\n    \n    DATABASES.update(pindb.populate_replicas(MASTER_DATABASES, DATABASE_SETS))\n    \n\n    PINDB_DELEGATE_ROUTERS = [\"myapp.router.Router\"]\n    DATABASE_ROUTERS = ['pindb.GreedyPinDbRouter']\n\n    # default values which you can override:\n    PINDB_ENABLED = True\n    PINDB_PINNING_COOKIE = 'pindb_pinned_set'\n    PINDB_PINNING_SECONDS = 15",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/votizen/django-pindb",
        "keywords": "django,multidb,router",
        "license": "MIT",
        "maintainer": null,
        "maintainer_email": null,
        "name": "django-pindb",
        "package_url": "https://pypi.org/project/django-pindb/",
        "platform": "any",
        "project_url": "https://pypi.org/project/django-pindb/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/votizen/django-pindb"
        },
        "release_url": "https://pypi.org/project/django-pindb/0.1.12/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Manages master/replica pinning for django",
        "version": "0.1.12"
    },
    "last_serial": 1377430,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ad167abb3a4d6b60a610bdd4e99214d9",
                    "sha256": "2e68e7bebb81da04fd71332d365ba641cbe2007a56b97d43b8524c6bce7ebeef"
                },
                "downloads": -1,
                "filename": "django-pindb-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ad167abb3a4d6b60a610bdd4e99214d9",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10701,
                "upload_time": "2012-02-22T21:00:47",
                "url": "https://files.pythonhosted.org/packages/dd/55/7ad8be615ea801128a7ef4e6144d615eb4a1ab5e3d9dfa703a2dd6c2a4a5/django-pindb-0.1.0.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b37182cac21481a0105ddafe9669142e",
                    "sha256": "8aebb68761971cd9fc8ac21ed7e3eee8c3fa0f58fbfda085fc9303934a2e2bb9"
                },
                "downloads": -1,
                "filename": "django-pindb-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "b37182cac21481a0105ddafe9669142e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10774,
                "upload_time": "2012-02-23T00:03:11",
                "url": "https://files.pythonhosted.org/packages/14/e0/84aaab73cca53eb42b792dee8f69ca8c370e155452be16921287a470ec13/django-pindb-0.1.1.tar.gz"
            }
        ],
        "0.1.12": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "743af728cf610a8d52a8790bc4a6fef8",
                    "sha256": "e255c5241ff293c9078b4661352f2c3418aa66b70d914a280b8efb3fa2b3bbb6"
                },
                "downloads": -1,
                "filename": "django-pindb-0.1.12.tar.gz",
                "has_sig": false,
                "md5_digest": "743af728cf610a8d52a8790bc4a6fef8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 20523,
                "upload_time": "2015-01-10T02:14:37",
                "url": "https://files.pythonhosted.org/packages/3e/25/359d719840c2c62ee6676d12103cff005d5ae0a326e344f0bd7dc0adb008/django-pindb-0.1.12.tar.gz"
            }
        ],
        "0.1.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "020e383b341e676f17792625f0937aa2",
                    "sha256": "9aa4c44ac7de6664cf8b83b2361143f165bcfbd088c271af86d586e6855027f4"
                },
                "downloads": -1,
                "filename": "django-pindb-0.1.4.tar.gz",
                "has_sig": false,
                "md5_digest": "020e383b341e676f17792625f0937aa2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11360,
                "upload_time": "2012-02-23T06:02:09",
                "url": "https://files.pythonhosted.org/packages/7f/43/114023391677766f3262315da330636ef085a648c4347cb90b7f0b8e5f25/django-pindb-0.1.4.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "68c4eef21967aaea18194fde4e62de2d",
                    "sha256": "78a5e6da75c1d4472fb298564c990a9ed4dcc8198c464027819035cc3089c0ec"
                },
                "downloads": -1,
                "filename": "django-pindb-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "68c4eef21967aaea18194fde4e62de2d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12072,
                "upload_time": "2012-02-23T12:56:11",
                "url": "https://files.pythonhosted.org/packages/21/16/9a2d0b06f7d0ca0fcd4d7ebe9e7f96387f1293ce2e5ebc3113997a18f625/django-pindb-0.1.5.tar.gz"
            }
        ],
        "0.1.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2cdf03d11db4f8f798b723e72a270b18",
                    "sha256": "cda1ea0267da5fff14d6a857333bcc47105b4b3b3abc62f7aa2713712daa577a"
                },
                "downloads": -1,
                "filename": "django-pindb-0.1.6.tar.gz",
                "has_sig": false,
                "md5_digest": "2cdf03d11db4f8f798b723e72a270b18",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12147,
                "upload_time": "2012-02-23T14:08:07",
                "url": "https://files.pythonhosted.org/packages/7e/77/3d3f416e10bb772835408ba703ffc77aaf8387a4e3fc726e9d152c823586/django-pindb-0.1.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "743af728cf610a8d52a8790bc4a6fef8",
                "sha256": "e255c5241ff293c9078b4661352f2c3418aa66b70d914a280b8efb3fa2b3bbb6"
            },
            "downloads": -1,
            "filename": "django-pindb-0.1.12.tar.gz",
            "has_sig": false,
            "md5_digest": "743af728cf610a8d52a8790bc4a6fef8",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 20523,
            "upload_time": "2015-01-10T02:14:37",
            "url": "https://files.pythonhosted.org/packages/3e/25/359d719840c2c62ee6676d12103cff005d5ae0a326e344f0bd7dc0adb008/django-pindb-0.1.12.tar.gz"
        }
    ]
}