{ "info": { "author": "Ben Timby", "author_email": "btimby@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "A `SmartFile`_ Open Source project. `Read more`_ about how SmartFile\nuses and contributes to Open Source software.\n\n.. figure:: http://www.smartfile.com/images/logo.jpg\n :alt: SmartFile\n\nIntroduction\n------------\n\nThis is a simple Django database backend that pools MySQL connections. This\nbackend is based on a blog post by Ed Menendez.\n\nhttp://menendez.com/blog/mysql-connection-pooling-django-and-sqlalchemy/\n\nThe main differences being:\n\n1. The work is done for you.\n2. Instead of cloning the Django mysql backend, we monkey-patch it.\n\nThe second point sounds bad, but it is the best option because it does not\nfreeze the Django MySQL backend at a specific revision. Using this method\nallows us to benefit from any bugs that the Django project fixes, while\nlayering on connection pooling.\n\nThe actual pooling is done using SQLAlchemy. While imperfect (this backend\nis per-process only) it has usefulness. The main problem it solves for us\nis that it restricts a process to a certain number of total connections.\n\nUsage\n-----\n\nConfigure this backend instead of the default Django mysql backend.\n\n::\n\n DATABASES = {\n 'default': {\n 'ENGINE': 'django_mysqlpool.backends.mysqlpool',\n 'NAME': 'db_name',\n 'USER': 'username',\n 'PASSWORD': 'password',\n 'HOST': '',\n 'PORT': '',\n },\n }\n\nConfiguration\n-------------\n\nYou can define the pool implementation and the specific arguments passed to it.\nThe available implementations (backends) and their arguments are defined within\nthe SQLAlchemy documentation.\n\nhttp://docs.sqlalchemy.org/en/rel_0_7/core/pooling.html\n\n * MYSQLPOOL_BACKEND - The pool implementation name ('QueuePool' by default).\n * MYSQLPOOL_ARGUMENTS - The kwargs passed to the pool.\n\nFor example, to use a QueuePool without threadlocal, you could use the following\nconfiguration.\n\n::\n\n MYSQLPOOL_BACKEND = 'QueuePool'\n MYSQLPOOL_ARGUMENTS = {\n 'use_threadlocal': False,\n }\n\nConnection Closing\n------------------\n\nWhile this has nothing to do directly with connection pooling, it is tangentially\nrelated. Once you start pooling (and limiting) the database connections it\nbecomes important to close them.\n\nThis is really only relevant when you are dealing with a threaded application. Such\nwas the case for one of our servers. It would create many threads for handling\nconncurrent operations. Each thread resulted in a connection to the database being\nopened persistently. Once we deployed connection pooling, this service quickly\nexhausted the connection limit of it's pool.\n\nThis sounds like a huge failure, but for us it was a great success. The reason is\nthat we implemented pooling specifically to limit each process to a certain\nnumber of connections. This prevents any given process from impacting other\nservices, turning a global issue into a local issue. Once we were able to identify\nthe specific service that was abusing our MySQL server, we were able to fix it.\n\nThe problem we were having with this threaded server is very well described below.\n\nhttp://stackoverflow.com/questions/1303654/threaded-django-task-doesnt-automatically-handle-transactions-or-db-connections\n\nTherefore, this library provides a decorator that can be used in a similar situation\nto help with connection management. You can use it like so.\n\n::\n\n from django_mysqlpool import auto_close_db\n\n @auto_close_db\n def function_that_uses_db():\n MyModel.objects.all().delete()\n\nWith pooling (and threads), closing the connection early and often is the key to good\nperformance. Closing returns the connection to the pool to be reused, thus the total\nnumber of connections is decreased. We also needed to disable the `use_threadlocal`\noption of the QueuePool, so that multiple threads could share the same connection.\nOnce we decorated all functions that utilized a connection, this service used less\nconnections than it's total thread count.\n\nForking\n-------\n\nIf you are using mysqlpool with a daemon (our project uses Django admin commands to\nbuild daemons) then you need to take care with the connection pool. After a fork()\nthe pool will be unusable. In our case, the file descriptors for the connections\nwere closed, and in the child, any new connections or files assumed the fd of the\nMySQL connection this caused the Django ORM to read/write on some non-MySQL\nconnection in our case, Redis, so Django would send SQL to redis an expect a\nreply! The solution is to close the pool before fork()ing. This will release the\npooled connections which will be reopened when the child first attempts to use\nthem.\n\n::\n\n from django_mysqlpool import close_pool\n\n close_pool()\n pid = os.fork()\n\n\n.. _SmartFile: http://www.smartfile.com/\n.. _Read more: http://www.smartfile.com/open-source.html", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/smartfile/django-mysqlpool/", "keywords": null, "license": "MIT", "maintainer": null, "maintainer_email": null, "name": "django-mysqlpool", "package_url": "https://pypi.org/project/django-mysqlpool/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/django-mysqlpool/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/smartfile/django-mysqlpool/" }, "release_url": "https://pypi.org/project/django-mysqlpool/0.1-9/", "requires_dist": null, "requires_python": null, "summary": "Django database backend for MySQL that provides pooling ala SQLAlchemy.", "version": "0.1-9" }, "last_serial": 802919, "releases": { "0.1-2": [], "0.1-3": [], "0.1-4": [], "0.1-5": [], "0.1-6": [], "0.1-7": [], "0.1-8": [ { "comment_text": "", "digests": { "md5": "7e5587a673bdbb7b66993f2403628210", "sha256": "a5bca27f94df0440a9846375f9c6f857eaa3108d51795a15c05c679f43fc9199" }, "downloads": -1, "filename": "django-mysqlpool-0.1-8.tar.gz", "has_sig": false, "md5_digest": "7e5587a673bdbb7b66993f2403628210", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5451, "upload_time": "2013-03-27T15:26:56", "url": "https://files.pythonhosted.org/packages/ba/f4/4fbec8de345cbb782face9dd0a2544970a120b639827e25ead657d678858/django-mysqlpool-0.1-8.tar.gz" } ], "0.1-9": [ { "comment_text": "", "digests": { "md5": "cbc6d88ea5129b799fe139aebb539d02", "sha256": "4575bd8792e6c919350caead243886077a55c8cce18f10ff3ca6e1d98004aaf8" }, "downloads": -1, "filename": "django-mysqlpool-0.1-9.tar.gz", "has_sig": false, "md5_digest": "cbc6d88ea5129b799fe139aebb539d02", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5473, "upload_time": "2013-06-24T18:21:08", "url": "https://files.pythonhosted.org/packages/7a/80/62e2fb5d24f366585a6a903b3a9585e2e33fbdc3b7aa886ef3a96af5f8e5/django-mysqlpool-0.1-9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "cbc6d88ea5129b799fe139aebb539d02", "sha256": "4575bd8792e6c919350caead243886077a55c8cce18f10ff3ca6e1d98004aaf8" }, "downloads": -1, "filename": "django-mysqlpool-0.1-9.tar.gz", "has_sig": false, "md5_digest": "cbc6d88ea5129b799fe139aebb539d02", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5473, "upload_time": "2013-06-24T18:21:08", "url": "https://files.pythonhosted.org/packages/7a/80/62e2fb5d24f366585a6a903b3a9585e2e33fbdc3b7aa886ef3a96af5f8e5/django-mysqlpool-0.1-9.tar.gz" } ] }