{ "info": { "author": "Operator Inc", "author_email": "chanind@operator.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3", "Topic :: Database :: Front-Ends" ], "description": "SQLAlchemy Bulk Lazy Loader\n===========================\n\nA custom lazy loader for SQLAlchemy relations which ensures relations are always loaded efficiently. This loader automatically solves the `n+1 query problem `_ without needing to manually add ``joinedload`` or ``subqueryload`` statements to all your queries.\n\nThe Problem\n-----------\n\nThe n + 1 query problem arises whenever relationships are lazy-loaded in a loop. For example:\n\n.. code:: python\n\n students = session.query(Student).limit(100).all()\n for student in students:\n print('{} studies at {}'.format(student.name, student.school.name))\n\nIn the above code 101 SQL queries will be generated - one to load the list of students and then 100 individual queries for each student to load that student's school. The statements look like:\n\n.. code:: sql\n\n SELECT * FROM students LIMIT 100;\n SELECT * FROM schools WHERE schools.student_id = 1 LIMIT 1;\n SELECT * FROM schools WHERE schools.student_id = 2 LIMIT 1;\n SELECT * FROM schools WHERE schools.student_id = 3 LIMIT 1;\n SELECT * FROM schools WHERE schools.student_id = 4 LIMIT 1;\n ...\n\nThis is bad.\n\n\nThe traditional way to solve this with SQLAlchemy is to add a ``joinedload`` or ``subqueryload`` to the initial query to include the schools along with the students, like below:\n\n.. code:: python\n\n students = (\n session.query(Student)\n .options(subqueryload(Student.school))\n .limit(100)\n .all()\n )\n for student in students:\n print('{} studies at {}'.format(student.name, student.school.name))\n\nWhile this works, it needs to be added to every query that's performed, and when there's a lot of related models being added you can easily end up with a massive list of ``subqueryload`` and ``joinedload``. If you forget even one anywhere then you're silently back to the n + 1 query problem. Also, if you stop needing a relation later you need to remember to remove it from the original query or else you're now loading too much data. Furthermore, it's just a huge pain to have to maintain these lists of related models throughout your code everywhere there's a database query.\n\nWouldn't it be great if you didn't have to worry about adding ``subqueryload`` and ``joinedload`` and yet also be guaranteed that all your relations are loading efficiently?\n\nHow The Bulk Lazy Loader Works\n------------------------------\n\n99% of the time, if there is a list of models loaded in memory and a relation on one of them is lazy-loaded then you're in a loop and the same relationship is going to be requested on every other model. SQLAlchemy Bulk Lazy Loader assumes this is the case and whenever a relation on a model is lazy-loaded, it will look through the current session for any other similar models that need that same relation loaded and will issue a single, bulk SQL statement to load them all at once.\n\nThis means you can load all the relations you want in loops while being guaranted that all relations are loaded performantly, and only the relations that are used are loaded. For example, here's the same code from above:\n\n.. code:: python\n\n students = session.query(Student).limit(100).all()\n for student in students:\n print('{} studies at {}'.format(student.name, student.school.name))\n\nThe Bulk Lazy Loader will issue only 2 SQL statements, the same as if you had specified ``subqueryload`` on the initial query, except that now your code is a lot cleaner and you're guaranteed to be loading just the relations you need. Yay!\n\nInstallation\n------------\n\nSQLAlchemy Bulk Lazy Loader can be installed via pip\n\n``pip install SQLAlchemy-bulk-lazy-loader``\n\nUsage\n-----\n\nBefore you declare your SQLAlchemy mappings you need to run the following:\n\n.. code:: python\n\n from sqlalchemy_bulk_lazy_loader import BulkLazyLoader\n BulkLazyLoader.register_loader()\n\n\nThis registers the loader with sqlalchemy and makes it available on your relations by specifying ``lazy='bulk'`` in your relation mappings. For example:\n\n.. code:: python\n\n class Student(db.model):\n id = db.Column(db.Integer, primary_key=True)\n school_id = db.Column(db.Integer, db.ForeignKey('school.id'))\n\n class School(db.model):\n id = db.Column(db.Integer, primary_key=True)\n students = db.relationship('Student', lazy='bulk', backref=db.backref('school', lazy='bulk'))\n\nAnd that's it! The bulk lazy loader will be used for ``student.school`` and ``school.students`` relations.\n\nLimitations\n-----------\n\nCurrently only relations on a single primary key or a simple secondary join are supported.\n\n.. code:: python\n\n students = relationship('Student', lazy='bulk') # OK!\n students = relationship('Student', lazy='bulk', order_by=Student.id) # OK!\n student = relationship('Student', lazy='bulk', uselist=False) # OK!\n students = relationship('Student', lazy='bulk', secondary=school_to_students) # OK!\n students = relationship('Student', lazy='bulk', secondary=school_to_students, primaryjoin='and_(...)') # NOT SUPPORTED\n\nPython 2 is not supported.\n\nBut I have this one case where I want to load the relations differently!\n------------------------------------------------------------------------\n\nIf you want to load relations in the query still using ``subqueryload`` or ``joinedload`` you can still do that - the bulk lazy loader will only kick in when it's asked for a relation on a model that isn't already loaded. If you really need fine-grained control of relation loading in a specific case you can also use ``attributes.set_committed_value(model, , )`` to explicitly set related models. In fact this is how ``BulkLazyLoader`` works behind the scenes.\n\nContributing\n------------\n\nContributions are welcome! Create a pull request and make sure to add test coverage. Tests use the SQLAlchemy test framework and can be run with ``py.test``. \n\nHappy loading!\n\n\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/operator/sqlalchemy_bulk_lazy_loader", "keywords": "sqlalchemy orm lazyload joinedload subqueryload", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "sqlalchemy-bulk-lazy-loader", "package_url": "https://pypi.org/project/sqlalchemy-bulk-lazy-loader/", "platform": "", "project_url": "https://pypi.org/project/sqlalchemy-bulk-lazy-loader/", "project_urls": { "Homepage": "https://github.com/operator/sqlalchemy_bulk_lazy_loader" }, "release_url": "https://pypi.org/project/sqlalchemy-bulk-lazy-loader/0.9.9/", "requires_dist": [ "SQLAlchemy (>=1.0)" ], "requires_python": "", "summary": "A Bulk Lazy Loader for Sqlalchemy that solves the n + 1 loading problem", "version": "0.9.9" }, "last_serial": 3446808, "releases": { "0.9.4": [ { "comment_text": "", "digests": { "md5": "b9e0eae575eccf64c7244ac0fabb1296", "sha256": "2de3535e6f4278c05453612098761e1bb8a1c736a738b9899bdc2fa0d3e97490" }, "downloads": -1, "filename": "sqlalchemy_bulk_lazy_loader-0.9.4-py3-none-any.whl", "has_sig": false, "md5_digest": "b9e0eae575eccf64c7244ac0fabb1296", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9807, "upload_time": "2017-05-03T09:13:57", "url": "https://files.pythonhosted.org/packages/27/2b/b0bdd8e27bf90b9bba56fadbff61f16c36b2ab92fae7b85385c4168c5ada/sqlalchemy_bulk_lazy_loader-0.9.4-py3-none-any.whl" } ], "0.9.5": [ { "comment_text": "", "digests": { "md5": "cc06d5ed9ae9b2e3f272973c19fd527b", "sha256": "dec866bb97f0f278f9bc95208534ccc15a8aa565294a30cd155baab3e9a1d27b" }, "downloads": -1, "filename": "SQLAlchemy_bulk_lazy_loader-0.9.5-py3-none-any.whl", "has_sig": false, "md5_digest": "cc06d5ed9ae9b2e3f272973c19fd527b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9836, "upload_time": "2017-05-03T09:22:21", "url": "https://files.pythonhosted.org/packages/86/9d/6a485a390a6a29d42140c690a0d84398e75fa2dd01f8ed14be4eab84145a/SQLAlchemy_bulk_lazy_loader-0.9.5-py3-none-any.whl" } ], "0.9.6": [ { "comment_text": "", "digests": { "md5": "d59d539fb009ea69bfa461d16e57803a", "sha256": "3047d4bb0fcb56aeb7fb3595c54049ce91d3438404bfa84a9a7a774759f941bd" }, "downloads": -1, "filename": "SQLAlchemy_bulk_lazy_loader-0.9.6-py3-none-any.whl", "has_sig": false, "md5_digest": "d59d539fb009ea69bfa461d16e57803a", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9876, "upload_time": "2017-05-03T12:51:14", "url": "https://files.pythonhosted.org/packages/37/24/e57d5abc6fa36ea744915c524575a2eb1a15dbb7035b19d4eb88d9f94ad9/SQLAlchemy_bulk_lazy_loader-0.9.6-py3-none-any.whl" } ], "0.9.7": [ { "comment_text": "", "digests": { "md5": "207f4c18ec369ad87bedd20322f7e10b", "sha256": "768ea955e3990755d5e483817d87421b2edc02f9fbaa4ab0e608865d19a69c22" }, "downloads": -1, "filename": "SQLAlchemy_bulk_lazy_loader-0.9.7-py3-none-any.whl", "has_sig": false, "md5_digest": "207f4c18ec369ad87bedd20322f7e10b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9807, "upload_time": "2017-05-05T03:11:17", "url": "https://files.pythonhosted.org/packages/e6/54/c8a11d29063246e0be2ddf6d8d58304a621ce8f1209b46baf70ba8adffcc/SQLAlchemy_bulk_lazy_loader-0.9.7-py3-none-any.whl" } ], "0.9.8": [ { "comment_text": "", "digests": { "md5": "3096cd92f0153eff81cec53c21c80257", "sha256": "28cb0e3b94b78143f0b84406b1bdd41530e1f7195ab0c269456197aa996455aa" }, "downloads": -1, "filename": "SQLAlchemy_bulk_lazy_loader-0.9.8-py3-none-any.whl", "has_sig": false, "md5_digest": "3096cd92f0153eff81cec53c21c80257", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9785, "upload_time": "2017-11-09T05:36:16", "url": "https://files.pythonhosted.org/packages/ce/24/34728943102c8325d8325d14e9a5be02bb0ec2563e2a882c3a0543963034/SQLAlchemy_bulk_lazy_loader-0.9.8-py3-none-any.whl" } ], "0.9.9": [ { "comment_text": "", "digests": { "md5": "5464b4e915313b66b6033b54890ce547", "sha256": "faf617ff54ed779f7fa399a43451c6df9e07a2b20935d4a4145f67019392331e" }, "downloads": -1, "filename": "SQLAlchemy_bulk_lazy_loader-0.9.9-py3-none-any.whl", "has_sig": false, "md5_digest": "5464b4e915313b66b6033b54890ce547", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9786, "upload_time": "2017-12-28T06:23:09", "url": "https://files.pythonhosted.org/packages/c8/78/9f06d212af677e5d8733bdeeb3362834ea38d40e0c0208f3682508862df3/SQLAlchemy_bulk_lazy_loader-0.9.9-py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5464b4e915313b66b6033b54890ce547", "sha256": "faf617ff54ed779f7fa399a43451c6df9e07a2b20935d4a4145f67019392331e" }, "downloads": -1, "filename": "SQLAlchemy_bulk_lazy_loader-0.9.9-py3-none-any.whl", "has_sig": false, "md5_digest": "5464b4e915313b66b6033b54890ce547", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9786, "upload_time": "2017-12-28T06:23:09", "url": "https://files.pythonhosted.org/packages/c8/78/9f06d212af677e5d8733bdeeb3362834ea38d40e0c0208f3682508862df3/SQLAlchemy_bulk_lazy_loader-0.9.9-py3-none-any.whl" } ] }