{ "info": { "author": "Gabriel Bordeaux", "author_email": "pypi@gab.lc", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: MacOS", "Operating System :: POSIX :: Linux", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Topic :: Database", "Topic :: Database :: Database Engines/Servers", "Topic :: Software Development" ], "description": "# pg-batch\n\n[![Build Status](https://travis-ci.org/gabfl/pg-batch.svg?branch=master)](https://travis-ci.org/gabfl/pg-batch)\n\nPostgreSQL equivalent of https://github.com/gabfl/mysql-batch.\n\nUpdating or deleting a large amount of rows in PostgreSQL will create locks that will paralyze other queries running in parallel.\n\nThis tool will run UPDATE and DELETE queries in small batches to limit locking. If a large number of rows has to be updated or deleted, it is also possible to limit the number of rows selected at once.\n\n## Installation\n\n```\npip3 install pg_batch\n```\n\n## UPDATE example\n\nYou can run this example with the schema available in [sample_table/schema.sql](sample_table/schema.sql)\n\nThe following example will be identical to the following update:\n\n```sql\nUPDATE batch_test SET date = NOW() WHERE number > 30 AND date is NULL;\n```\n\nThis is the equivalent to process this update with batches of 20 rows:\n\n```bash\npg_batch --host localhost \\\n --user postgres \\\n --password secret_password \\\n --database \"test\" \\\n --table \"batch_test\" \\\n --write_batch_size 20 \\\n --where \"number > 30 AND date IS NULL\" \\\n --set \"date = NOW()\"\n```\n\nOutput sample:\n\n```bash\n* Selecting data...\n query: SELECT id as id FROM batch_test WHERE number > 30 AND date IS NULL AND id > 0 ORDER BY id LIMIT 10000\n* Preparing to modify 70 rows...\n* Updating 20 rows...\n query: UPDATE batch_test SET date = NOW() WHERE id IN (31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50)\n* Start updating? [Y/n]\n* Updating 20 rows...\n query: UPDATE batch_test SET date = NOW() WHERE id IN (51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70)\n* Updating 20 rows...\n query: UPDATE batch_test SET date = NOW() WHERE id IN (71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90)\n* Updating 10 rows...\n query: UPDATE batch_test SET date = NOW() WHERE id IN (91, 92, 93, 94, 95, 96, 97, 98, 99, 100)\n* Selecting data...\n query: SELECT id as id FROM batch_test WHERE number > 30 AND date IS NULL AND id > 100 ORDER BY id LIMIT 10000\n* No more rows to modify!\n* Program exited\n```\n\n## DELETE example\n\nThe following example will be identical to the following delete:\n\n```sql\nDELETE FROM batch_test WHERE number > 30 AND date is NULL;\n```\n\nThis is the equivalent to process this delete with batches of 20 rows:\n\n```bash\npg_batch --host localhost \\\n --user postgres \\\n --password secret_password \\\n --database \"test\" \\\n --table \"batch_test\" \\\n --write_batch_size 20 \\\n --where \"number > 30 AND date IS NULL\" \\\n --action \"delete\"\n```\n\nOutput sample:\n\n```bash\n* Selecting data...\n query: SELECT id as id FROM batch_test WHERE number > 30 AND date IS NULL AND id > 0 ORDER BY id LIMIT 10000\n* Preparing to modify 70 rows...\n* Deleting 20 rows...\n query: DELETE FROM batch_test WHERE id IN (31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50)\n* Start deleting? [Y/n]\n* Deleting 20 rows...\n query: DELETE FROM batch_test WHERE id IN (51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70)\n* Deleting 20 rows...\n query: DELETE FROM batch_test WHERE id IN (71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90)\n* Deleting 10 rows...\n query: DELETE FROM batch_test WHERE id IN (91, 92, 93, 94, 95, 96, 97, 98, 99, 100)\n* Selecting data...\n query: SELECT id as id FROM batch_test WHERE number > 30 AND date IS NULL AND id > 100 ORDER BY id LIMIT 10000\n* No more rows to modify!\n* Program exited\n```\n\n## Usage\n\n```bash\nusage: pg_batch [-h] [-H HOST] [-P PORT] -U USER [-p PASSWORD] -d DATABASE -t\n TABLE [-id PRIMARY_KEY] -w WHERE [-s SET]\n [-rbz READ_BATCH_SIZE] [-wbz WRITE_BATCH_SIZE] [-S SLEEP]\n [-a {update,delete}] [-n]\n\noptional arguments:\n -h, --help show this help message and exit\n -H HOST, --host HOST PostgreSQL server host\n -P PORT, --port PORT PostgreSQL server port\n -U USER, --user USER PostgreSQL user\n -p PASSWORD, --password PASSWORD\n PostgreSQL password\n -d DATABASE, --database DATABASE\n PostgreSQL database name\n -t TABLE, --table TABLE\n PostgreSQL table\n -id PRIMARY_KEY, --primary_key PRIMARY_KEY\n Name of the primary key column\n -w WHERE, --where WHERE\n Select WHERE clause\n -s SET, --set SET Update SET clause\n -rbz READ_BATCH_SIZE, --read_batch_size READ_BATCH_SIZE\n Select batch size\n -wbz WRITE_BATCH_SIZE, --write_batch_size WRITE_BATCH_SIZE\n Update/delete batch size\n -S SLEEP, --sleep SLEEP\n Sleep after each batch\n -a {update,delete}, --action {update,delete}\n Action ('update' or 'delete')\n -n, --no_confirm Don't ask for confirmation before to run the write\n queries\n```\n\n## License\n\nThis program is under MIT license ([view license](LICENSE)).\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/gabfl/pg-batch", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pg-batch", "package_url": "https://pypi.org/project/pg-batch/", "platform": "", "project_url": "https://pypi.org/project/pg-batch/", "project_urls": { "Homepage": "https://github.com/gabfl/pg-batch" }, "release_url": "https://pypi.org/project/pg-batch/1.1/", "requires_dist": [ "psycopg2-binary", "argparse" ], "requires_python": "", "summary": "Run large PostgreSQL UPDATE and DELETE queries with small batches to prevent locks", "version": "1.1" }, "last_serial": 3893373, "releases": { "1.0.3": [ { "comment_text": "", "digests": { "md5": "5c726712cb7f71e47288943e651a6f83", "sha256": "47afcaa768c87961acd5634d9155cea5a4e96eaed53d71782a1e8df175aec234" }, "downloads": -1, "filename": "pg_batch-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "5c726712cb7f71e47288943e651a6f83", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7951, "upload_time": "2017-07-06T14:11:53", "url": "https://files.pythonhosted.org/packages/25/f7/7bbb7ef00d4eb7a73a71cb7224a969c6158454e5cad606bdbe9b265e8a41/pg_batch-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "aad8224a6545a63c8e181dabe063534a", "sha256": "bdbc992cb0cc0fa0bb178d3017c4a54f991f6f1213b26db9cf89beb4b3583557" }, "downloads": -1, "filename": "pg_batch-1.0.3.tar.gz", "has_sig": false, "md5_digest": "aad8224a6545a63c8e181dabe063534a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4865, "upload_time": "2017-07-06T14:11:54", "url": "https://files.pythonhosted.org/packages/b8/3c/e0170daa90cd708f7358acb6054d9d60ba2cf8f16a0f1df34e8616240b79/pg_batch-1.0.3.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "a25b2f681972449cde52e1a1e3045ed7", "sha256": "d2798c6d32a7260821cee3bdcc4f8159963f4afabd9a4bced40825ebbef52b01" }, "downloads": -1, "filename": "pg_batch-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a25b2f681972449cde52e1a1e3045ed7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7942, "upload_time": "2017-07-10T01:38:56", "url": "https://files.pythonhosted.org/packages/1b/66/0bb71d75fc641acb641a11575853824c325719ab5f1121c2b46a9ab1cfe7/pg_batch-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1a5a508f651b9409ff4dcb34964bad47", "sha256": "53566efece3fa36e5f4ca2041863613aee32225be09552fc4d837871642a1eab" }, "downloads": -1, "filename": "pg_batch-1.0.4.tar.gz", "has_sig": false, "md5_digest": "1a5a508f651b9409ff4dcb34964bad47", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4871, "upload_time": "2017-07-10T01:38:57", "url": "https://files.pythonhosted.org/packages/4b/6e/a6610fe1a25b15cda018d40295e8f8ab1bd5dce6fdee3d32646851052fc2/pg_batch-1.0.4.tar.gz" } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "6e26c3fa6e1fa5f88afd659f240f8121", "sha256": "f098549ab8b8be2ee0c954298970ce8d85bc1abcf39d6656a7da72388945b046" }, "downloads": -1, "filename": "pg_batch-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "6e26c3fa6e1fa5f88afd659f240f8121", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7941, "upload_time": "2017-07-12T15:48:48", "url": "https://files.pythonhosted.org/packages/cf/9d/ebc6208a6eb507bce05ce96bc3e28d0cc20ef4fd16fd59473d41c73d3ee5/pg_batch-1.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1800a963a6a765b2cd1559abba1186f3", "sha256": "2f424352dc1e662afb6b5bf57b371074815ffe070b71b3d4d94bab8250a54279" }, "downloads": -1, "filename": "pg_batch-1.0.5.tar.gz", "has_sig": false, "md5_digest": "1800a963a6a765b2cd1559abba1186f3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5989, "upload_time": "2017-07-12T15:48:50", "url": "https://files.pythonhosted.org/packages/09/4d/524fe56665a8ba2959cb8dcc26dd182e4c14d9fec6776cba877dcd7c5a6e/pg_batch-1.0.5.tar.gz" } ], "1.0.6": [ { "comment_text": "", "digests": { "md5": "fc9d16abf2422e93f65028e8c342488b", "sha256": "0b9e3e4ce39603360ba00a24a2b0cc4dbc995480da038134f665070be4ed4c01" }, "downloads": -1, "filename": "pg_batch-1.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "fc9d16abf2422e93f65028e8c342488b", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8275, "upload_time": "2017-07-14T13:00:39", "url": "https://files.pythonhosted.org/packages/af/81/5bb15265dd3f1c44ecfb4111c019675ace32a4967830fd20ad89b400efc3/pg_batch-1.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9828feda649a41b3167669eb39cf0ca8", "sha256": "8a1aa8102c7b14755ca6533f3f93b32e2b462e3280d92543102fa7622b97f080" }, "downloads": -1, "filename": "pg_batch-1.0.6.tar.gz", "has_sig": false, "md5_digest": "9828feda649a41b3167669eb39cf0ca8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6260, "upload_time": "2017-07-14T13:00:40", "url": "https://files.pythonhosted.org/packages/87/e8/f7cf9af7eb36e34dc7f1c2e9d65194659636226d0af0413acfd39b07734c/pg_batch-1.0.6.tar.gz" } ], "1.0.7": [ { "comment_text": "", "digests": { "md5": "b0584ce4967d7e3670e2fd67731725bf", "sha256": "44ba8b1b19b904d141d44ff069648abf002a8a7ad66bccca5114540024a629f8" }, "downloads": -1, "filename": "pg_batch-1.0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b0584ce4967d7e3670e2fd67731725bf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8317, "upload_time": "2018-02-25T00:33:52", "url": "https://files.pythonhosted.org/packages/05/6e/a23a4bc6d8a127db21cf87df9649ab95ffecf2c1da8143e004321c91e6d8/pg_batch-1.0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "0695aa86bf2e4b5c5b7bdbf1536064ed", "sha256": "5184134a55af1e4a14dc12fc8585e5039edf7219fcad2c77fbcee453a3495310" }, "downloads": -1, "filename": "pg_batch-1.0.7.tar.gz", "has_sig": false, "md5_digest": "0695aa86bf2e4b5c5b7bdbf1536064ed", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6268, "upload_time": "2018-02-25T00:33:53", "url": "https://files.pythonhosted.org/packages/6f/df/5e286d7b5f6b6509d0bd39a09ddf8dcfdca5d4395acf59e9d1577bb7da4c/pg_batch-1.0.7.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "af673a97c4c350fcfd0a624eb675951c", "sha256": "a1b3e85453d95194dd83d917b9b56cf6a61c326989444486c774d3cfa702b984" }, "downloads": -1, "filename": "pg_batch-1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "af673a97c4c350fcfd0a624eb675951c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8830, "upload_time": "2018-05-24T01:47:45", "url": "https://files.pythonhosted.org/packages/59/23/608480255f9f84ce1f4eee5e0cd763fd1da38276d8fae11ae0da6270ec8c/pg_batch-1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1d60b7eba6eaff7e742f1c85a0b373f1", "sha256": "6cfadf76bef8c220f30761783ab516a588ea4f2a55bee950701ae4c3f1301520" }, "downloads": -1, "filename": "pg_batch-1.1.tar.gz", "has_sig": false, "md5_digest": "1d60b7eba6eaff7e742f1c85a0b373f1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6533, "upload_time": "2018-05-24T01:47:46", "url": "https://files.pythonhosted.org/packages/1b/80/1eb9ed301b8945ed049d4cbc5ff4b9abe26d5ffbe1d204b6826f2d171aa0/pg_batch-1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "af673a97c4c350fcfd0a624eb675951c", "sha256": "a1b3e85453d95194dd83d917b9b56cf6a61c326989444486c774d3cfa702b984" }, "downloads": -1, "filename": "pg_batch-1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "af673a97c4c350fcfd0a624eb675951c", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8830, "upload_time": "2018-05-24T01:47:45", "url": "https://files.pythonhosted.org/packages/59/23/608480255f9f84ce1f4eee5e0cd763fd1da38276d8fae11ae0da6270ec8c/pg_batch-1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1d60b7eba6eaff7e742f1c85a0b373f1", "sha256": "6cfadf76bef8c220f30761783ab516a588ea4f2a55bee950701ae4c3f1301520" }, "downloads": -1, "filename": "pg_batch-1.1.tar.gz", "has_sig": false, "md5_digest": "1d60b7eba6eaff7e742f1c85a0b373f1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6533, "upload_time": "2018-05-24T01:47:46", "url": "https://files.pythonhosted.org/packages/1b/80/1eb9ed301b8945ed049d4cbc5ff4b9abe26d5ffbe1d204b6826f2d171aa0/pg_batch-1.1.tar.gz" } ] }