{ "info": { "author": "Ben Timby", "author_email": "btimby@smartfile.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Web Environment", "Framework :: Django", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python" ], "description": ".. image:: https://travis-ci.org/btimby/tpq.svg?branch=master\n :alt: Travis CI Status\n :target: https://travis-ci.org/btimby/tpq\n\n.. image:: https://coveralls.io/repos/github/btimby/tpq/badge.svg?branch=master\n :target: https://coveralls.io/github/btimby/tpq?branch=master\n :alt: Code Coverage\n\n.. image:: https://badge.fury.io/py/tpq.svg\n :target: https://badge.fury.io/py/tpq\n\nTrivial Postgres Queue\n======================\n\nThis is a simple library that can place JSON blobs into a FIFO queue and later\nretrieve them.\n\nThe difference between this queue and other similar queues is that it utilizes\n``FOR UPDATE SKIP LOCKED``.\n\nThe advantage here is that queue items remain in the queue until the current\ntransaction is committed. Upon rollback, the item is left in the queue\nuntouched.\n\nIf other records are modified as part of a larger transaction, those changes are\nalso rolled back. Making the larger queue processing operation atomic. Take the\nfollow order of operations\n\n\n.. code:: sql\n\n BEGIN\n SELECT ... FROM queue FOR UPDATE SKIP LOCKED\n\n INSERT INTO ...\n DELETE FROM ...\n\n ROLLBACK\n\nIn the above, none of the statements have any affect, and the queue item remains\nin the table to be \"retried\" by another consumer. Since `FOR UPDATE` is used,\nthe queue item remains locked to avoid multiple consumers obtaining that item\nfrom the queue.\n\nPython Library Usage\n--------------------\n\nDatabase connection information can be provided via the library API.\n\n.. code:: python\n\n import tpq\n\n # Explicitly provide database connection information\n q = tpq.Queue('queue_name', host='localhost', dbname='foobar')\n q.put('{\"foo\": \"bar\"}')\n\n # Or use shortcut functions:\n tpq.put('queue_name', '{\"foo\": \"bar\"}', host='localhost', dbname='foobar')\n tpq.get('queue_name', host='localhost', dbname='foobar')\n\n # Or to take advantage of cooperative transactions, provide a connection:\n q = tpq.Queue('queue_name', conn=connection)\n q.put('{\"foo\": \"bar\"}')\n\n # Which is also supported by shortcut functions:\n tpq.put('queue_name', '{\"foo\": \"bar\"}', conn=connection)\n tpq.get('queue_name', conn=connection)\n\nOr, you can set the connection info in the environment:\n\n::\n\n $ # Export as URL\n $ export TPQ_URL=\"postgresql://user:pass@localhost/foobar\"\n\n $ # Or separately\n $ export TPQ_HOST=localhost\n $ export TPQ_DB=foobar\n $ export TPQ_USER=user\n $ export TPQ_PASS=pass\n\nThen omit the parameters:\n\n.. code:: python\n\n import tpq\n\n # Use an instance for multiple operations\n with tpq.Queue('queue_name') as q:\n q.put('{\"foo\": \"bar\"}')\n data = q.get()\n\n # Or use shortcut functions:\n tpq.put('queue_name', '{\"foo\": \"bar\"}')\n tpq.get('queue_name')\n\nWaiting\n-------\n\nYou can wait for an item to arrive using the `wait` argument.\n\n.. code:: python\n\n import tpq\n\n # Wait forever\n tpq.get('queue_name', wait=0)\n\n # Don't wait (also can omit the param).\n tpq.get('queue_name', wait=-1)\n\n # Wait specified number of seconds.\n tpq.get('queue_name', wait=5)\n\nCommand Line Interface\n----------------------\n\nCommand line interface is also provided. JSON can be provided via a file or\nstdin (the default).\n\n::\n\n $ # Configure your database.\n $ export TPQ_URL=\"postgresql://user:pass@localhost/foobar\"\n\n $ # JSON via stdin (default).\n $ echo \"{\\\"foo\\\": \\\"bar\\\"}\" | tpq produce queue_name\n\n $ # JSON via file.\n $ tpq produce queue_name --file=message.json\n\n $ # Explicitly provide JSON via stdin.\n $ tpq produce queue_name --file=- < message.json\n\n $ # Then read the item to stdout.\n $ tpq consume queue_name\n {'foo': 'bar'}\n\n $ # If you have trouble (or for logging). Debug output goes to stderr.\n $ TPQ_URL=\"postgresql://user:pass@localhost/foobar\" tpq consume queue_name --debug\n Read database config from environment\n Parsing TPQ_URL\n Database config found\n Attempting to read item\n Item read, returning\n {'foo': 'bar'}\n\n $ # You can wait on the CLI too...\n $ # Forever:\n $ tpq consume queue_name --wait=0\n\n $ # Specified number of seconds:\n $ tpq consume queue_name --wait=5\n\n $ # The return code signals whether an item was received or not.\n $ tpq consume queue_name --wait=-1\n {'foo': 'bar'}\n $ echo $?\n 0\n\n # For an empty queue, you get 1\n $ tpq consume queue_name --wait=-1\n Queue empty\n Traceback (most recent call last):\n File \"/home/btimby/Code/tpq/tpq/__main__.py\", line 24, in consume\n print(get(opt[''], wait=opt['--wait']))\n File \"/home/btimby/Code/tpq/tpq/__init__.py\", line 266, in get\n return q.get(wait=wait)\n File \"/home/btimby/Code/tpq/tpq/__init__.py\", line 233, in get\n raise QueueEmpty()\n queue.Empty\n $ echo $?\n 1\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/btimby/tpq/", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "tpq", "package_url": "https://pypi.org/project/tpq/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/tpq/", "project_urls": { "Homepage": "https://github.com/btimby/tpq/" }, "release_url": "https://pypi.org/project/tpq/1.7/", "requires_dist": null, "requires_python": "", "summary": "Trivial Postgres Queue", "version": "1.7" }, "last_serial": 2874415, "releases": { "1.6": [ { "comment_text": "", "digests": { "md5": "ec1a513fb4e40dbdcb199c82b7f46368", "sha256": "afbb84692b4ded6a0a35a3e52c902600d47517f3bf271894124c0828b2bd0106" }, "downloads": -1, "filename": "tpq-1.6.tar.gz", "has_sig": false, "md5_digest": "ec1a513fb4e40dbdcb199c82b7f46368", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8707, "upload_time": "2017-05-14T13:24:21", "url": "https://files.pythonhosted.org/packages/10/39/a5503c16af64cfb0b8a3a50ff328cc45b0f1c821a7a52a179f106f0ff8ad/tpq-1.6.tar.gz" } ], "1.7": [ { "comment_text": "", "digests": { "md5": "ff98d451479593f0a7f6770d15a88b29", "sha256": "bede15b866a9d65664a18737beb0adc0888409fe7b9fe8be2506d9f069f13143" }, "downloads": -1, "filename": "tpq-1.7.tar.gz", "has_sig": false, "md5_digest": "ff98d451479593f0a7f6770d15a88b29", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8728, "upload_time": "2017-05-15T03:12:36", "url": "https://files.pythonhosted.org/packages/7e/28/033339c424e017b5bfbd98f11a3a5d189835be0982ac39bed743bf41140d/tpq-1.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ff98d451479593f0a7f6770d15a88b29", "sha256": "bede15b866a9d65664a18737beb0adc0888409fe7b9fe8be2506d9f069f13143" }, "downloads": -1, "filename": "tpq-1.7.tar.gz", "has_sig": false, "md5_digest": "ff98d451479593f0a7f6770d15a88b29", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8728, "upload_time": "2017-05-15T03:12:36", "url": "https://files.pythonhosted.org/packages/7e/28/033339c424e017b5bfbd98f11a3a5d189835be0982ac39bed743bf41140d/tpq-1.7.tar.gz" } ] }