{ "info": { "author": "Jim Fulton", "author_email": "jim@zope.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: Zope Public License", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "==================\nAmazon SQS support\n==================\n\nThis is a small wrapper around SQS that provides some testing support\nand and some abstraction over the boto SQS APIs.\n\nThere are 2 basic parts, a producer API and a worker API.\n\nNote that these APIs don't let you pass AWS credentials. This means\nthat you must either pass credentials through ~/.boto configuration,\nthrough environment variables, or through temporary credentials\nprovided via EC2 instance roles.\n\nProducing jobs\n==============\n\nTo send work to workers, instantiate a Queue:\n\n >>> import zc.sqs\n >>> queue = zc.sqs.Queue(\"myqueue\")\n Connected to region us-east-1.\n\nThe SQS queue must already exist. Creating queues is outside the\nscope of these APIs. Trying to create a Queue instance with a\nnonexistent queue name will result in an exception being raised.\n\n >>> import mock\n >>> with mock.patch(\"boto.sqs.connect_to_region\") as conn:\n ... conn().get_queue.return_value = None\n ... zc.sqs.Queue(\"nonexistent\") # doctest: +IGNORE_EXCEPTION_DETAIL\n Traceback (most recent call last):\n ...\n NonExistentQueue: nonexistent\n\nTo place data in the queue, you call it. You can pass positional,\nand/or keyword arguments.\n\n >>> queue(1, 2, x=3)\n [[1, 2], {'x': 3}]\n\nIn this example, we're running in test mode. In test mode, data are\nsimply echoed back (unless we wire up a worker, as will be discussed\nbelow).\n\nArguments must be json encodable.\n\nWorkers\n=======\n\nWorkers are provided as factories that accept configuration data and\nreturn callables that are called with queued messages. A worker\nfactory could be implemented with a class that has __init__ and\n__call__ methods, or with a function that takes configuration data and\nreturns a nested function to handle messages.\n\nNormally, workers don't return anything. If the input is bad, the\nworker should raise an exception. The exception will be logged, as\nwill the input data. If the input is good, but the worker can't\nperform the request, it should raise zc.sqs.TransientError to indicate\nthat the work should be retried later.\n\nContainers\n==========\n\nTo attach your workers to queues, you use a container, which is just a\nprogram that polls an SQS queue and calls your worker. There are\ncurrently 2 containers:\n\nsequential\n The sequential container pulls requests from an SQS queue and hands\n them to a worker, one at a time.\n\n This is a script entry point and accepts an argument list,\n containing the path to an ini file. It uses \"long polling\" to loop\n efficiently.\n\ntest\n The test container is used for writing tests. It supports\n integration tests of producer and worker code. When running in\n test mode, it replaces (part of) the sequential container.\n\nThe sequential entry point takes the name of an ini file with 2 sections:\n\ncontainer\n The container section configures the container with options:\n\n worker MODULE:expr\n The worker constructor\n\n queue\n The name of an sqs queue to listen to.\n\n loggers\n A ZConfig-based logger configuration string.\n\nworker (optional)\n Worker options, passed to the worker constructor as a dictionary.\n\n If not provided, an empty dictionary will be passed.\n\nHere's a simple (pointless) example to illustrate how this is wired\nup. First, we'll define a worker factory::\n\n def scaled_addr(config):\n scale = float(config.get('scale', 1))\n\n def add(a, b, x):\n if x == 'later':\n print (\"not now\")\n raise zc.sqs.TransientError # Not very imaginative, I know\n print (scale * (a + b + x))\n\n return add\n\n.. -> src\n\n >>> import zc.sqs.tests\n >>> exec(src, zc.sqs.tests.__dict__)\n\nNow, we'll define a container configuration::\n\n [container]\n worker = zc.sqs.tests:scaled_addr\n queue = adder\n loggers =\n \n level INFO\n \n path STDOUT\n format %(levelname)s %(name)s %(message)s\n \n \n \n level INFO\n propagate false\n name zc.sqs.messages\n \n path messages.log\n format %(message)s\n \n \n\n [worker]\n scale = 2\n\n.. -> ini\n\n >>> with open('ini', 'w') as f:\n ... _ = f.write(ini)\n\nNow, we'll run the container.\n\n >>> import zc.thread\n >>> @zc.thread.Thread\n ... def thread():\n ... zc.sqs.sequential(['ini'])\n\n.. give it some time\n\n >>> import time\n >>> time.sleep(.1)\n Connected to region us-east-1.\n\nWe ran the container in a thread because it runs forever and wouldn't\nreturn.\n\nNormally, the entry point would run forever, but since we're running\nin test mode, the container just wires the worker up to the test\nenvironment.\n\nNow, if we create a queue (in test mode):\n\n >>> adder = zc.sqs.Queue(\"adder\")\n Connected to region us-east-1.\n\nand send it work:\n\n >>> adder(1, 2, 3)\n 12.0\n deleted '[[1, 2, 3], {}]'\n\nWe see that the worker ran.\n\nWe also see a testing message showing that the test succeeded.\n\nIf a worker can't perform an action immediately, it indicates that the\nmessage should be delayed by raising TransientError as shown in the\nworker example above:\n\n >>> adder(1, 2, 'later')\n not now\n\nIn this case, since the worker raised TransientError, the message\nwasn't deleted from the queue. This means that it'll be handled later\nwhen the job times out.\n\nIf the worker rasies an exception, the exception and the message are\nlogged:\n\n >>> adder(1, 2, '') # doctest: +ELLIPSIS\n ERROR zc.sqs Handling a message\n Traceback (most recent call last):\n ...\n TypeError: unsupported operand type(s) for +: 'int' and '...'\n deleted '[[1, 2, \"\"], {}]'\n\n >>> with open(\"messages.log\") as f:\n ... print(f.read())\n [[1, 2, \"\"], {}]\n \n\nSilencing tests\n===============\n\nSometimes, you don't want the testing infrastructure to output\ninformation when sending messages. There testing ``setUp`` method\nadds an ``sqs_queues`` attribute to globals. You can call\n``be_silent`` to make it stop outputting infomation:\n\n >>> sqs_queues.be_silent()\n\nAfter calling this, any subsequent queues will be quiet:\n\n >>> queue = zc.sqs.Queue(\"quiet\")\n >>> queue(1)\n\nYou can get the queued data:\n\n >>> [m.get_body() for m in sqs_queues.get_queue(\"quiet\").get_messages()]\n ['[[1], {}]']\n\nYou can switch back to being noisy:\n\n >>> sqs_queues.be_silent()\n\n >>> queue = zc.sqs.Queue(\"loud\")\n >>> queue(1)\n\n\n.. cleanup\n\n >>> print('Stopping'); adder.queue.queue.put('STOP'); time.sleep(.01) # doctest: +ELLIPSIS\n Stopping...\n\nChanges\n=======\n\n1.0.0\n-----\n\n- Python 3 support.\n\n0.3.0 (2014-10-17)\n------------------\n\n- Use long polling instead of a configurable polling interval.\n\n0.2.1 (2013-05-15)\n------------------\n\n- Better error handling when SQS queues don't exist.\n\n0.2.0 (2013-05-15)\n------------------\n\n- A new silent mode for test queues.\n\n0.1.0 (2013-04-23)\n------------------\n\nInitial release.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "UNKNOWN", "keywords": null, "license": "ZPL 2.1", "maintainer": null, "maintainer_email": null, "name": "zc.sqs", "package_url": "https://pypi.org/project/zc.sqs/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/zc.sqs/", "project_urls": { "Download": "UNKNOWN", "Homepage": "UNKNOWN" }, "release_url": "https://pypi.org/project/zc.sqs/1.0.0/", "requires_dist": null, "requires_python": null, "summary": "==================", "version": "1.0.0" }, "last_serial": 1596577, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "0e6b66cd52cd51aa2dcbc0a1c9f3b1eb", "sha256": "7159bc7d43cd6844e9113a9a33f60d0d908203fff190eb7e038f585106cee1f2" }, "downloads": -1, "filename": "zc.sqs-0.1.0.tar.gz", "has_sig": false, "md5_digest": "0e6b66cd52cd51aa2dcbc0a1c9f3b1eb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7047, "upload_time": "2013-04-23T19:16:44", "url": "https://files.pythonhosted.org/packages/8e/8c/a0bfc5bd4f77a4f7c31014795bca830eddc4f81c7af745dbfe385d9843e7/zc.sqs-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "124cc6e9e41a00d866f0e11f2a1d34f8", "sha256": "cd3f360aa25edc342de8eab5f8774d7ecd4434a8b18a3e9d8ab1391f0fc43f46" }, "downloads": -1, "filename": "zc.sqs-0.2.0.tar.gz", "has_sig": false, "md5_digest": "124cc6e9e41a00d866f0e11f2a1d34f8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6959, "upload_time": "2013-05-15T17:15:57", "url": "https://files.pythonhosted.org/packages/de/22/f47954857ce879aaa47d87d391b168e26cff1b71c06eb6aa898ece8df914/zc.sqs-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "e0c066b4a37d0fbc582bb6fe7a9da85c", "sha256": "9a769c703a64a1ba5870e9aa8f8f5fbf31e2ba1a7bc77597c367f8342a8c376c" }, "downloads": -1, "filename": "zc.sqs-0.2.1.tar.gz", "has_sig": false, "md5_digest": "e0c066b4a37d0fbc582bb6fe7a9da85c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7195, "upload_time": "2013-05-15T17:29:17", "url": "https://files.pythonhosted.org/packages/29/bb/77609fc8c563bfb5226ab69df01a5d34e2cc61b263a860a2159185fb349b/zc.sqs-0.2.1.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "51504da223afdf7f975f38582dec64b6", "sha256": "8be049be22f38436fd3036542691a08c25eca63ddfd91a6e4dd53a7089977181" }, "downloads": -1, "filename": "zc.sqs-0.3.0.tar.gz", "has_sig": false, "md5_digest": "51504da223afdf7f975f38582dec64b6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6753, "upload_time": "2014-10-17T20:21:19", "url": "https://files.pythonhosted.org/packages/45/41/314595cfbc67c2f38fd90e752a9de9e1f859390bde729cd6e96f158b0d9f/zc.sqs-0.3.0.tar.gz" } ], "1.0.0": [ { "comment_text": "", "digests": { "md5": "fc2b8eae7cb681d9cf8bde9a29483f5f", "sha256": "b1ca7282e01e445bfe825d56d0d8f003bc853bcdc8815553de391b63dc446359" }, "downloads": -1, "filename": "zc.sqs-1.0.0.tar.gz", "has_sig": false, "md5_digest": "fc2b8eae7cb681d9cf8bde9a29483f5f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8124, "upload_time": "2015-06-17T22:46:17", "url": "https://files.pythonhosted.org/packages/3f/bc/7a0e3469b889a7658fa588838da928d3a7587e14148736ad24f49d7cf1b3/zc.sqs-1.0.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "fc2b8eae7cb681d9cf8bde9a29483f5f", "sha256": "b1ca7282e01e445bfe825d56d0d8f003bc853bcdc8815553de391b63dc446359" }, "downloads": -1, "filename": "zc.sqs-1.0.0.tar.gz", "has_sig": false, "md5_digest": "fc2b8eae7cb681d9cf8bde9a29483f5f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8124, "upload_time": "2015-06-17T22:46:17", "url": "https://files.pythonhosted.org/packages/3f/bc/7a0e3469b889a7658fa588838da928d3a7587e14148736ad24f49d7cf1b3/zc.sqs-1.0.0.tar.gz" } ] }