{ "info": { "author": "Nauman Tariq", "author_email": "nauman3128@gmail.com", "bugtrack_url": null, "classifiers": [ "Environment :: Web Environment", "Framework :: Django", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python" ], "description": "django-sqs-qc\r\n ==========\r\n\r\nAuthor: Maciej Pasternacki \r\nDate: 2010-01-15 Fri\r\nMaintainer: Nauman Tariq \r\n\r\nIntroduction:\r\nModified Version of django-sqs (https://pypi.python.org/pypi/django-sqs/0.2)\r\nActual django-sqs creates new queue every time worker listener is started\r\nIn this modified version, it uses the existing queue for messages.\r\n\r\n\r\nIntegrate Amazon Simple Queue Service in your Django project\r\n\r\nTable of Contents\r\n=================\r\n1 Setup \r\n2 Receivers \r\n 2.1 Register using a decorator \r\n 2.2 Register manually \r\n 2.3 Example receiver \r\n3 Receiving \r\n4 Sending \r\n 4.1 Using decorated function \r\n 4.2 Manually \r\n5 Custom message classes \r\n 5.1 ModelInstanceMessage class \r\n6 Management \r\n 6.1 manage.py sqs_status \r\n 6.2 manage.py sqs_clear \r\n 6.3 manage.py sqs_wait \r\n7 Views \r\n8 FIXME \r\n 8.1 Sensible forking/threading or multiplexing instead of the fork hack? \r\n 8.2 Autoimporting receivers.py from apps \r\n 8.3 docstrings \r\n 8.4 Minimize polling \r\n 8.5 Custom exception to leave message in queue \r\n\r\n\r\n1 Setup \r\n~~~~~~~~\r\n Boto library for accessing Amazon Web Services is required.\r\n\r\n 1. Add `django_sqs' to your Python path\r\n 2. Add `django_sqs' to INSTALLED_APPS setting\r\n 3. Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY\r\n 4. Optionally set SQS_QUEUE_PREFIX to prefix your queues and avoid\r\n clashes with other developers, production/staging env and so on.\r\n SQS_QUEUE_PREFIX is required when DEBUG is true, and recommended\r\n even in production mode.\r\n 5. Optionally set SQS_DEFAULT_VISIBILITY_TIMEOUT (default is 60 seconds)\r\n 6. Optionally set SQS_POLL_PERIOD (default is 10 seconds)\r\n 7. Optionally set AWS_REGION (default to \"us-east-1\")\r\n\r\n2 Receivers \r\n~~~~~~~~~~~~\r\n Create receiver function that accepts one argument, which will be an\r\n instance of `boto.sqs.message.Message' or its custom subclass.\r\n Then, register it as queue receiver:\r\n\r\n2.1 Register using a decorator \r\n===============================\r\n Decorate receiver function with:\r\n\r\n django_sqs.receiver([queue_name=None, visibility_timeout=None, message_class=None, delete_on_start=False, close_database=False, suffixes=()])\r\n\r\n Decorated function will become an instance of\r\n `django_sqs.registered_queue.RegisteredQueue.ReceiverProxy' class.\r\n Instance is callable - you may call it with an instance of\r\n appropriate message class, or its constructor's keyword arguments,\r\n and the message will be added to the queue. Instance provides also\r\n attributes `direct' with original decorated function, and\r\n `registered_queue' with appropriate\r\n `django_sqs.registered_queue.RegisteredQueue' instance.\r\n\r\n If `queue_name' is omitted or false, function's `__module__' and\r\n `__name__' is used, with dots converted to double underscores.\r\n\r\n The `suffixes' argument is a tuple of known queue name suffixes. If\r\n an unknown suffix is used, a warning will be issued.\r\n\r\n If `delete_on_start' is true, received message will be deleted from\r\n queue right after it's been received, before receiver function is\r\n called. If it is false (which is the default), it will be deleted\r\n after receiver function has finished, when message has been fully\r\n processed.\r\n\r\n If `close_database' is true, all database connections will be\r\n closed after processing each message to prevent pending unclosed\r\n transactions.\r\n\r\n Queue name suffixes can be used to split processing similar items\r\n to multiple queues (e.g. use separate queue for big input items to\r\n distribute load).\r\n\r\n2.2 Register manually \r\n======================\r\n Alternatively, you can avoid decoration, and register a receiver\r\n manually by calling:\r\n\r\n django_sqs.register(queue_name, [fn=None, visibility_timeout=None, message_class=None, delete_on_start=False, suffixes=()])\r\n\r\n If `fn' is None or not given, no handler is assigned: messages can\r\n be sent, but won't be received.\r\n\r\n Create function in modules that will be imported by default\r\n (recommendation: use `receivers.py' and import them in `models.py',\r\n autoimporting TBD).\r\n\r\n2.3 Example receiver \r\n=====================\r\n @receiver(\"test\")\r\n def receive_message(msg):\r\n print 'received:', msg.get_body()\r\n\r\n3 Receiving \r\n~~~~~~~~~~~~\r\n python manage.py runreceiver [--message-limit=N] [--suffix=SUFFIX] [queue_name [queue_name [...]]]\r\n\r\n If no `queue_name' parameters are given, receive from all configured\r\n queues.\r\n\r\n If more than one queue is registered, a new process is forked for\r\n each queue.\r\n\r\n For each message received on the queue, registered receiver function\r\n is called with the message instance as argument. If receiver\r\n function returns successfully, message is then deleted from queue.\r\n If receiver message raises an exception, exception traceback is\r\n logged using logging module, and message is deleted. If receiver\r\n sees a restartable error and wants to keep message in queue, it\r\n should raise `django_sqs.RestartLater' exception - this exception will\r\n leave the message in queue.\r\n\r\n Options:\r\n\r\n `--message-limit=N': exit after receiving `N' messages\r\n `--suffix=SUFFIX': Use queue name suffix\r\n\r\n4 Sending \r\n~~~~~~~~~~\r\n\r\n4.1 Using decorated function \r\n=============================\r\n You can simply call function decorated with `@receiver' decorator,\r\n providing a message instance or keyword arguments (like for `send'\r\n function described below).\r\n\r\n4.2 Manually \r\n=============\r\n To send a message manually, use following function:\r\n\r\n django_sqs.send(queue_name, message=None, suffix=None, **kwargs)\r\n\r\n `message' should be an instance of `message_class' configured with\r\n `receiver' decorator or `register' function for the queue (or\r\n `boto.sqs.message.Message').\r\n\r\n When `message' is omitted or `None', new instance of queue's message\r\n class will be instantiated using `**kwargs'. With default message\r\n class, `boto.sqs.message.Message', we can simply provide body:\r\n\r\n django_sqs.send(\"a_queue\", body='Lorem ipsum dolor sit amet')\r\n\r\n `suffix' is a queue name suffix to use.\r\n\r\n5 Custom message classes \r\n~~~~~~~~~~~~~~~~~~~~~~~~~\r\n For sending other values than raw, non-unicode strings, any of\r\n classes provided in `boto.sqs.message' or their subclasses may be\r\n used. The module is well commented (much better than this one), so\r\n go ahead and read the fine source!\r\n\r\n\r\n5.1 ModelInstanceMessage class \r\n===============================\r\n The `django_sqs.message.ModelInstanceMessage' class is provided for\r\n convenience. It encodes a single model instance, using Django's\r\n ContentType framework (as app/model/primary key triple). It\r\n accepts `instance' keyword parameter in constructor, and provides\r\n `get_instance()' method.\r\n\r\n There is no support for passing additional information except the\r\n instance yet.\r\n\r\n6 Management \r\n~~~~~~~~~~~~~\r\n\r\n6.1 manage.py sqs_status \r\n=========================\r\n Prints the (approximate) count of messages in the queue.\r\n\r\n6.2 manage.py sqs_clear \r\n========================\r\n Clears all queues (by default), or queues named as arguments.\r\n Prints number of messages deleted.\r\n\r\n If queue receivers are running or were running recently, some\r\n messages may be still locked and won't be deleted. Command may\r\n need to be re-run.\r\n\r\n6.3 manage.py sqs_wait \r\n=======================\r\n Waits until specified (or all) queues are empty.\r\n\r\n7 Views \r\n~~~~~~~~\r\n A single view, `django_sqs.views.status', is provided for simple,\r\n plain text queue status report (same as `manage.py sqs_status').\r\n\r\n8 FIXME \r\n~~~~~~~~\r\n\r\n8.1 TODO Sensible forking/threading or multiplexing instead of the fork hack? \r\n==============================================================================\r\n\r\n8.2 TODO Autoimporting receivers.py from apps \r\n==============================================\r\n\r\n8.3 TODO docstrings \r\n====================\r\n\r\n8.4 TODO Minimize polling \r\n==========================\r\n Amazon charges for every call. Less polling, lower invoice. Some\r\n exponential backoff + out-of-band signal (view?) to wake up a running\r\n receiver process may be a good thing.\r\n\r\n8.5 TODO Custom exception to leave message in queue \r\n====================================================\r\n Provide a custom exception class that won't be handled by receive\r\n loop (i.e. no backtrace) that can be used by receiver function to\r\n explicitly leave message in queue without printing backtrace and\r\n alarming everyone.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/nauman3128/sqs-django-qc", "keywords": "", "license": "UNKNOWN", "maintainer": "Nauman Tariq", "maintainer_email": "nauman3128@gmail.com", "name": "django-sqs-qc", "package_url": "https://pypi.org/project/django-sqs-qc/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/django-sqs-qc/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/nauman3128/sqs-django-qc" }, "release_url": "https://pypi.org/project/django-sqs-qc/0.3/", "requires_dist": null, "requires_python": null, "summary": "django package to use existing Amazon SQS Queue", "version": "0.3" }, "last_serial": 1820074, "releases": { "0.2": [ { "comment_text": "", "digests": { "md5": "1edd6240e2ce5709610903fb2f18f306", "sha256": "c00c11fb6fcd34eb60eb2036a3dada7845ef4b6ba774ae2e2ee5acc69f11e87d" }, "downloads": -1, "filename": "django-sqs-qc-0.2.zip", "has_sig": false, "md5_digest": "1edd6240e2ce5709610903fb2f18f306", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14950, "upload_time": "2015-11-17T06:48:30", "url": "https://files.pythonhosted.org/packages/08/ec/09e0e5ce996e6c81b4090e75f78bc8a458af58cc7af2185d36d3ee44aeed/django-sqs-qc-0.2.zip" } ], "0.3": [] }, "urls": [] }