{ "info": { "author": "Frederic Wenzel", "author_email": "fwenzel@mozilla.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Web Environment", "Framework :: Django", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "django-gearman\n==============\n\n*django-gearman* is a convenience wrapper for the [Gearman][Gearman]\n[Python Bindings][python-gearman].\n\nWith django-gearman, you can code jobs as well as clients in a Django project\nwith minimal overhead in your application. Server connections etc. all take\nplace in django-gearman and don't unnecessarily clog your application code.\n\n[Gearman]: http://gearman.org\n[python-gearman]: http://github.com/samuel/python-gearman\n\nInstallation\n------------\nIt's the same for both the client and worker instances of your django project:\n\n pip install -e git://github.com/fwenzel/django-gearman.git#egg=django-gearman\n\nAdd ``django_gearman`` to the `INSTALLED_APPS` section of `settings.py`.\n\nSpecify the following setting in your local settings.py file:\n\n # One or more gearman servers\n GEARMAN_SERVERS = ['127.0.0.1']\n\nWorkers\n-------\n### Registering jobs\nCreate a file `gearman_jobs.py` in any of your django apps, and define as many\njobs as functions as you like. The jobs must accept a single argument as\npassed by the caller and must return the result of the operation, if\napplicable. (Note: It must accept an argument, even if you don't use it).\n\nMark each of these functions as gearman jobs by decorating them with\n`django_gearman.decorators.gearman_job`.\n\nFor an example, look at the `gearman_example` app's `gearman_jobs.py` file.\n\n### Job naming\nThe tasks are given a default name of their import path, with the phrase\n'gearman_jobs' stripped out of them, for readability reasons. You can override\nthe task name by specifying `name` parameter of the decorator. Here's how:\n\n @gearman_job(name='my-task-name')\n def my_task_function(foo):\n pass\n\n### Gearman-internal job naming: ``GEARMAN_JOB_NAME``\nThe setting ``GEARMAN_JOB_NAME`` is a function which takes the original task\nname as an argument and returns the gearman-internal version of that task\nname. This allows you to map easily usable names in your application to more\ncomplex, unique ones inside gearman.\n\nThe default behavior of this method is as follows:\n\n new_task_name = '%s.%s' % (crc32(getcwd()), task_name)\n\nThis way several instances of the same application can be run on the same\nserver. You may want to change it if you have several, independent instances\nof the same application run against a shared gearman server.\n\nIf you would like to change this behavior, simply define the\n``GEARMAN_JOB_NAME`` function in the ``settings.py``:\n\n GEARMAN_JOB_NAME = lambda name: name\n\nwhich would leave the internal task name unchanged.\n\n### Task parameters\nThe gearman docs specify that the job function can accept only one parameter\n(usually refered to as the ``data`` parameter). Additionally, that parameter\nmay only be a string. Sometimes that may not be enough. What if you would like\nto pass an array or a dict? You would need to serialize and deserialize them.\nFortunately, django-gearman can take care of this, so that you can spend\nall of your time on coding the actual task.\n\n @gearman_job(name='my-task-name')\n def my_task_function(foo):\n pass\n\n client.submit_job('my-task-name', {'foo': 'becomes', 'this': 'dict'})\n client.submit_job('my-task-name', Decimal(1.0))\n\n### Tasks with more than one parameter\n\nYou can pass as many arguments as you want, of whatever (serializable) type\nyou like. Here's an example job definition:\n\n @gearman_job(name='my-task-name')\n def my_task_function(one, two, three):\n pass\n\nYou can execute this function in two different ways:\n\n client.submit_job('my-task-name', one=1, two=2, three=3)\n client.submit_job('my-task-name', args=[1, 2, 3])\n\nUnfortunately, executing it like this:\n\n client.submit_job('my-task-name', 1, 2, 3)\n\nwould produce the error, because ``submit_job`` from Gearman's Python bindings\ncontains __a lot__ of arguments and it's much easier to specify them via\nkeyword names or a special ``args`` keyword than to type something like seven\n``None``s instead:\n\n client.submit_job('my-task-name', None, None, None, None, None, None, None, 1, 2, 3)\n\nThe only limitation that you have are gearman reserved keyword parameters. As of\nGearman 2.0.2 these are:\n\n * data\n * unique\n * priority\n * background\n * wait_until_complete\n * max_retries\n * poll_timeout\n\nSo, if you want your job definition to have, for example, ``unique`` or\n``background`` keyword parameters, you need to execute the job in a special,\nmore verbose way. Here's an example of such a job and its execution.\n\n @gearman_job(name='my-task-name')\n def my_task_function(background, unique):\n pass\n\n client.submit_job('my-task-name', kwargs={\"background\": True, \"unique\": False})\n client.submit_job('my-task-name', args=[True, False])\n\nFinally:\n\n client.submit_job('my-task-name', background=True, unique=True, kwargs={\"background\": False, \"unique\": False})\n\nDon't panic, your task is safe! That's because you're using ``kwargs``\ndirectly. Therefore, Gearman's bindings would receive ``True`` for\n``submit_job`` function, while your task would receive ``False``.\n\nAlways remember to double-check your parameter names with the reserved words\nlist.\n\n### Starting a worker\nTo start a worker, run `python manage.py gearman_worker`. It will start\nserving all registered jobs.\n\nTo spawn more than one worker (if, e.g., most of your jobs are I/O bound),\nuse the `-w` option:\n\n python manage.py gearman_worker -w 5\n\nwill start five workers.\n\nSince the process will keep running while waiting for and executing jobs,\nyou probably want to run this in a _screen_ session or similar.\n\n### Task queues\nQueues are a virtual abstraction layer built on top of gearman tasks. An\neasy way to describe it is the following example: Imagine you have a task\nfor fetching e-mails from the server, another task for sending the emails\nand one more task for sending SMS via an SMS gateway. A problem you may\nencounter is that the email fetching tasks may effectively \"block\" the worker\n(there could be so many of them, it could be so time-consuming, that no other\ntask would be able to pass through). Of course, one solution would be to add\nmore workers (via the ``-w`` parameter), but that would only temporarily\nsolve the problem. This is where queues come in.\n\nThe first thing to do is to pass a queue name into the job description, like\nthis:\n\n @gearman_job(queue=\"my-queue-name\")\n def some_job(some_arg):\n pass\n\nYou may then proceed to starting the worker that is bound to the specific\nqueue:\n\n python manage.py gearman_worker -w 5 -q my-queue-name\n\nBe aware of the fact that when you don't specify the queue name, the worker\nwill take care of all tasks.\n\nClients\n-------\nTo make your workers work, you need a client app passing data to them. Create\nand instance of the `django_gearman.GearmanClient` class and execute a\n`django_gearman.Task` with it:\n\n from django_gearman import GearmanClient, Task\n client = GearmanClient()\n res = client.do_task(Task(\"gearman_example.reverse\", sentence))\n print \"Result: '%s'\" % res\n\nThe notation for the task name is `appname.jobname`, no matter what pattern\nyou have defined in `GEARMAN_JOB_NAME`.\n\nDispatching a background event without waiting for the result is easy as well:\n\n client.dispatch_background_task('gearman_example.background_counting', None)\n\nFor a live example look at the `gearman_example` app, in the\n`management/commands/gearman_example_client.py` file.\n\nExample App\n-----------\nFor a full, working, example application, add `gearman_example` to your\n`INSTALLED_APPS`, then run a worker in one shell:\n\n python manage.py gearman_worker -w 4\n\nand execute the example app in another:\n\n python manage.py gearman_example_client\n\nYou can see the client sending data and the worker(s) working on it.\n\nLicensing\n---------\nThis software is licensed under the [Mozilla Tri-License][MPL]:\n\n ***** BEGIN LICENSE BLOCK *****\n Version: MPL 1.1/GPL 2.0/LGPL 2.1\n\n The contents of this file are subject to the Mozilla Public License Version\n 1.1 (the \"License\"); you may not use this file except in compliance with\n the License. You may obtain a copy of the License at\n http://www.mozilla.org/MPL/\n\n Software distributed under the License is distributed on an \"AS IS\" basis,\n WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License\n for the specific language governing rights and limitations under the\n License.\n\n The Original Code is django-gearman.\n\n The Initial Developer of the Original Code is Mozilla.\n Portions created by the Initial Developer are Copyright (C) 2010\n the Initial Developer. All Rights Reserved.\n\n Contributor(s):\n Frederic Wenzel \n\n Alternatively, the contents of this file may be used under the terms of\n either the GNU General Public License Version 2 or later (the \"GPL\"), or\n the GNU Lesser General Public License Version 2.1 or later (the \"LGPL\"),\n in which case the provisions of the GPL or the LGPL are applicable instead\n of those above. If you wish to allow use of your version of this file only\n under the terms of either the GPL or the LGPL, and not to allow others to\n use your version of this file under the terms of the MPL, indicate your\n decision by deleting the provisions above and replace them with the notice\n and other provisions required by the GPL or the LGPL. If you do not delete\n the provisions above, a recipient may use your version of this file under\n the terms of any one of the MPL, the GPL or the LGPL.\n\n ***** END LICENSE BLOCK *****\n\n[MPL]: http://www.mozilla.org/MPL/", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/fwenzel/django-gearman", "keywords": null, "license": "MPL", "maintainer": null, "maintainer_email": null, "name": "django-gearman", "package_url": "https://pypi.org/project/django-gearman/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/django-gearman/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/fwenzel/django-gearman" }, "release_url": "https://pypi.org/project/django-gearman/0.2/", "requires_dist": null, "requires_python": null, "summary": "A convenience wrapper for Gearman clients and workers in Django/Python", "version": "0.2" }, "last_serial": 1369771, "releases": { "0.2": [ { "comment_text": "", "digests": { "md5": "08c9f2401a545f55f2bef1df3a578096", "sha256": "7c0ebfc2255ad88778defd7e2058d51f283ad7be41c846a332f75d06324d17f3" }, "downloads": -1, "filename": "django-gearman-0.2.tar.gz", "has_sig": false, "md5_digest": "08c9f2401a545f55f2bef1df3a578096", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9432, "upload_time": "2015-01-04T05:52:15", "url": "https://files.pythonhosted.org/packages/87/03/3dd4e7161ca929f63bd675566bc7c09a4b40ad8bce332b1b8b8c342e0a2e/django-gearman-0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "08c9f2401a545f55f2bef1df3a578096", "sha256": "7c0ebfc2255ad88778defd7e2058d51f283ad7be41c846a332f75d06324d17f3" }, "downloads": -1, "filename": "django-gearman-0.2.tar.gz", "has_sig": false, "md5_digest": "08c9f2401a545f55f2bef1df3a578096", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9432, "upload_time": "2015-01-04T05:52:15", "url": "https://files.pythonhosted.org/packages/87/03/3dd4e7161ca929f63bd675566bc7c09a4b40ad8bce332b1b8b8c342e0a2e/django-gearman-0.2.tar.gz" } ] }