{
    "info": {
        "author": "Jozef \u0160ev\u010d\u00edk",
        "author_email": "info@codescale.net",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Console",
            "Environment :: Web Environment",
            "Framework :: Django",
            "Intended Audience :: Developers",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: BSD License",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Topic :: Utilities"
        ],
        "description": "=========================\n Django Gearman Commands\n=========================\n\ndjango-gearman-commands is set of Django management commands\nand base classes aiming to simplify developing and submitting\ntasks to Gearman job server from Django.\n\nAbout Gearman\n=============\n\nGearman, as stated on project website, provides 'a generic application framework to farm out work to other machines or processes that are better suited to do the work'.\nPractically, Gearman is a daemon, service, running on TCP port and waiting for Clients wishing to get job done and Workers who handle and process the jobs.\nGearman - anagram for \"Manager\" itself does exactly what manager does - accepts requests from Clients and distribute them to Workers.\n\nIllustration how it works with one Gearman server::\n\n ----------     ----------     ----------     ----------\n | Client |     | Client |     | Client |     | Client |\n ----------     ----------     ----------     ----------\n      \\             |             |                 /               \n       \\            |             |                /\n        \\           |             |               /\n         \\          |             |              /\n          \\         |             |             /\n\t   \\        |             |            /\n\t    \\       |             |           /\n\t     \\      |             |          /\n              \\     |             |         / \n\t       ------------------------------\n               |      Gearman Job Server    |\n               ------------------------------\n                             |       \n       ----------------------------------------------\n       |              |              |              |\n   ----------     ----------     ----------     ----------\n   | Worker |     | Worker |     | Worker |     | Worker |\n   ----------     ----------     ----------     ----------\n\n\n**Clients** request job to be done. This is your (web)application.\n\n**Gearman** servers inserts job requests into the queue and notifies workers.\n\n**Workers** asks for jobs and process them. Worker is your standalone running script/program.\n\nNote that single Gearman Job Server in illustration above is only meant to simplify illustration.\nEven Gearman itself is really stable, in production it is recommended to run at least 2 job servers\nfor redundancy.\n\nGearman has API for writing Clients and Workers for a lot of environments - Python, Ruby, PHP, Java, Perl, C and probably some others.\nIn practice, you can write workers in same or different languages.\nFor example, if you have PHP application, you can use Gearman PHP API but your workers can be written in Python.\nGearman is designed with flexibility in mind.\n\nYou can read more about Gearman itself on http://gearman.org/index.php\n\nWhy django-gearman-commands\n===========================\n\nFor Python, there is a great python-gearman 2.x API by Yelp - https://github.com/Yelp/python-gearman.\npython-gearman API allows you to write Clients, Workers and perform some additional administration tasks\nin Python without assuming you use some specific web application framework.\n\nThe main issue solved by django-gearman-commands is writing Workers in a way they are aware of your Django application.\n\nGearman Worker is standalone script running on background and waiting for job requests.\nIn theory, you write Python script implementing worker and logic of job and that's it.\nIn practice, your worker needs to use Django facilities.\nWhen you use virtualenv (which you should), you also need to think of imports, DJANGO_SETTINGS_MODULE\nand all that jazz in your Worker script.\n\nDjango provides a way to write standalone, console-based scripts - Commands - https://docs.djangoproject.com/en/dev/howto/custom-management-commands/\nSimilar to './manage.py syncdb' or './manage.py migrate', it is possible to write your own commands invoked by './manage.py mycommand'.\nThis is recommended way to write scripts in Django project which are not run from web application itself.\nDjango Commands can be called from console or from Django application itself, which makes them quite flexible\nfor both manual command-line invocation or programmatic invocation.\n\nWhat django-gearman-commands provides\n-------------------------------------\n\n**Base Class for your Workers**\n\ndjango_gearman_commands.GearmanWorkerBaseCommand is base class for writing Gearman Workers\nas Django commands. No need to hassle with low-level python-gearman API, just inherit GearmanWorkerBaseCommand,\noverride task_name and do_job and you are ready to go.\n\n**Submitting Jobs Easily**\n\ndjango-gearman-commands provides 'gearman_submit_job' command that can be used to submit new jobs\nto gearman. Instead of writing your own class to submit jobs and handle job arguments,\ninvoke ./manage.py gearman_submit_job task_name job_data\n\n**Gearman Administration Overview**\n\nGearman itself provides administration functions returning version of Gearman server, active workers\nand list of registered tasks with their relations to workers, running and pending jobs.\nSimply run ./manage.py gearman_server_info to get current status of Gearman servers.\nIf you want to output that information yourself, you can use django_gearman_commands.GearmanServerInfo class.\n\nGetting started\n===============\n\nSetup\n-----\n\nSo you have your Django application and want to install django-gearman-commands.\ndjango-gearman-commands is standard Django app which provides no models, views or urls,\nonly few classes, custom management commands and tests.\n\nThere is only one new dependency to add to your app (except Django) - python-gearman API::\n\n $ pip install -e git+https://github.com/Yelp/python-gearman.git@2ed9d88941e31e3358a0b80787254d0c2cfaa78a#egg=gearman-dev\n\nWe install python-gearman from git, because there are some fixes on master used by django-gearman-commands.\n\nThere is one optional depedency - prettytable, used by gearman_server_info to output server status in pretty tables::\n\n $ pip install prettytable==0.5\n\nIf you do not install prettytable, ./manage.py gearman_server_info will output server info 'as-is' raw form.\n\nTo install django-gearman-commands itself::\n\n $ pip install django-gearman-commands\n\n\nFinally, add django-gearman-commands to your INSTALLED_APPS::\n\n INSTALLED_APPS = (\n        # ...installed apps...\n        'django_gearman_commands',\n )\n\nAnd add list of Gearman job servers to settings.py::\n\n GEARMAN_SERVERS = ['127.0.0.1:4730']\n\nWriting workers\n---------------\n\n**django_gearman_commands.GearmanWorkerBaseCommand** is base class for your custom django commands acting like Gearman workers.\nYou should write custom command for each specific task.\n\nSuppose we want to write worker to import some complex data which can take a long time.\nCreate file 'gearman_worker_data_import.py' in your Django app management/commands directory\nwith following content::\n\n import django_gearman_commands\n\n class Command(django_gearman_commands.GearmanWorkerBaseCommand):\n    \"\"\"Gearman worker performing 'data_import' job.\"\"\"\n    \n    @property\n    def task_name(self):\n        return 'data_import'\n\n    def do_job(self, job_data):\n        # perform complex data import\n        your_code_performing_job_logic()\n        \n\nAs you see, you need to do three things:\n\n* create file 'management/commands/gearman_worker_MY_TASK_NAME.py' in your django app\n* create Command class inheriting from django_gearman_commands.GearmanWorkerBaseCommand class\n* override task_name property and do_job() method\n\n**task_name** is unique identification of task, which your worker is supposed to do. Submitting jobs is done via sending task name and optional job parameters. Task name can be also easily name-spaced.\n\n**do_job()** is method which will be invoked when job is submitted. If job was submitted with arguments, 'job_data' is not None.\n\n\nRun your worker::\n\n $ ./manage.py gearman_worker_data_import\n\nWorker will start, register itself to Gearman server(s) and wait for jobs. \n\nSubmitting jobs\n---------------\n\nSo now you have your first worker up and running.\nYou can submit first job easily with 'gearman_submit_job' commands::\n\n $ ./manage.py gearman_submit_job data_import\n\n Submitting job: data_import, job data: (empty).\n Job submission done, result: <GearmanJobRequest task='data_import', unique='8e610a031cef8aaf50c30f451d77808d', priority=None, background=True, state='CREATED', timed_out=False>.\n\nBy default, jobs are submitted in background and 'gearman_submit_job' does not wait for job to finish.\nYou can override this with '--foreground' CLI option. See './manage.py gearman_submit_job --help'.\nIf you did everything right, your worker method 'your_code_performing_job_logic()' should be now running in background.\n\nThis method is fine if you want to run job manually or from cron.\nFor example, if you want to run data_import for cron every 5 minutes, you can add something like this to cron::\n\n */5 * * * * /path-to-your-virtualenv/bin/python /path-to-your-project/manage.py gearman_submit_job data_import\n\nHowever, in lot of cases, you want to run job on-demand, for example in some Django view, user makes some action\nand you want to run job immediately - sending email, importing data or anything else you need and don't want to block\nuser's web request until task is completed.\nDjango can call custom management commands programatically, via django.core.management.call_command method::\n\n from django.core.management import call_command\n \n def some_view(request):\n     # ....process your view logic....\n     # submit job to queue\n     call_command('gearman_submit_job', 'data_import')    \n\nBy using job submission wrapper Command 'gearman_submit_job',\nyou are now able submit jobs from console, cron and your app with same API.\n\nTask namespaces\n---------------\n\nSometimes it can be useful to distinguish between jobs submitted with same task name from several applications connected\nto same gearman servers. For example, you may have several instances of same django project deployed for individual\nclients.\nIn that case, you can add GEARMAN_CLIENT_NAMESPACE to your django settings to uniquely identify tasks\nsubmitted by project::\n\n GEARMAN_CLIENT_NAMESPACE = 'MyCustomer1'\n\n\nGearman server info\n-------------------\n\ngearman_server_info outputs current status of Gearman servers.\nIf you installed prettytable dependency, here is how output looks like::\n\n $ ./manage.py gearman_server_info\n +---------------------+------------------------+--------------------+\n | Gearman Server Host | Gearman Server Version | Ping Response Time |\n +---------------------+------------------------+--------------------+\n |    127.0.0.1:4730   |        OK 1.1.3        | 0.0006051063537598 |\n +---------------------+------------------------+--------------------+.\n\n +---------------+---------------+--------------+-------------+\n |   Task Name   | Total Workers | Running Jobs | Queued Jobs |\n +---------------+---------------+--------------+-------------+\n | data_unlock   |       1       |      0       |      0      |\n | data_import   |       1       |      1       |      0      |\n | cache_cleanup |       1       |      0       |      0      |\n +---------------+---------------+--------------+-------------+.\n\n +-----------+------------------+-----------+-----------------+\n | Worker IP | Registered Tasks | Client ID | File Descriptor |\n +-----------+------------------+-----------+-----------------+\n | 127.0.0.1 |   data_unlock    |     -     |        35       |\n | 127.0.0.1 |   data_import    |     -     |        36       |\n | 127.0.0.1 |  cache_cleanup   |     -     |        37       |\n +-----------+------------------+-----------+-----------------+\n\n\nIf you have a lot of workers, you can filter output using command argument (case-sensitive)::\n\n $ ./manage.py gearman_server_info cleanup\n +---------------------+------------------------+--------------------+\n | Gearman Server Host | Gearman Server Version | Ping Response Time |\n +---------------------+------------------------+--------------------+\n |    127.0.0.1:4730   |        OK 1.1.3        | 0.0006871223449707 |\n +---------------------+------------------------+--------------------+.\n\n +---------------+---------------+--------------+-------------+\n |   Task Name   | Total Workers | Running Jobs | Queued Jobs |\n +---------------+---------------+--------------+-------------+\n | cache_cleanup |       1       |      0       |      0      |\n +---------------+---------------+--------------+-------------+.\n\n +-----------+------------------+-----------+-----------------+\n | Worker IP | Registered Tasks | Client ID | File Descriptor |\n +-----------+------------------+-----------+-----------------+\n | 127.0.0.1 |  cache_cleanup   |     -     |        37       |\n +-----------+------------------+-----------+-----------------+\n\n\nPractical production deployment with Supervisor\n===============================================\n\nIn production, you need to be sure about two things:\n\n * your Gearman server is running\n * your Workers are running\n\nSupervisor -  http://supervisord.org/ is babysitter for processes.\nIt allows you to launch, restart and monitor running processes.\n\nSupervisor + Gearman\n--------------------\n\nAssuming you have supervisor installed and know the basics,\nyou can create 'gearman.conf' in /etc/supervisor/conf.d with following content::\n\n [program:gearman]\n command=/home/gearman/gearmand/gearmand/gearmand -q libsqlite3 --libsqlite3-db=/home/gearman/gearmand-sqlite.db -L 127.0.0.1\n numprocs=1\n directory=/home/gearman\n stdout_logfile=/var/log/gearman.log\n autostart=true\n autorestart=true\n user=gearman\n\nThis will start Gearman server compiled in /home/gearman/earmand/gearmand/gearmand with SQLite persistent queue on localhost.\nOf course, your variables may vary.\n\nSupervisor + Workers\n--------------------\n\nYou can create single .conf file for all workers relevant to single application.\nThis will create process 'group' and allows you to reload of all workers related to some application\nat once when you redeploy new code.\n\nFor example, create 'myapp.conf' in /etc/supervisor/conf.d with all workers relevant to 'myapp':::\n\n [program:myapp_data_import]\n command=/path-to-your-virtualenv/bin/python /path-to-your-project/manage.py gearman_worker_data_import\n numprocs=1\n directory=/home/myapp/\n stdout_logfile=/var/log/myapp_data_import.log\n autostart=true\n autorestart=true\n user=myapp\n\n [program:myapp_send_invoices]\n command=/path-to-your-virtualenv/bin/python /path-to-your-project/manage.py gearman_worker_send_invoices\n numprocs=1\n directory=/home/myapp/\n stdout_logfile=/var/log/myapp_send_invoices.log\n autostart=true\n autorestart=true\n user=myapp\n\n [group:myapp]\n programs=myapp_data_import, myapp_send_invoices\n\nAfter redeployment, you can restart all workers:::\n\n $ supervisorctl reread\n $ supervisorctl update\n $ supervisorctl restart myapp:*\n\nI recommend automating this with Fabric - http://docs.fabfile.org/\n\nContributing to django-gearman-commands\n=======================================\n\nContributions are welcome.\nIf possible, please use following workflow:\n\n * find out what is bothering you\n * check Issues page if problem is not already discussed\n * fork django-gearman-commands\n * fix it in your fork and add test to tests/__init__.py\n * add yourself as Contributor to 'Authors and Contributors'\n * and make Pull Request with description what change is supposed to do\n\nRunning tests\n-------------\n\nTests are located in tests/__init__.py file.\nThere is a wrapper 'runtests.py' in root directory to setup Django environment with minimal dependencies and run tests.\nThe point is to allow testing of django-gearman-commands during development without full-blown Django application::\n\n $ python runtests.py\n\nAs you can read from runtests.py, tests expect Gearman server running on localhost on standard port 4730.\n\nLicense\n=======\n\nBSD, see LICENSE for details\n\nAuthors and Contributors\n========================\n\nAuthor: Jozef \u0160ev\u010d\u00edk, sevcik@codescale.net\n\nContributors:\n\n * Vladim\u00edr Gorej (gorej@codescale.net)",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "http://github.com/CodeScaleInc/django-gearman-commands/tarball/master",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://www.codescale.net/en/community#django-gearman-commands",
        "keywords": "django gearman gearmand jobs queue",
        "license": "BSD",
        "maintainer": null,
        "maintainer_email": null,
        "name": "django-gearman-commands",
        "package_url": "https://pypi.org/project/django-gearman-commands/",
        "platform": "any",
        "project_url": "https://pypi.org/project/django-gearman-commands/",
        "project_urls": {
            "Download": "http://github.com/CodeScaleInc/django-gearman-commands/tarball/master",
            "Homepage": "http://www.codescale.net/en/community#django-gearman-commands"
        },
        "release_url": "https://pypi.org/project/django-gearman-commands/0.6.1/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Django management commands for working with Gearman job server from Django framework",
        "version": "0.6.1"
    },
    "last_serial": 1011826,
    "releases": {
        "0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e1019ac06b2e121974dbba4fad39fe97",
                    "sha256": "b5bf6d0a84717e9b4ffc033118bea7ad85eb6584f2aa8365cd90ebc892132437"
                },
                "downloads": -1,
                "filename": "django-gearman-commands-0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "e1019ac06b2e121974dbba4fad39fe97",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 11599,
                "upload_time": "2012-04-07T19:23:15",
                "url": "https://files.pythonhosted.org/packages/49/a3/f20af7f9adeb221eb5db33da8f62ad26dbdae62634f6e854cfbfecb893b5/django-gearman-commands-0.1.tar.gz"
            }
        ],
        "0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "dd29df6937629b618c6f361c13e7f90c",
                    "sha256": "9ac206a874ab0c75c5f28e19f6f6be1eb4dc9c9b7dd07781f6bafbde8249fd9a"
                },
                "downloads": -1,
                "filename": "django-gearman-commands-0.2.zip",
                "has_sig": false,
                "md5_digest": "dd29df6937629b618c6f361c13e7f90c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27122,
                "upload_time": "2012-12-24T16:03:00",
                "url": "https://files.pythonhosted.org/packages/81/58/b1d03678ddc99c3d1fe50c6f46805301114989cc11fa63000a2f78f1e8f0/django-gearman-commands-0.2.zip"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ead3fdddaa7ae3e4d3d5aba30b26b84c",
                    "sha256": "7c8a9db341adbb9a8f594d3a1ab8c3b919575095f1ee302399b51d6fc4d05239"
                },
                "downloads": -1,
                "filename": "django-gearman-commands-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ead3fdddaa7ae3e4d3d5aba30b26b84c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12931,
                "upload_time": "2014-02-18T14:29:04",
                "url": "https://files.pythonhosted.org/packages/07/38/50d93f212f09fb728d399271c18b8b46e8f7390fc425ec2d6b163c3fdaf5/django-gearman-commands-0.4.0.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b6e31bcc3b2d827d2fc28663abf7c970",
                    "sha256": "fd968ed0db4f2443b1b37c4021edf504d4a51f7b527d3fdf7911d41e8db2b83a"
                },
                "downloads": -1,
                "filename": "django-gearman-commands-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "b6e31bcc3b2d827d2fc28663abf7c970",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13451,
                "upload_time": "2014-02-18T15:31:00",
                "url": "https://files.pythonhosted.org/packages/3b/5d/72d2728ba203a0a08c8976274f6497dac69e854af4bc75d70bad974d813b/django-gearman-commands-0.5.0.tar.gz"
            }
        ],
        "0.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8ea0c6715e7df31ed2da02b9a21541df",
                    "sha256": "10d343eb83fb0da3000359c1a5ab695e7b9127444933c2fb68b5008179c0edaf"
                },
                "downloads": -1,
                "filename": "django-gearman-commands-0.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8ea0c6715e7df31ed2da02b9a21541df",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13479,
                "upload_time": "2014-02-19T14:31:15",
                "url": "https://files.pythonhosted.org/packages/07/a0/2cea903fc7546111d7becce377aebcb10e0534a32712413337e4dbca8d55/django-gearman-commands-0.6.0.tar.gz"
            }
        ],
        "0.6.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d28cc9e736749338ba2e2008ccb22646",
                    "sha256": "eb40f897daa08ba28772f8680d7347b9072a36ed3a09839bdde3931966f3404b"
                },
                "downloads": -1,
                "filename": "django-gearman-commands-0.6.1.tar.gz",
                "has_sig": false,
                "md5_digest": "d28cc9e736749338ba2e2008ccb22646",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 13493,
                "upload_time": "2014-02-25T17:05:34",
                "url": "https://files.pythonhosted.org/packages/50/45/4023bf3cb0225ff745ea97cfeb1b749a3434a6d24eb468452d0f06a69b92/django-gearman-commands-0.6.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d28cc9e736749338ba2e2008ccb22646",
                "sha256": "eb40f897daa08ba28772f8680d7347b9072a36ed3a09839bdde3931966f3404b"
            },
            "downloads": -1,
            "filename": "django-gearman-commands-0.6.1.tar.gz",
            "has_sig": false,
            "md5_digest": "d28cc9e736749338ba2e2008ccb22646",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 13493,
            "upload_time": "2014-02-25T17:05:34",
            "url": "https://files.pythonhosted.org/packages/50/45/4023bf3cb0225ff745ea97cfeb1b749a3434a6d24eb468452d0f06a69b92/django-gearman-commands-0.6.1.tar.gz"
        }
    ]
}