{ "info": { "author": "Thomas Weholt", "author_email": "thomas@weholt.org", "bugtrack_url": null, "classifiers": [ "Development Status :: 2 - Pre-Alpha", "Environment :: Web Environment", "Framework :: Django", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Utilities" ], "description": "Django-kolibri\r\n==============\r\n\r\nVersion : 0.2.0\r\nAuthor : Thomas Weholt \r\nLicense : Modified BSD\r\nStatus : pre-alpha\r\nUrl : https://bitbucket.org/weholt/django-kolibri\r\n\r\n\r\nBackground\r\n----------\r\n\r\nKolibri is a reusable django app for designing and executing asynchronous processes\r\nand workflows. A workflow is a collections of steps in a defined order,\r\nprocessing data in each step. A step can break the flow if an exception is\r\nraised and/or a specified step can be executed to handle a specific exception.\r\nKolibri uses celery to handle processing in the background. All processors\r\nand workflows can only be started by staff members, but more fine grained access\r\ncontrol might be implemented in future versions.\r\n\r\nThe project got started because I needed to control how I added content to a\r\nphoto project I'm developing in django. The project involved lots of heavy processes\r\nlike thumbnail generation and metadata processing. Adding content consists of steps that\r\nneeds to be done in a specific order, and I need to control what action to take\r\nif one step throws an exception. I was using celery, but adding a new step or\r\nprocess was tedious and I wanted more dynamic way of defining and managing processors.\r\n\r\nThe current implementation is not stable and a proof of concept. Comments very\r\nwelcome, especially on how to monitor status of celery processes and provide\r\nfeedback to the user.\r\n\r\n\r\nFeatures\r\n--------\r\n\r\n* asynchronous processes, which can process items/querysets or execute processes not related to specific models or instances (sending email, scanning filesystems etc)\r\n\r\n* connect several processors into workflows, with exception handling, clean-up steps and an optional fluent interface\r\n\r\n* template tags to handle execution of processors/workflows for an item or queryset in your templates\r\n\r\n* admin action integration for your models\r\n\r\n* dashboard listing running processors\r\n\r\n* a concept of pending processors and a history of what has been processed so you don't execute unnecessary processesors or workflows\r\n\r\n* user exclusive processors so two users can execute the same processor at the same time without touching the same data\r\n\r\n* logging and history, with direct link to processed instances\r\n\r\n* ajax integration using jquery\r\n\r\n\r\nPlanned features\r\n----------------\r\n\r\n* better examples, more detailed tutorial and actual documentation in the source\r\n\r\n* full-blown dashboard with feedback and progress from running processes and some way of killing processes\r\n\r\n* nicely formatted logs and history for processed items\r\n\r\n* a way of telling users that something is going on with the item they're looking at (progressbar, growl notification etc.)\r\n\r\n\r\nInstallation\r\n------------\r\n\r\npip install django-kolibri\r\n\r\nor\r\n\r\nhg clone https://bitbucket.org/weholt/django-kolibri\r\npython setup.py install\r\n\r\n* set STATIC_ROOT and STATIC_URL in settings.py\r\n* add 'kolibri' to your installed apps\r\n* add url(r'^kolibri/', include('kolibri.urls')), to your urls.py\r\n\r\nIt would be smart to read through usage.txt first for a more detailed tutorial or experiment with\r\nthe working example project provided in the source, available at bitbucket.\r\n\r\n\r\nRequirements\r\n------------\r\n\r\n* Django\r\n* Celery / django-celery\r\n\r\n\r\nExample usage\r\n-------------\r\n\r\nThe simplest processor you can define looks something like::\r\n\r\n from kolibri.core import *\r\n from models import *\r\n\r\n dirty_words = ('foo', 'fudge', 'bar',)\r\n\r\n class RemoveProfanity(Processor):\r\n model = Article\r\n\r\n def process(self, user, article, **kwargs):\r\n for dirty_word in dirty_words:\r\n article.text = article.text.replace(dirty_word,'*'*len(dirty_word))\r\n article.save()\r\n\r\n manager.register.processor(RemoveProfanity())\r\n\r\nIt's a very simple processor which replaces all dirty words, defined in\r\ndirty_words, with * from instances of a model called Article.\r\n\r\nTo create a workflow, connecting a series of processors::\r\n\r\n from kolibri.core import manager\r\n from kolibri.core.workflow import Workflow\r\n\r\n workflow = Workflow('Publish article', model=Article)\r\n workflow.first(RemoveProfanity()).on_exception(ValueError, DirtyWordRemover()).\\\r\n then(PublishArticle()).then(ArchiveArticle())\r\n\r\n manager.register.workflow(workflow)\r\n\r\nHere we create a workflow called \"Publish article\" for the Article-model. First\r\nwe remove all profanity using the RemoveProfanity, if RemoveProfanity raises\r\nan ValueError we run the DirtyWordRemover-processor, then we publish\r\nthe article using a processor called PublishArticle and finally we archive it.\r\n\r\nSee the usage.txt document in the source for more examples and in-depth\r\nexplanation of features.\r\n\r\n\r\nRelease notes\r\n-------------\r\n\r\n* 0.2.0 - support for user input. See bottom of usage description for more info.\r\n* 0.1.1 - Added support for only running a processor once for an instance.\r\n* 0.1.0 - Initial release. Pre-alpha state.\r\n\r\nKolibri usage\r\n-------------\r\n\r\nThis documentation is related to the 0.1.0-release of kolibri and syntax and functionality WILL change\r\nin future releases as long as it is labeled as pre-alpha/alpha. Reaching Beta-status only small changes in code\r\nwill be introduced.\r\n\r\nNB! The following assumes to have installed celery/django-celery and configured it to run as\r\nstated in the celery documentation. The example project in the source is using djkombu, which makes it alot easier to get up and running.\r\n\r\nA simple app called article defines a model::\r\n\r\n class Article(models.Model):\r\n title = models.CharField(max_length=128)\r\n text = models.TextField()\r\n parental_advisory = models.BooleanField(default=False)\r\n author = models.ForeignKey(User, related_name='articles')\r\n publish = models.BooleanField(default=False)\r\n archived = models.BooleanField(default=False)\r\n\r\n def __unicode__(self):\r\n return self.title\r\n\r\nWe define a processor to remove dirty words in a file called processors.py in the\r\nsame app-folder. The name of the file doesn't matter as long as the processor is\r\nregistered using the manager, as shown at the bottom of this snippet of code::\r\n\r\n from kolibri.core import *\r\n from models import *\r\n\r\n dirty_words = ('foo', 'bar', 'fudge',)\r\n\r\n class RemoveProfanity(Processor):\r\n model = Article\r\n\r\n def process(self, user, item, **kwargs):\r\n for dirty_word in dirty_words:\r\n item.text = item.text.replace(dirty_word, '*'*len(dirty_word))\r\n item.save()\r\n\r\n manager.register.processor(RemoveProfanity())\r\n\r\nNote:\r\n\r\n1. You must subclass Processor from kolibri.core\r\n\r\n2. The name of your processor will be used in the admin. Using CamelCase (http://en.wikipedia.org/wiki/CamelCase)\r\nthe name will be transformed into a text more suitable for reading. In our example here RemoveProfanity will become\r\n\"Remove profanity\" in the admin.\r\n\r\n3. All processors you want to show up in the admin must specify a model they're related to.\r\n\r\n4. All processors MUST implement the processor-method and the signature of the method MUST look like the\r\none in the example above, with the exception if the item parameter, which can be called whatever you like. In\r\nthis example it would be nicer to call it article so go ahead.\r\n\r\n5. The processor should not touch any other data than the item provided.\r\n\r\n\r\nSimple testing::\r\n\r\n $ python manage.py shell\r\n >>> from django.contrib.auth.models import User\r\n >>> usr = User.objects.all()[0] # an allready registered user we can use for testing purposes\r\n >>> from article.models import Article\r\n >>> article1 = Article.objects.create(title=\"Dirty words\", text=\"Some dirty words: foo bar fudge.\", author=usr)\r\n >>> from article.processors import RemoveProfanity\r\n >>> RemoveProfanity().process(usr, article1)\r\n >>> article1.text\r\n 'Some dirty words: *** *** *****.'\r\n\r\nTo make it available in the admin, add an admin.py file to your app-folder::\r\n\r\n from django.contrib import admin\r\n from models import *\r\n from kolibri.core import manager\r\n\r\n class ArticleAdmin(admin.ModelAdmin):\r\n fields = ('title', 'text', 'parental_advisory', 'author', 'publish', 'archived', )\r\n list_display = ('title', 'parental_advisory', 'author', 'publish', 'archived', )\r\n list_filter = ('parental_advisory', 'author', 'publish', 'archived', )\r\n actions = manager.admin.actions_for_model(Article)\r\n\r\n admin.site.register(Article, ArticleAdmin)\r\n\r\nThe important part here is the \"actions =\"-line. It assigns all available processors related to the article-model.\r\nNow you can select several articles in the admins change_list and apply a processor to all of them.You can also\r\nmake the processors available in the change_form for an instance, but extending the change_form in your app. Create a\r\ntemplates-folder in your app-folder, with a subfolder called admin, with a subfolder article and inside it put a\r\nfile called change_form.html::\r\n\r\n {% extends \"admin/change_form.html\" %}\r\n {% load kolibri_tags %}\r\n\r\n {% block extrahead %}\r\n \r\n \r\n \r\n\r\n \r\n {% endblock %}\r\n\r\n {% block footer %}\r\n
\r\n {% kolibrify_admin original %}\r\n
\r\n {% endblock %}\r\n\r\nIt will load the kolibri template tags, add some javascript references to jquery and some kolibri js, but the magic\r\nhappens when you call {% kolibrify_admin original %}. It will insert a list of available processors and workflows you can\r\nexecute for the object you're looking at.\r\n\r\nNB! The kolibrify-templatetag changed in version 0.2.0. Inside you'll have to use the kolibrify_admin templatetag to\r\nuse the \"admin/base_site.html\" template to render your page.\r\n\r\nTo make the same list of processors available outside the admin, you can do something like this for a list of objects::\r\n\r\n {% load kolibri_tags %}\r\n \r\n \r\n {% kolibri_imports %}\r\n \r\n\r\n \r\n \r\n\r\n

Articles

\r\n\r\n {% kolibrify articles \"article/article_list.html\" %}\r\n\r\n \r\n \r\n\r\nHere we \"kolibrifies\" a queryset, a bunch of articles, using a template defined in your apps template folder, like so::\r\n\r\n \r\n\r\nThe only requirement for this to work is that you have a value named pk_id_SomethingUniqueForEachItem with value of\r\nthe model.id to apply the processors to. Something like this for a details-page for an article::\r\n\r\n {% load kolibri_tags %}\r\n \r\n \r\n {% kolibri_imports %}\r\n \r\n \r\n \r\n Index\r\n

\r\n

{{ article.title }}

\r\n by {{ article.author }}. {% if article.parental_advisory %}Warning! This article contains explicit language.{% endif %}\r\n {% if article.publish %} This article has been published.{% endif %}\r\n\r\n
{{ article.text }}
\r\n\r\n {% kolibrify article %}\r\n\r\n \r\n \r\n\r\nAnd sometimes we want to do more things with our data, but in a specific order. This is were Workflows comes in::\r\n\r\n workflow = Workflow('Publish article', model=Article)\r\n workflow.first(SetParentalAdvisory()).on_exception(ValueError, RemoveDirtyWords()).\\\r\n then(PublishArticle()).and_finally(CleanUpAfterPublishing())\r\n manager.register.workflow(workflow)\r\n\r\nThis workflow first attempts to mark all articles containing profanity with a \"Parental Advisory\" flag. If that fails\r\nand a ValueError-exception is raised, RemoveDirtyWords will remove all dirty words. You can specify several processors\r\nto handle different exceptions. Then the workflow will publish the article and finally it will do some house cleaning\r\nusing the CleanUpAfterPublishing-processor. When using and_finally(SomeProcessor) that processor always be called last\r\nin a try...finally-block surrounding the other steps in your workflow.\r\n\r\nI know - not a very good example. Hopefully the examples will improve with future releases as well ;-).\r\n\r\nWorkflows are available in the admin and in your templates when you \"kolibrifies\" an instance or queryset, just like\r\nprocessors.\r\n\r\nFinally a processor can be executed not related to a specific model, for insance scanning a filesystem looking for\r\nnew images or sending an email ( for some reason ). To be able to do this we must implement a new method in our processor::\r\n\r\n import os\r\n\r\n class ImportArticleFromHomeFolder(Processor):\r\n model = Article\r\n user_exclusive = True\r\n\r\n def execute(self, user, **kwargs):\r\n path = os.path.join('/home', user.username)\r\n for filename in os.listdir(path):\r\n fname, ext = os.path.splitext(filename)\r\n if ext == '.txt':\r\n Article.objects.create(title=fname, text=open(os.path.join(path, filename)).read(), author=user)\r\n\r\nThe execute-method takes a user-instance and optional kwargs as parameter. The processor is quite dumb\r\nand only scans the users homefolder, adding all files with a .txt-extension to the database using the filecontent\r\nas text for an article and the filename as a title.\r\n\r\nTo execute such processors, click on the \"Kolibri\"-link in the admin, the link taking you to the app-index page. This\r\npage contains what will hopefully become a usable dashboard for kolibri later on. For now it lists registered processors\r\nand workflows, shows a list of pending processors and each listed processor can be click for more details. If you've\r\nimplemented the execute-method as shown above there will be an \"Execute\"-button available. Click it and your test\r\ndatabase will have some articles in it (assuming you got some textfiles in your home folder ).\r\n\r\nThe processor shown above also introduce another concept, user_exclusive. When setting this to true you indicate that\r\nyour processor will only change data related to the user provided as parameter to the execute-method ( or the\r\nprocess-method). This makes it possible to let several users execute or process data without the risk of updating data\r\nrelated to another user.\r\n\r\n\r\n--Getting using input--\r\n\r\nIntroduced in version 0.2.0. To enable your processor to take user input using a form, extend your processor like so::\r\n\r\n class PublishArticle(Processor):\r\n model = Article\r\n has_form = True\r\n form_comment = \"This processor will publish your article on CoolNewsSite.com.\"\r\n\r\n def process(self, user, article,**kwargs):\r\n article.publish = True\r\n article.save()\r\n\r\n def execute(self, user,**kwargs):\r\n for article in Article.objects.all():\r\n self.process(user, article, **kwargs)\r\n\r\n def get_form(self):\r\n class PublishForm(forms.Form):\r\n username = forms.CharField()\r\n password = forms.CharField()\r\n return PublishForm, {}\r\n\r\nNotice the has_form = True line in the beginning and the new get_form-method. This method should return two things;\r\n\r\n* the form, not an instance, but the actual class\r\n* a dictionary for initial values for your form, if any. If no initial data is specified you'll still have to return a dict\r\n\r\nYou can also provide the user with some additional info, perhaps a more detailed explanation of what you expect him/her\r\nto enter in the form. This is done by adding a form_comment attribute like shown above. The form used for form rendering\r\nextends base_site.html so you'll have to define one of those.\r\n\r\nChange to the kolibri templatetags:\r\n\r\n* kolibri_index will no render an index which extends base_site.html. If you want to use it in the admin, call kolibri_index_admin instead.\r\n* kolibrify is now meant to be used outside the admin. Call kolibrify_admin when inside the admin.", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://bitbucket.org/weholt/django-kolibri", "keywords": "django celery asynchronous processing workflow", "license": "Modified BSD", "maintainer": "", "maintainer_email": "", "name": "Django-Kolibri", "package_url": "https://pypi.org/project/Django-Kolibri/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/Django-Kolibri/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://bitbucket.org/weholt/django-kolibri" }, "release_url": "https://pypi.org/project/Django-Kolibri/0.2.0a/", "requires_dist": null, "requires_python": null, "summary": "Asynchronous Processors/Workflow management for django.", "version": "0.2.0a" }, "last_serial": 2122095, "releases": { "0.2.0a": [] }, "urls": [] }