{ "info": { "author": "Trey Morris", "author_email": "trey@treymorris.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: Apache Software License" ], "description": "functastic - retry all the tasks you need\n=========================================\n\nfunctastic is used to manage tasks that you would like to retry until a\nsuccess condition is met, or potentially forever. it can be run single\nthreaded or in a separate thread. task start times, success conditions,\nretry attempts, retry interval, and time interval back off can be\nconfigured. it's important to note that all tasks are run in a single\nthread one after the other, so if you have a longer running task it may\ndelay the actual start time of the task or tasks behind it in line. with\nthis in mind, functastic works better with quick tasks for now. I'd like\nto have it spawn tasks as side threads in the future, but I didn't want\nto require the usage of any particular threading library at present.\n\nfunctastic provides two classes: ``TaskHeap`` and ``Task``. ``Tasks``\nwrap a function and are appended to the ``TaskHeap`` which provides a\n``run()`` function handle running/scheduling/retrying the ``Tasks``\nuntil the success condition is met. ``Task's`` default success condition\nis that the function does not raise any Exception and returns any non\n``None`` value.\n\ncalling a ``Task`` object can never raise an exception. calling a task\neither manually or using a ``TaskHeap`` will only log exceptions and\npotentially use them to determine success, but they won't be raised. the\none exception to this rule is if your custom ``success_condition``\nfunction raises an exception, so be careful writing them. ``Task`` will\nalso raise on initialization if you provide a ``success_condition`` that\nisn't callable.\n\ninstall\n-------\n\n``pip install functastic`` or clone the repo and\n``python setup.py install`` or ``pip install -e`` if you want to play\nwith the code\n\n``Task`` usage\n--------------\n\nthe basic task is a wrapped function that has some attributes for\ndetermining success and when a function should be run. note that\nanything to do with scheduled times, delays, or timeouts are \"best try\".\nlong running tasks can get in the way of the task schedule. the\nconfigurable traits for a task include:\n\n- ``func``, the function to be run\n- ``args``, list of args to pass to the function\n- ``kwargs``, dictionary of keyword args to pass to the function\n- ``attempts``, number of times to retry (set to 0 means until success)\n- ``task_timeout``, the number of seconds the function may be retried\n- ``delay``, the time in between each run of the function (modified by\n backoff)\n- ``backoff``, delay multiplier, extends the delay exponentially each\n iteration. ``backoff = 1`` is standard interval, ``backoff = 2``\n doubles the time in between each retry\n- ``start_time``, the timestamp at which the function will be run the\n first time ex ``time.time() + 30`` run 30 seconds from now\n- ``success_condition``, function used to determine whether the task\n was successful this iteration. defaults to no exceptions raised and a\n non None return value. an exception will be raised during ``Task``\n init if ``success_condition`` isn't callable\n\nhere are a few examples of creating tasks.\n\n.. code:: python\n\n from functastic import Task\n import time\n f = some_function\n # this is the basic task, it will have retry set to True\n # until it returns a non None value and doesn't raise\n task = Task(f, args['a'])\n\n # let's give it only 10 tries. after 10 tries, retry will be set to False\n task = Task(f, args['a'], attempts=10)\n\n # and slow it down a bit (wait 1 second between each attempt).\n # the delay schedules the task again 1 second after the previous\n # attempt. without delay, schedule for task's next run will be immediately\n # TaskHeap manages the schedules if you want to use it\n task = Task(f, args['a'], attempts=10, delay=1)\n\n # and now let's make it backoff if at first it doesn't succeed\n # this will be run at t=[0, 1, 2, 4, 8, 16, 32, 64, 128, 256] seconds\n task = Task(f, args['a'], attempts=10, delay=1, backoff=2)\n\n # another way to think of a task only having a certain number of attempts\n # is to give it a timeout\n # this function will be run approximately every 1 second for 60 seconds\n # CAUTION! long running tasks may delay things, tasks with task_timeout\n # can only be guaranteed to run once\n task = Task(f, args['a'], task_timeout=60, delay=1)\n\n # want to schedule a task to start running 60 seconds from now?\n # note that the task_timeout doesn't start counting until the first run\n # so this function will start running in 60 seconds and retry every 1\n # second for 30 seconds\n task = Task(f, args['a'], start_time=time.time()+60, delay=1,\n task_timeout=30))\n\n # define your own success condition for a task\n task = Task(f, args['a'], delay=1,\n success_condition=lambda t: t.result == 'a')\n # or change it later\n task.success_condition = lambda t: t.result == 'b'\n # you could also define a more involved function instead of lambdas\n def success(task):\n if 'some key' in task.result:\n return True\n\n task = Task(f, args['a'], delay=1, success_condition=success)\n\ntasks which \"succeed\" by failing in some way\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nsometimes it makes sense to think of ``success_condition`` as a\n``break`` condition. let's say there is a function that makes requests\nto a webservice. it makes sense to retry if you get a ``404`` because\nyou are polling for a resource to become available, but if you get\nanother error, like ``401 unauthorized``, retrying wouldn't make sense,\nso we want to ``break`` (or \"succeed\") if we get a non ``404``\nexception:\n\n.. code:: python\n\n import requests\n from functastic import Task\n\n def handle_resource(res):\n pass\n\n def get_thing(url):\n r = requests.get(url)\n r.raise_for_status()\n handle_resource(r.json())\n\n # every 5 seconds attempt to fetch the resource and handle it,\n # quit when there is no exception (actual success) or there is\n # a non 404 exception (call failed in a bad way, so don't try it again)\n task = Task(get_thing, args['http://whatever.com/resource/id'], delay=5,\n success_condition=lambda t: (t.exception and '404' not in t.exception.message or\n not t.exception))\n\nusing tasks without ``TaskHeap``\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: python\n\n from functastic import Task\n\n task = Task(f, args['a'], attempts=10)\n\n # task.retry always starts as True\n while task.retry:\n # calling the task calls the function\n task()\n # at this point, task.retry may have become False depending on the task\n # optionally sleep in between calls\n time.sleep(2)\n\n``RecurringTask`` usage\n-----------------------\n\nrecurring tasks are the for things you want to run again and again\nregardless of outcome. imagine a task with no success condition and in\nfact asides from the log messages being slightly different, you can\nachieve a ``RecurringTask`` logic while using ``Task`` by setting an\nalways fail ``success_condition``. use ``RecurringTask`` only if you\nwant something to be run again and again forever, otherwise if you have\na number of attempts, task timeouts, or success conditions you should\nuse ``Task``.\n\n.. code:: python\n\n from functastic import Task\n from functastic import RecurringTask\n # using Task, set the success condition to `lambda t: None`\n # and do not specify attempts\n task = Task(f, args=['a'], delay=10,\n success_condition=lambda t: None)\n # and the equivalent RecurringTask:\n rtask = RecurringTask(f, args=['a'], delay=10)\n\n # this task will be run 5 times, 10 seconds after the previous run,\n # regardless of what f returns or raises. this behavior cannot be\n # achieved with RecurringTask\n task = Task(f, args=['a'], delay=10, attempts=5,\n success_condition=lambda t: None)\n\n``TaskHeap`` usage\n------------------\n\nputting ``Task`` together with the ``TaskHeap``, I'll use a simple\nfunction that fails pretty often both with Exceptions and return values\n\n.. code:: python\n\n def usually_fails(arg):\n if random.randint(1, 4) != 1:\n raise Exception('everything is ruined')\n if random.randint(1, 4) != 2:\n return None\n print '%s ran at %s' % (arg, datetime.today())\n return arg\n\n``TaskHeap`` is iterable and works as a ``bool`` and ``str(tasks)``\ngives a pretty good output\n\n.. code:: python\n\n from functastic import Task\n from functastic import TaskHeap\n tasks = TaskHeap()\n tasks.append(Task(usually_fails, args=['a'], delay=1))\n tasks.append(Task(usually_fails, args=['b'], attempts=10, delay=1))\n if tasks:\n print len(tasks)\n print str(tasks)\n for task in tasks:\n print task\n\nunthreaded use\n^^^^^^^^^^^^^^\n\nrun a task or set of tasks and return when they finish. without\n``stop=True`` the ``tasks.run()`` call will block forever because it\nwon't stop iterating every ``TaskHeap`` ``interval``.\n\n.. code:: python\n\n from functastic import Task\n from functastic import TaskHeap\n # add tasks and then run run(stop=True)\n tasks = TaskHeap()\n tasks.append(Task(usually_fails, args=['a'], delay=1))\n tasks.append(Task(usually_fails, args=['b'], attempts=10, delay=1))\n tasks.run(stop=True)\n\nuse with threading library\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n``TaskHeap`` works well with threading libraries. this will run the task\nloop in another thread and add tasks willy nilly while they run\n\n.. code:: python\n\n import eventlet\n from functastic import Task\n from functastic import TaskHeap\n # note the use of eventlet.sleep here to specify which sleep\n # function TaskHeap should use, or use monkey patching\n # interval can also be passed if you don't like the default 0.1s\n # this sets the interval task run interval to 3 seconds\n tasks = TaskHeap(sleep=eventlet.sleep, interval=3)\n eventlet.spawn(tasks.run)\n tasks.append(Task(usually_fails, args=['a'], delay=1))\n tasks.append(Task(usually_fails, args=['b'], attempts=10, delay=1))\n\n # have to sleep here to surrender execution to the green thread\n while tasks:\n tasks.sleep()\n\nstopping tasks\n^^^^^^^^^^^^^^\n\nonce a ``TaskHeap`` has been started with ``run()``, it will run\nindefinitely unless ``stop=True`` is passed in ``run(stop=True)``. it\ncan be stopped in two different ways: - ``stop_after()``, causes the\ntask loop to exit once all tasks are completed. it should be fairly\nobvious that with even one ``RecurringTask`` in the heap,\n``stop_after()`` won't do anything. - ``stop_now()``, causes the task\nloop to stop as soon as possible. since the task loop is single\nthreaded, it will only exit after finishing the current iteration. this\nmeans the current task, if there is one, will continue as planned, but\nall future tasks will not be run unless ``run()`` is called again.\n\n.. code:: python\n\n import eventlet\n from functastic import Task\n from functastic import TaskHeap\n\n tasks = TaskHeap(sleep=eventlet.sleep)\n gt = eventlet.spawn(tasks.run)\n tasks.append(Task(usually_fails, args=['a'], delay=1))\n tasks.append(Task(usually_fails, args=['b'], attempts=10, delay=1))\n\n # stop the tasks thread after 5 second, gt.wait() will return almost\n # instantly\n tasks.sleep(5)\n tasks.stop_now()\n gt.wait() # <-- this line should return quickly\n\n # start the tasks again\n gt = eventlet.spawn(tasks.run)\n\n # this time tell the tasks loop to exit once finished\n tasks.sleep()\n tasks.stop_after()\n gt.wait() # <-- this line should return when all tasks complete", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/tr3buchet/functastic", "keywords": "task,tasks,retry,func,functions", "license": "Apache Software License", "maintainer": null, "maintainer_email": null, "name": "functastic", "package_url": "https://pypi.org/project/functastic/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/functastic/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/tr3buchet/functastic" }, "release_url": "https://pypi.org/project/functastic/2.2/", "requires_dist": null, "requires_python": null, "summary": "threadable task retry module", "version": "2.2" }, "last_serial": 2265112, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "8513b893c2e71214b447418df1e3ad2e", "sha256": "204687cdc8519c0de258250cc7a541598af066b5492b03b5545eb8a333559d1a" }, "downloads": -1, "filename": "functastic-1.0.0.tar.gz", "has_sig": false, "md5_digest": "8513b893c2e71214b447418df1e3ad2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9145, "upload_time": "2015-08-31T19:57:04", "url": "https://files.pythonhosted.org/packages/f8/70/4018ce0b747d3e24d62065ea298a1860056b6bee610c53d1107706f8f376/functastic-1.0.0.tar.gz" } ], "1.0.1": [ { "comment_text": "", "digests": { "md5": "9975a4715250b39da6b25042d4564a78", "sha256": "53fa5ccc46ac571c646b41f842a463cf9f19a037b952d4619ccaa328e6f8f0e4" }, "downloads": -1, "filename": "functastic-1.0.1.tar.gz", "has_sig": false, "md5_digest": "9975a4715250b39da6b25042d4564a78", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9327, "upload_time": "2015-09-01T23:06:03", "url": "https://files.pythonhosted.org/packages/fe/ac/6dd75207fe23e1d163066417a1c884da9c8ced91f77ebd06443edcf55729/functastic-1.0.1.tar.gz" } ], "1.0.2": [ { "comment_text": "", "digests": { "md5": "02d1e6768a037f5ba8f3e364155557a2", "sha256": "001109835ee901fc4c540acbb84a1dbfae71f8bdf16a35e9a10108818249c3ba" }, "downloads": -1, "filename": "functastic-1.0.2.tar.gz", "has_sig": false, "md5_digest": "02d1e6768a037f5ba8f3e364155557a2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9344, "upload_time": "2015-09-16T20:02:12", "url": "https://files.pythonhosted.org/packages/bf/3b/85ae1d62ae480b457f5528bd4f9319092b4e9dc26050ee668e55e28ca219/functastic-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "7742791dcadd6f39b961c8b56946ef99", "sha256": "270b346024b46b49e62b1241e957067d408f3a7a6f104863a7f932376b119aca" }, "downloads": -1, "filename": "functastic-1.0.3.tar.gz", "has_sig": false, "md5_digest": "7742791dcadd6f39b961c8b56946ef99", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9348, "upload_time": "2015-09-16T21:08:27", "url": "https://files.pythonhosted.org/packages/be/b1/f6450d4a10306a9220af9575304b47bde48dd7b91ea9870e4efc7b0814ec/functastic-1.0.3.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "e40c3c65a6633b116d1a87d6b74b9654", "sha256": "5aa366e4454b6f8c05f745088482fd4f32df485d5f0b7e1824c4c2fe3f5b2087" }, "downloads": -1, "filename": "functastic-1.1.tar.gz", "has_sig": false, "md5_digest": "e40c3c65a6633b116d1a87d6b74b9654", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10866, "upload_time": "2016-07-14T00:27:07", "url": "https://files.pythonhosted.org/packages/fc/68/6511e0c9f9520d4fe2deb3c100ee84c945a2b360ef453ce17ad2f9346424/functastic-1.1.tar.gz" } ], "2.0": [ { "comment_text": "", "digests": { "md5": "704af158cda0c571d5cd8f7de884fda8", "sha256": "6a787f282811a66a437b106484c213b7d1a30731c146aa961e2dc289d8cc98fb" }, "downloads": -1, "filename": "functastic-2.0.tar.gz", "has_sig": false, "md5_digest": "704af158cda0c571d5cd8f7de884fda8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10854, "upload_time": "2016-07-14T00:28:28", "url": "https://files.pythonhosted.org/packages/dd/6a/b6ef1f7d45e943d7c4e26d3a91072268656156d0c8ef21ba0dadcaed5964/functastic-2.0.tar.gz" } ], "2.1": [ { "comment_text": "", "digests": { "md5": "b62531fabc2a6d32d738080b8179705b", "sha256": "3cb6361677aa2ef91169a8f712829113e273da6945b9fc5f8046c8d5825ac825" }, "downloads": -1, "filename": "functastic-2.1.tar.gz", "has_sig": false, "md5_digest": "b62531fabc2a6d32d738080b8179705b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12019, "upload_time": "2016-07-20T00:35:29", "url": "https://files.pythonhosted.org/packages/9f/15/b0f6ebe69f9a2a537154d8eca9133024bd39bf8e34faa9db5c5aced125d3/functastic-2.1.tar.gz" } ], "2.2": [ { "comment_text": "", "digests": { "md5": "c81f36aa9f899b08e695cd715dfd7861", "sha256": "0969cf0db93b1ee219aefaf0c98bd094daa62ab0a7e057030dd66308d0662c64" }, "downloads": -1, "filename": "functastic-2.2.tar.gz", "has_sig": false, "md5_digest": "c81f36aa9f899b08e695cd715dfd7861", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12529, "upload_time": "2016-08-05T23:08:14", "url": "https://files.pythonhosted.org/packages/c0/b5/91cfd94870eb7d4381b087c2cb524568c360654412244a3716bfd2175993/functastic-2.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "c81f36aa9f899b08e695cd715dfd7861", "sha256": "0969cf0db93b1ee219aefaf0c98bd094daa62ab0a7e057030dd66308d0662c64" }, "downloads": -1, "filename": "functastic-2.2.tar.gz", "has_sig": false, "md5_digest": "c81f36aa9f899b08e695cd715dfd7861", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12529, "upload_time": "2016-08-05T23:08:14", "url": "https://files.pythonhosted.org/packages/c0/b5/91cfd94870eb7d4381b087c2cb524568c360654412244a3716bfd2175993/functastic-2.2.tar.gz" } ] }