{ "info": { "author": "Max Hollmann", "author_email": "maxhollmann@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy" ], "description": "\nProvides a decorator for Luigi tasks that allows them to fail *softly*. Tasks that fail softly are seen by Luigi as complete, and thus don't prevent dependent tasks from running. The dependent task can check if it's dependencies actually ran successfully or failed softly.\n\n\n# Installation\n\n pip install luigi_soft_failures\n\n\n# Usage\n\n```python\nfrom luigi_soft_failures import softly_failing\n```\n\nUse as class decorator:\n\n```python\n@softly_failing(catch_all=True, propagate=True)\nclass SomeTask(luigi.Task):\n ...\n```\n\nOr on demand in the requiring task:\n\n```python\ndef requires(self):\n return softly_failing(catch_all=True)(SomeTask)(some_param=42)\n```\n\nThe dependent task can check the status of it's dependencies using `failed_softly`:\n\n```python\ndef run(self):\n if self.requires().failed_softly():\n ...\n```\n\nFor a complete example see [as_decorator.py](https://github.com/maxhollmann/luigi_soft_failures/blob/master/examples/as_decorator.py).\n\n\n## API\n\n`softly_failing` accepts the following parameters:\n\n* `catch_all` (`bool`, default `False`):\n\n When `True`, any exception thrown in the task's `run` method will lead to a soft failure. Otherwise, soft failures can be generated manually by calling `self.fail_softly('Some error message')` from the task's `run` method and exiting the method without exception.\n\n* `propagate` (`bool`, default `False`):\n\n When `True`, the task fails softly if any of it's dependencies failed softly, and `run` is never executed. Otherwise, `run` is executed as if the dependencies ran successfully.\n\n* `output_dir` (`str`, default `None`): Described [below](#storage-of-soft-failure-reports).\n\n\n## Storage of soft failure reports\n\nWhan the wrapped task fails softly, it creates a report with the failure message or exception traceback to indicate this. These reports are stored in a directory specified in one of the following ways (in this order of precedence):\n\n* `output_dir` parameter passed to `softly_failing`\n* Specified in `luigi.cfg`:\n\n [luigi_soft_failures.Config]\n output_dir=/some/path\n\n* Environment variable `LUIGI_SOFT_FAILURES_OUTPUT_DIR`\n* Default `./soft_failures/`\n\n\n# Limitations\n\n* Soft failure status is stored using a `LocalTarget`, so a local storage that all workers have access to is required.\n* In the Luigi visualizer, softly failed tasks are shown as complete.\n\nPull requests to address these are very welcome!\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/maxhollmann/luigi_soft_failures", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "luigi-soft-failures", "package_url": "https://pypi.org/project/luigi-soft-failures/", "platform": "", "project_url": "https://pypi.org/project/luigi-soft-failures/", "project_urls": { "Homepage": "https://github.com/maxhollmann/luigi_soft_failures" }, "release_url": "https://pypi.org/project/luigi-soft-failures/0.1.1/", "requires_dist": [ "luigi" ], "requires_python": ">=3.6.0", "summary": "Allow Luigi tasks to fail softly, without cancelling downstream tasks.", "version": "0.1.1" }, "last_serial": 4555453, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "badb6fa83d52cd47c2e24234d310d930", "sha256": "ef92c34b66756e57110d512386bc524ee8d49cd363ceb824fd68aeeedbb73d55" }, "downloads": -1, "filename": "luigi_soft_failures-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "badb6fa83d52cd47c2e24234d310d930", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 5292, "upload_time": "2018-12-03T11:58:11", "url": "https://files.pythonhosted.org/packages/8d/58/01e2b16cd6f130db55284ca7de01e09e808a89274fa98dd2bc0020ea3a51/luigi_soft_failures-0.1.0-py2.py3-none-any.whl" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "ee95cee622438bcd1167e9fd470f4538", "sha256": "3da2aedc41696c5f791879fe19ab1a7beec1571c9998eeea19cec7909d2e0639" }, "downloads": -1, "filename": "luigi_soft_failures-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ee95cee622438bcd1167e9fd470f4538", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 5294, "upload_time": "2018-12-03T11:59:52", "url": "https://files.pythonhosted.org/packages/74/6d/0856f9ad304afd24a650545fdee20aa3355784dbe868162fd6b1b5f8346d/luigi_soft_failures-0.1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "781bcec85261c094ffedf54ae0ecfb1b", "sha256": "7ca408cc52de8123390ef573ce1641dcd6b6e7b83b70d35233f2af41955cd8ce" }, "downloads": -1, "filename": "luigi_soft_failures-0.1.1.tar.gz", "has_sig": false, "md5_digest": "781bcec85261c094ffedf54ae0ecfb1b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 5383, "upload_time": "2018-12-03T11:59:53", "url": "https://files.pythonhosted.org/packages/c9/2e/1eff9987869e29d87304cc6e5662db9481a6b761a4f2151fd2d799acf148/luigi_soft_failures-0.1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ee95cee622438bcd1167e9fd470f4538", "sha256": "3da2aedc41696c5f791879fe19ab1a7beec1571c9998eeea19cec7909d2e0639" }, "downloads": -1, "filename": "luigi_soft_failures-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ee95cee622438bcd1167e9fd470f4538", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 5294, "upload_time": "2018-12-03T11:59:52", "url": "https://files.pythonhosted.org/packages/74/6d/0856f9ad304afd24a650545fdee20aa3355784dbe868162fd6b1b5f8346d/luigi_soft_failures-0.1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "781bcec85261c094ffedf54ae0ecfb1b", "sha256": "7ca408cc52de8123390ef573ce1641dcd6b6e7b83b70d35233f2af41955cd8ce" }, "downloads": -1, "filename": "luigi_soft_failures-0.1.1.tar.gz", "has_sig": false, "md5_digest": "781bcec85261c094ffedf54ae0ecfb1b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 5383, "upload_time": "2018-12-03T11:59:53", "url": "https://files.pythonhosted.org/packages/c9/2e/1eff9987869e29d87304cc6e5662db9481a6b761a4f2151fd2d799acf148/luigi_soft_failures-0.1.1.tar.gz" } ] }