{ "info": { "author": "Adam Boscarino", "author_email": "adam@boscarino.me", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy" ], "description": "\n# dag-factory\n\n[![Travis CI](https://img.shields.io/travis/ajbosco/dag-factory.svg)](https://travis-ci.org/ajbosco/dag-factory)\n[![Coverage](https://img.shields.io/coveralls/github/ajbosco/dag-factory.svg)](https://coveralls.io/github/ajbosco/dag-factory)\n[![PyPi](https://img.shields.io/pypi/v/dag-factory.svg)](https://pypi.org/project/dag-factory/)\n[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)\n[![Downloads](https://pepy.tech/badge/dag-factory)](https://pepy.tech/project/dag-factory)\n\n*dag-factory* is a library for dynamically generating [Apache Airflow](https://github.com/apache/incubator-airflow) DAGs from YAML configuration files.\n- [Installation](#installation)\n- [Usage](#usage)\n- [Benefits](#benefits)\n- [Contributing](#contributing)\n\n## Installation\n\nTo install *dag-factory* run `pip install dag-factory`. It requires Python 3.6.0+ and Apache Airflow 1.9+.\n\n## Usage\n\nAfter installing *dag-factory* in your Airflow environment, there are two steps to creating DAGs. First, we need to create a YAML configuration file. For example:\n\n```\nexample_dag1:\n default_args:\n owner: 'example_owner'\n start_date: 2018-01-01\n schedule_interval: '0 3 * * *'\n description: 'this is an example dag!'\n tasks:\n task_1:\n operator: airflow.operators.bash_operator.BashOperator\n bash_command: 'echo 1'\n task_2:\n operator: airflow.operators.bash_operator.BashOperator\n bash_command: 'echo 2'\n dependencies: [task_1]\n task_3:\n operator: airflow.operators.bash_operator.BashOperator\n bash_command: 'echo 3'\n dependencies: [task_1]\n```\n\nThen in the DAGs folder in your Airflow environment you need to create a python file like this:\n\n```\nfrom airflow import DAG\nimport dagfactory\n\ndag_factory = dagfactory.DagFactory(\"/path/to/dags/config_file.yml\")\n\ndag_factory.generate_dags(globals())\n```\n\nAnd this DAG will be generated and ready to run in Airflow!\n\n![screenshot](/img/example_dag.png)\n\n## Benefits\n\n* Construct DAGs without knowing Python\n* Construct DAGs without learning Airflow primitives\n* Avoid duplicative code\n* Everyone loves YAML! ;)\n\n## Contributing\n\nContributions are welcome! Just submit a Pull Request or Github Issue.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ajbosco/dag-factory", "keywords": "airflow", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "dag-factory", "package_url": "https://pypi.org/project/dag-factory/", "platform": "", "project_url": "https://pypi.org/project/dag-factory/", "project_urls": { "Homepage": "https://github.com/ajbosco/dag-factory" }, "release_url": "https://pypi.org/project/dag-factory/0.3.0/", "requires_dist": [ "apache-airflow (>=1.9.0)", "pyyaml", "black; extra == 'dev'", "pytest; extra == 'dev'", "pylint; extra == 'dev'", "pytest-cov; extra == 'dev'" ], "requires_python": ">=3.6.0", "summary": "Dynamically build Airflow DAGs from YAML files", "version": "0.3.0" }, "last_serial": 5962387, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "cc471434dd9ffdeaa69b5360db6d32cc", "sha256": "dfe66ae62d85ec7b4e317d3f03c557315a399fbf6faf864408bae3b0df9b33de" }, "downloads": -1, "filename": "dag-factory-0.1.0.tar.gz", "has_sig": false, "md5_digest": "cc471434dd9ffdeaa69b5360db6d32cc", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 6205, "upload_time": "2018-11-21T01:54:47", "url": "https://files.pythonhosted.org/packages/df/cc/ce253c4615836ff7db459932fd8b099de359fae053be13fb80420172a3e8/dag-factory-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "e0a160b6fc728310ea6744cf530a805e", "sha256": "f994a0c351f4a5cf78b1c2c27ef44185b65d743e958662d9926ef39adf6ba1f7" }, "downloads": -1, "filename": "dag_factory-0.1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e0a160b6fc728310ea6744cf530a805e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 5575, "upload_time": "2018-11-21T02:18:12", "url": "https://files.pythonhosted.org/packages/b0/67/6a2bf6b6c8fbdf70bdc0c627f40e967680fd38486607c0d7ea46d7678c40/dag_factory-0.1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "f378788c189e8d8d96d20de840ff889c", "sha256": "ce6d4151a231c8eeba22a2186265056e0ebb22d43d2505e4aaa762228c8c4c64" }, "downloads": -1, "filename": "dag-factory-0.1.1.tar.gz", "has_sig": false, "md5_digest": "f378788c189e8d8d96d20de840ff889c", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 6102, "upload_time": "2018-11-21T02:18:13", "url": "https://files.pythonhosted.org/packages/c9/57/0a46c681638666531ad4763fe78c30d612f49519e7a8aae773476361b971/dag-factory-0.1.1.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "aafe7e8ff9ce5d3217a7ec22078ec632", "sha256": "0f099212378c47b8b0decb69ae066ad12f120ddbf77d1b588805bb9ce10b56e0" }, "downloads": -1, "filename": "dag_factory-0.2.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "aafe7e8ff9ce5d3217a7ec22078ec632", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 5753, "upload_time": "2018-11-28T18:06:58", "url": "https://files.pythonhosted.org/packages/b1/45/3d23231cf8dafc3951c45af31b501aa66f21d80af822f4dd13ced3fc16cd/dag_factory-0.2.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "56ea5965a40adb3729ac4ea47bf84f63", "sha256": "eff67870a2926714a294db73229859164d9072fab78cb771cdf170034aac4724" }, "downloads": -1, "filename": "dag-factory-0.2.0.tar.gz", "has_sig": false, "md5_digest": "56ea5965a40adb3729ac4ea47bf84f63", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 6251, "upload_time": "2018-11-28T18:07:00", "url": "https://files.pythonhosted.org/packages/e1/ff/c24bdaccebcb3af7bb402a90ec221d6809b8bc6164fb86b206b8338427ee/dag-factory-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "02f88566e2a0b43c3d4eecf7ba1736ab", "sha256": "d757adad56fd8ade4b4447d1ca92cbb75b426033fa3eabd947695fad5b8b2106" }, "downloads": -1, "filename": "dag_factory-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "02f88566e2a0b43c3d4eecf7ba1736ab", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 6386, "upload_time": "2019-02-26T21:10:49", "url": "https://files.pythonhosted.org/packages/95/03/0c5398b938cc4cd3706ea88e9236b16c204e40984f199e8229e80020c76b/dag_factory-0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e3b5136d232abe71c0343c645fdc2d7a", "sha256": "e30953c433bc30ec8f4de6e2f029d94708cc8d5a55e2396c1cdadf0aae92dd64" }, "downloads": -1, "filename": "dag-factory-0.2.1.tar.gz", "has_sig": false, "md5_digest": "e3b5136d232abe71c0343c645fdc2d7a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 6776, "upload_time": "2019-02-26T21:10:50", "url": "https://files.pythonhosted.org/packages/2f/4d/c134102d8e951f33df1983ff4d1aa6cdc75e5e5aa1f3b31dd8d9adecee8e/dag-factory-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "af24d47792b95f39330f2cd6422ac9cc", "sha256": "0ca5c4c53e8d059064963df1cdfc1651fe452ef1e877dfe0aebef1b9f767607d" }, "downloads": -1, "filename": "dag_factory-0.2.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "af24d47792b95f39330f2cd6422ac9cc", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 6455, "upload_time": "2019-09-08T20:38:26", "url": "https://files.pythonhosted.org/packages/ad/e1/1f6b9239429306b70277552c87a8c7dffc7db83ea53bec05679875f29106/dag_factory-0.2.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8fa5a8360b1ff098eea90e5de1b9303b", "sha256": "e7ff8e4c2df851fae946ce7dbb2085302482faf8eaf4234b0a393e21c2067587" }, "downloads": -1, "filename": "dag-factory-0.2.2.tar.gz", "has_sig": false, "md5_digest": "8fa5a8360b1ff098eea90e5de1b9303b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 6865, "upload_time": "2019-09-08T20:38:27", "url": "https://files.pythonhosted.org/packages/0b/81/792bb295d5803edd9e01275c610b72319f2914617af7dda6aa2171369dca/dag-factory-0.2.2.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "cadc1b6494a32a31ed55cd60d2a9b067", "sha256": "59f1418483638d96b4ea44d65c2757f89c38122572c01ead8407620c7673bf3a" }, "downloads": -1, "filename": "dag_factory-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "cadc1b6494a32a31ed55cd60d2a9b067", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 7266, "upload_time": "2019-10-12T00:19:55", "url": "https://files.pythonhosted.org/packages/98/2a/7fb5333d610a3ac295422987e6b45469a7bfabdc6cce2c6aa9b0d4994b43/dag_factory-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "fc5cae1eb7dda34ac226a216a3158276", "sha256": "681ee898ea645d36f46731dc5ea3039ab331161d00deea2a73ecb0bfaef0a113" }, "downloads": -1, "filename": "dag-factory-0.3.0.tar.gz", "has_sig": false, "md5_digest": "fc5cae1eb7dda34ac226a216a3158276", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 7518, "upload_time": "2019-10-12T00:19:56", "url": "https://files.pythonhosted.org/packages/43/cd/15b5156a449e094b17b6c0170536a49e905a403fcad484e4f44795069627/dag-factory-0.3.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "cadc1b6494a32a31ed55cd60d2a9b067", "sha256": "59f1418483638d96b4ea44d65c2757f89c38122572c01ead8407620c7673bf3a" }, "downloads": -1, "filename": "dag_factory-0.3.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "cadc1b6494a32a31ed55cd60d2a9b067", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=3.6.0", "size": 7266, "upload_time": "2019-10-12T00:19:55", "url": "https://files.pythonhosted.org/packages/98/2a/7fb5333d610a3ac295422987e6b45469a7bfabdc6cce2c6aa9b0d4994b43/dag_factory-0.3.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "fc5cae1eb7dda34ac226a216a3158276", "sha256": "681ee898ea645d36f46731dc5ea3039ab331161d00deea2a73ecb0bfaef0a113" }, "downloads": -1, "filename": "dag-factory-0.3.0.tar.gz", "has_sig": false, "md5_digest": "fc5cae1eb7dda34ac226a216a3158276", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.0", "size": 7518, "upload_time": "2019-10-12T00:19:56", "url": "https://files.pythonhosted.org/packages/43/cd/15b5156a449e094b17b6c0170536a49e905a403fcad484e4f44795069627/dag-factory-0.3.0.tar.gz" } ] }