{ "info": { "author": "mayansalama", "author_email": "micsalama@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7" ], "description": "# Windmill\n\n[![Build Status](https://travis-ci.org/mayansalama/windmill.svg?branch=master)](https://travis-ci.org/mayansalama/windmill)\n\nDrag'n'drop web app to manage and create Airflow DAGs. The goal is to\nhave a Web UI that can generate YML Dag Definitions, integrating with\ncustom operators and potentially existing DAGs. YML DAGs can then be\nsynced to a remote repo\n\n- Front end is built using React on Typescript\n- Back end is built using Flask on Python 3.6+\n\n## Getting Started\n\n1. Install with `pip install airflow-windmill`\n 1. Airflow is expected to be installed on the system. This allows Windmill to run with arbitrary versions of Airflow\n 2. Otherwise it can be packaged with windmill using `pip install airflow-windmill[airflow]`. The version is defined in `pyproject.toml`\n2. Run `windmill init` to create a local Windmill project\n3. `cd windmill-project`\n4. Run `windmill run` from this folder to run the app locally\n\n## MVP Required Features\n\n### Front-End Features\n\n- [x] Dynamic Operators\n- [x] Menu Dropdowns\n- [x] Load Operators from App\n- [x] Format operator display into classes\n- [x] Search functionality for operators\n- [x] Basic operator level properties\n- [x] Implement DAG level properties\n- [x] New DAG Functionality\n- [x] Parameter Tooltips\n- [x] Render arbitrary viewport windows for New/Save/Load etc\n- [x] Overwrite/Save prompt on New\n- [x] DAG renaming and save functionality\n- [x] Open dag from menu\n- [x] Make save/load more efficient by removing non-essential values\n- [x] Switch nav menu to icons \n- [x] Add convert DAG call\n- [ ] Add hotkeys to menu functions\n- [ ] Make input/output nodes more clear\n- [ ] Check if file already exists on rename\n- [ ] Prompt save if there are nodes on open\n- [x] Fix loss of state on refresh bug\n- [ ] Put File details in File Browse\n- [ ] Make Flask Backend URI configurable\n- [ ] Add a last saved time to NavBar\n- [ ] Add error handling to backend calls\n- [ ] Add tests\n- [ ] Get task descriptions from Operator list\n\n### Back-End Features\n\n- [x] Generate Operator Lists\n- [x] CLI to start Web and Front End\n- [x] Generate DAG Spec\n- [x] CLI to create new windmill project\n- [x] CLI to start windmill from a windmill project\n- [x] Implement windmill-dev start\n- [x] Save/Load Windmill Files functionality\n- [x] Get default values\n- [x] Pull parameters from parent classes\n- [x] Move airflow dependency as extra\n- [x] Convert WML into Python DAG\n- [x] API Endpoint to trigger WML -> DAG\n- [ ] Edge cases for WML -> DAG\n- [ ] Get WML owner and last-modified details during wml list\n- [ ] Allow custom operators\n- [ ] Add defaults to CLI --help commands\n- [ ] Strategy for Python Opjects (e.g. callables) - allow either a import ref or an inline statement\n- [ ] Backport existing Python DAGs to WMLs\n- [ ] Allow DAG updates to propogate to WMLs\n- [ ] Add tests for different airflow version\n\n### Other features\n\n- [ ] Validate on backend or front end or both?\n- [ ] Doco\n- [ ] Add permission restrictions for valid tags \n\n## Dev User Guide\n\nTo run as a dev:\n\n1. Clone from git\n2. Run `poetry install -E airflow`\n3. Run `windmill-dev start-backend`\n4. Run `windmill-dev start-frontend`\n\n## Future Usage Patterns\n\n- Auto-sync for windmill project to git\n\n## Deployment\n\nDeployment to PyPi is managed using Travis and should be done in the following steps:\n\n1. Run `poetry version {patch|minor|major}`\n2. Increment the version number in `windmill/__init__.py`\n3. Commit and merge code into the master branch\n4. Ensure that the travis build is green\n5. Create a git tag for the new build\n6. Push the tag to origin\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/mayansalama/windmill", "keywords": "", "license": "Apache-2.0", "maintainer": "mayansalama", "maintainer_email": "micsalama@gmail.com", "name": "airflow-windmill", "package_url": "https://pypi.org/project/airflow-windmill/", "platform": "", "project_url": "https://pypi.org/project/airflow-windmill/", "project_urls": { "Homepage": "https://github.com/mayansalama/windmill", "Repository": "https://github.com/mayansalama/windmill" }, "release_url": "https://pypi.org/project/airflow-windmill/0.0.2/", "requires_dist": [ "marshmallow (>=3.0,<4.0)", "docstring-parser (>=0.3.0,<0.4.0)", "decli (>=0.5.1,<0.6.0)", "flask-cors (>=3.0.8,<4.0.0)", "pyyaml (>=5.1.2,<6.0.0)", "networkx (>=2.3,<3.0)", "jinja2 (>=2.10.1,<3.0.0)", "inflection (>=0.3.1,<0.4.0)", "black (>=18.3-alpha.0,<19.0)", "apache-airflow (==1.10.4); extra == \"airflow\"" ], "requires_python": ">=3.6,<4.0", "summary": "Drag'N'Drop Web Frontend for Building and Managing Airflow DAGs", "version": "0.0.2" }, "last_serial": 5882709, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "9fb6ae5e7d91443f57cecc7c6cb92fd0", "sha256": "1f96b9d37366d0c37e5710f5002c4258deab2a01ee98f291068f5e3357323726" }, "downloads": -1, "filename": "airflow_windmill-0.0.1-py3-none-any.whl", "has_sig": false, "md5_digest": "9fb6ae5e7d91443f57cecc7c6cb92fd0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 1479443, "upload_time": "2019-09-22T07:33:50", "url": "https://files.pythonhosted.org/packages/c9/7c/cbeccc883b71353296f09c69f1d6b59f120bab996b200c75b517c4871abe/airflow_windmill-0.0.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "21408e709a8e15e80120a1162efebb6a", "sha256": "419a12b427bb92a8275023ce98f04a5333fc9a931906c221baeb9da8db96bc9e" }, "downloads": -1, "filename": "airflow-windmill-0.0.1.tar.gz", "has_sig": false, "md5_digest": "21408e709a8e15e80120a1162efebb6a", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 1456846, "upload_time": "2019-09-22T07:33:33", "url": "https://files.pythonhosted.org/packages/c4/0f/4804a59e35e1456ee44f0033ecf5ce0dde88767257c6fa5b8f1e18789c8b/airflow-windmill-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "50e5b38623a665e630f40bb41222c7d0", "sha256": "fd96f1797759210073bd5cce65284ff43d651c7ae0cc62a1b297a3b0edb74493" }, "downloads": -1, "filename": "airflow_windmill-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "50e5b38623a665e630f40bb41222c7d0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 1488583, "upload_time": "2019-09-25T02:07:20", "url": "https://files.pythonhosted.org/packages/7a/1e/f59d8da19c362598b050939ec1731120861b34c7eff857a454ef71687fa4/airflow_windmill-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "30f56976387eb44481c2dda0534068bb", "sha256": "56a05a048d32e2537654521a9b7a2fd13519306b576a6ca25b99fa90c967e799" }, "downloads": -1, "filename": "airflow-windmill-0.0.2.tar.gz", "has_sig": false, "md5_digest": "30f56976387eb44481c2dda0534068bb", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 1463156, "upload_time": "2019-09-25T02:07:17", "url": "https://files.pythonhosted.org/packages/a3/81/68ce98632e4563203bb4def4959da0e0edaf65004b8db8609a41c0d0e7c1/airflow-windmill-0.0.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "50e5b38623a665e630f40bb41222c7d0", "sha256": "fd96f1797759210073bd5cce65284ff43d651c7ae0cc62a1b297a3b0edb74493" }, "downloads": -1, "filename": "airflow_windmill-0.0.2-py3-none-any.whl", "has_sig": false, "md5_digest": "50e5b38623a665e630f40bb41222c7d0", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6,<4.0", "size": 1488583, "upload_time": "2019-09-25T02:07:20", "url": "https://files.pythonhosted.org/packages/7a/1e/f59d8da19c362598b050939ec1731120861b34c7eff857a454ef71687fa4/airflow_windmill-0.0.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "30f56976387eb44481c2dda0534068bb", "sha256": "56a05a048d32e2537654521a9b7a2fd13519306b576a6ca25b99fa90c967e799" }, "downloads": -1, "filename": "airflow-windmill-0.0.2.tar.gz", "has_sig": false, "md5_digest": "30f56976387eb44481c2dda0534068bb", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6,<4.0", "size": 1463156, "upload_time": "2019-09-25T02:07:17", "url": "https://files.pythonhosted.org/packages/a3/81/68ce98632e4563203bb4def4959da0e0edaf65004b8db8609a41c0d0e7c1/airflow-windmill-0.0.2.tar.gz" } ] }