{ "info": { "author": "Telegraph Data Platform team", "author_email": "", "bugtrack_url": null, "classifiers": [], "description": "Azkaban Orchestrator\n====================\n\nAn Orchestrator for Azkaban pipelines\n\nQuick Start\n----------\n.. code-block:: console\n\n pip install azkaban-orchestrator\n\nDefine Orchestration logic\n--------------------------\nCreate a file and define the dependencies between pipelines using the following notation.\n\n* Pipeline name is a project name in Azkaban. Project and flow name of a pipeline should be the same.\n* use '->' for **hard dependency** between two pipelines. A -> B means B runs if A runs successfully.\n* use '.>' for **soft dependency** between two pipelines. A .> B means B runs after A despite A runs successfully or not.\n* if pipeline has some **parameters** put them in parentheses by the pipeline name like A(status=1|date). use vertical bar '|' to separate the parameters.\n* to define **clusters** use colon ':' e.g. S: A,B,C means cluster S includes pipelines A,B and C. to move from one cluster all the pipelines within the cluster should run successfully.\n\nSample 1\n--------\n\n.. code-block:: console\n\n ----------\n | a b | s1\n ----------\n |\n v\n ----------\n | c d | s2\n ----------\n |\n v\n e\n\nSample1 diagram file\n\n.. code-block:: console\n\n s1 : a, b\n s2 : c, d\n s1 -> s2\n s2 -> e\n\n \nSample 2\n--------\n\n.. code-block:: console\n\n a -> b -> d(date)\n | ___/|\n | | |\n v v v\n c -> e <- f(status=1)\n |\n v\n f(status=2)\n\nSample2 diagram file\n\n.. code-block:: console\n\n a -> b\n b -> d(date)\n d(date) -> f(status=1)\n d(date) -> e\n f(status=1) -> e\n a -> c\n c -> e\n c -> f(status=2)\n\n\nUsage\n-----\nTo run the orchestrator\n\n.. code:: python\n\n import logging\n from azkaban_orchestrator import orchestrator\n\n client = orchestrator.Client(\n diagram_file_name='/path/to/diargam_file',\n host='azkaban_host',\n username='azkaban_username',\n password='azkaban_passwpord',\n logger=logging.getLogger(__name__)\n )\n\n # define the parameters need to pass to the orchestrator\n params = {'date':'20171202'}\n\n # define the initial pipeline\n # if you need to start orchestration from a specific pipeline\n initial = None\n\n client.run(initial, params)\n\nTo draw the diagram\n\n.. code:: python\n\n from azkaban_orchestrator import diagram\n\n d = diagram.Diagram('test diagram', 'path/to/diagram_file')\n d.show()", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/telegraph/azkaban-orchestrator", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "azkaban-orchestrator", "package_url": "https://pypi.org/project/azkaban-orchestrator/", "platform": "", "project_url": "https://pypi.org/project/azkaban-orchestrator/", "project_urls": { "Homepage": "https://github.com/telegraph/azkaban-orchestrator" }, "release_url": "https://pypi.org/project/azkaban-orchestrator/0.0.1/", "requires_dist": null, "requires_python": "", "summary": "Azkaban Orchestrator", "version": "0.0.1" }, "last_serial": 3167761, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "8494a64f25ad997cbc0126fdc4881327", "sha256": "9bab558b46cf2a3dcf58e3a1b079f8995deb2a5928069a0115adbb785ed518df" }, "downloads": -1, "filename": "azkaban-orchestrator-0.0.1.tar.gz", "has_sig": false, "md5_digest": "8494a64f25ad997cbc0126fdc4881327", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6823, "upload_time": "2017-09-12T11:37:31", "url": "https://files.pythonhosted.org/packages/7f/5b/8a3da6d982c08892509e86be4daafea976b4344ac7095a94232c03ae5d15/azkaban-orchestrator-0.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "8494a64f25ad997cbc0126fdc4881327", "sha256": "9bab558b46cf2a3dcf58e3a1b079f8995deb2a5928069a0115adbb785ed518df" }, "downloads": -1, "filename": "azkaban-orchestrator-0.0.1.tar.gz", "has_sig": false, "md5_digest": "8494a64f25ad997cbc0126fdc4881327", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6823, "upload_time": "2017-09-12T11:37:31", "url": "https://files.pythonhosted.org/packages/7f/5b/8a3da6d982c08892509e86be4daafea976b4344ac7095a94232c03ae5d15/azkaban-orchestrator-0.0.1.tar.gz" } ] }