{ "info": { "author": "Adam Jenkins", "author_email": "adam@thejenkins.org", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2.7", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# pyflow\n\nA Python workflow framework based on the AWS [Simple Workflow Service][SWF].\n\n[SWF]: https://aws.amazon.com/documentation/swf/\n\n## Summary\n\nPyflow is a Python library which supports defining distributed\nasynchronous workflow applications using ordinary procedural python\ncode. It is implemented using [SWF]. Workflow components can can be\nimplemented as AWS Lambda functions, or activity functions implemented\nin Python, Ruby or Java which run on any computing resource capable of\nconnecting to SWF.\n\nPyflow is heavily inspired by the AWS [Flow Framework for Java][Java Flow]\nand [Flow Framework for Ruby][Ruby Flow], but makes no attempt to be compatible\nwith either of those frameworks.\n\n[Java Flow]: http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/welcome.html\n[Ruby Flow]: http://docs.aws.amazon.com/amazonswf/latest/awsrbflowguide/\n\n## Programming Model\n\nThis page gives a good overview of the concepts used in a pyflow\napplication: [AWS Flow Framework Basic Concepts][]. It's\nabout [Java Flow][] but the concepts are the same for pyflow. The\ndiagram on that page doesn't mention Lambda functions, but if they\nwere added to the diagram, the Lambda service would be another\nactivity worker, with Lambda functions being the activity methods.\n\n[AWS Flow Framework Basic Concepts]: http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/awsflow-basics-application-structure.html\n\n\n### Implementing a Workflow\n\nTo implement a workflow, you subclass the `pyflow.Workflow` class, and\nimplement the `run` method to define the workflow's behavior.\n\n``` python\nimport pyflow\n\nclass MyWorkflow(pyflow.Workflow):\n NAME = 'MyWorkflow'\n VERSION = '1.0'\n \n some_func = pyflow.LambdaDescriptor('some_lambda_func')\n other_func = pyflow.LambdaDescriptor('other_lambda_func')\n \n def run(self, input_arg):\n if input_arg == 'bad_input':\n raise pyflow.WorkflowFailedException('BAD_INPUT', 'Received bad input')\n \n future1 = self.some_func(input_arg)\n \n x = future1.result() + 2\n \n future2 = self.other_func(x)\n \n return future2.result()\n```\n\nThe `input_arg` argument to the `run` method is an arbitrary value\nthat can be passed to the workflow when invoking it. The workflow\ninstance has an `swf` attribute, which is\na [WorkflowInvocationHelper](./pyflow/workflow_invocation_helper.py)\nobject that provides the interface for invoking remote tasks and\nretrieving information about the workflow execution context.\n\nThe example above demonstrates invoking lambda functions. You first\ndefine a class attribute for each lambda function you want to invoke,\nand then use it like a method inside the `run` method. There are\nsimilar descriptor classes for invoking SWF activities, and other SWF\nworkflows. These methods are asynchronous. They immediately return a\n`Future` object, which can be used to retrieve the result of the\ninvocation when it is done. Calling the `result()` method on a future\n\"blocks\" until the result is ready. If the invocation succeeded, its\nresult will be returned. If the invocation failed, the `result`\nmethod will raise an `InvocationException`.\n\nBlocking methods such as `Future.result()` don't actually block the\npython process, but rather allow control to transfer back to the\nWorkflow Worker process so it can process other SWF events. I'll\nexplain later in this document how the context switching is\nimplemented, as well as some rules you need to follow in your workflow\nimplementation code as a result of this implementation.\n\nThe code above also demonstrates how to signal a workflow failure.\nRaise the `WorkflowFailedException` with two arguments. The first is\na short `reason` string, and the second is a longer `details` string.\n\n\n### Executing a Workflow\n\nTo execute a worklow you need to do two things. First you need to\nstart a Workflow Worker process, which will manage the execution of\none or more workflow definitions, and then you need to tell SWF to\ninvoke a workflow. Here is how to create a workflow worker with SWF.\n\n``` python\ndomain = 'SWFSampleDomain'\ntask_list = 'my-workflow-tasklist'\n\n\n# Will poll indefinitely for events\npyflow.poll_for_executions([MyWorkflow], domain=domain, task_list=task_list,\n identity='My Workflow Worker')\n```\n\nExecuting the above will first ensure that the workflow type is\nregistered with SWF, and then enter an endless loop waiting to receive\nevents from the SWF service and executing workflow instances.\n\nOptionally, a `max_time` parameter can be passed to\n`poll_for_executions` to make it only perform `max_time` seconds\nbefore returning. The workflow runner is stateless; execution state\nof workflow instances is maintained by the SWF service. This means\nit's possible to have `poll_for_executions` process several events,\nthen exit the python process and start it again with the same\narguments, and have it pick up workflow executions where it left off.\n\nTo actually start a workflow instance, you can run code like this:\n\n``` python\ndomain = 'SWFSampleDomain'\ntask_list = 'my-workflow-tasklist'\nworkflow_name = 'MyWorkflow'\nworkflow_version = '1.0'\nlambda_role='arn:aws:iam::528461152743:role/swf-lambda'\n\nworkflow_id = pyflow.start_workflow(\n domain=domain,\n workflow_name=workflow_name,\n workflow_version=workflow_version,\n task_list=task_list,\n lambda_role=lambda_role,\n input='\"Hello\"')\n\nprint \"Workflow started with workflow_id {}\".format(workflow_id)\n```\n\nOr using the AWS CLI:\n\n```\naws swf start-workflow-execution --domain SWFSampleDomain \\\n --workflow-id my-unique-workflow-id \\\n --workflow-type name=MyWorkflow,version=1.0 \\\n --task-list name=string-transformer-decider \\\n --lambda-role arn:aws:iam::528461152743:role/swf-lambda \\\n --input '\"Hello\"'\n```", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ajenkins-cargometrics/pyflow", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "pyflow-swf", "package_url": "https://pypi.org/project/pyflow-swf/", "platform": "", "project_url": "https://pypi.org/project/pyflow-swf/", "project_urls": { "Homepage": "https://github.com/ajenkins-cargometrics/pyflow" }, "release_url": "https://pypi.org/project/pyflow-swf/1.6.3/", "requires_dist": null, "requires_python": "", "summary": "Python Workflow Library built on the AWS SWF service", "version": "1.6.3" }, "last_serial": 3518353, "releases": { "1.0": [], "1.0.1": [ { "comment_text": "", "digests": { "md5": "343b53876eaead9bc0c40058bb855a43", "sha256": "7a8cdc16e63a275349f4d43c2f1a4acaf7ac6856deda30b94bb31d452483344e" }, "downloads": -1, "filename": "pyflow-swf-1.0.1.tar.gz", "has_sig": false, "md5_digest": "343b53876eaead9bc0c40058bb855a43", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12970, "upload_time": "2017-04-07T12:06:25", "url": "https://files.pythonhosted.org/packages/38/0c/ae54cac8b4e6de48ead2a81681c97f6ab9d7a2a3fcf60720b807250a760d/pyflow-swf-1.0.1.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "912beace3999014d5d2a1cb4c4b874fc", "sha256": "353242cce0298c19eaec27a5d89a8b6824630d886ac3104c4cf07df999652b9e" }, "downloads": -1, "filename": "pyflow-swf-1.1.0.tar.gz", "has_sig": false, "md5_digest": "912beace3999014d5d2a1cb4c4b874fc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14114, "upload_time": "2017-04-12T15:42:30", "url": "https://files.pythonhosted.org/packages/f2/22/182bd0213f2ae09d9c03a070915b604f9f979ec05cb3bd011107f7f33103/pyflow-swf-1.1.0.tar.gz" } ], "1.2.0": [ { "comment_text": "", "digests": { "md5": "f131a713e45b17d91abb155a527ad888", "sha256": "290173d33df37ae3a1f0550a758a97fc2ae475bf2a0f1c2f0e1935435d365dbd" }, "downloads": -1, "filename": "pyflow-swf-1.2.0.tar.gz", "has_sig": false, "md5_digest": "f131a713e45b17d91abb155a527ad888", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15204, "upload_time": "2017-04-13T17:19:06", "url": "https://files.pythonhosted.org/packages/3c/dd/39f497ea1150697a36ef61870fa42be4fc48d866a1c785aae91a24926654/pyflow-swf-1.2.0.tar.gz" } ], "1.2.1": [ { "comment_text": "", "digests": { "md5": "074bf1fd00c6e8d54647800b02a3cbc1", "sha256": "f7ba0fec646bca0d33bf7ed4043f1b2205cdd510d9981ac2a6331279709f001b" }, "downloads": -1, "filename": "pyflow-swf-1.2.1.tar.gz", "has_sig": false, "md5_digest": "074bf1fd00c6e8d54647800b02a3cbc1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15250, "upload_time": "2017-04-13T18:08:25", "url": "https://files.pythonhosted.org/packages/f7/de/90c480d065715a623400fd50d96df970fb25e70931efc0c0bfd959039a9f/pyflow-swf-1.2.1.tar.gz" } ], "1.3.0": [ { "comment_text": "", "digests": { "md5": "2986dffa593a4fcf3ca61187878dfb46", "sha256": "030f80d287c890cb62291bb8264c0770cb16986f9c8c8896a0bf0e8f3a229a8d" }, "downloads": -1, "filename": "pyflow-swf-1.3.0.tar.gz", "has_sig": false, "md5_digest": "2986dffa593a4fcf3ca61187878dfb46", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16109, "upload_time": "2017-05-04T20:04:27", "url": "https://files.pythonhosted.org/packages/fb/10/72cd0a525e2b14875f25ae2f049d62ca361ad20a3cd6e2bacc67b3c9ec20/pyflow-swf-1.3.0.tar.gz" } ], "1.4.0": [ { "comment_text": "", "digests": { "md5": "ab58b5b4e8f93574a1e153bd65f351c2", "sha256": "48f49ee57adc74166e37517ef2d0e603d8ba9b3499f5abbeeab5472dd8d5e830" }, "downloads": -1, "filename": "pyflow-swf-1.4.0.tar.gz", "has_sig": false, "md5_digest": "ab58b5b4e8f93574a1e153bd65f351c2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16748, "upload_time": "2017-05-05T20:12:59", "url": "https://files.pythonhosted.org/packages/ab/8d/0757a7d915dafa2864fa66ed7e04aa84e11dc4c571f24b69b2567126a6b5/pyflow-swf-1.4.0.tar.gz" } ], "1.4.1": [ { "comment_text": "", "digests": { "md5": "f3fcc143725ded5337f010136fc46d93", "sha256": "2f5b5743dca87eb3423511c4f587062351a0a02701fd7832f52847af9060005d" }, "downloads": -1, "filename": "pyflow-swf-1.4.1.tar.gz", "has_sig": false, "md5_digest": "f3fcc143725ded5337f010136fc46d93", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16892, "upload_time": "2017-05-12T15:15:46", "url": "https://files.pythonhosted.org/packages/39/03/cb82cee576bde504a04d7def31fceb0e76fa8a39c20a8290db05ae05405d/pyflow-swf-1.4.1.tar.gz" } ], "1.4.2": [ { "comment_text": "", "digests": { "md5": "0f00a3359407df690097e16fb328ea92", "sha256": "96a38b28278ed8d2942b5ccef67f83b153a0666b97ca3a62a652269ce60e527c" }, "downloads": -1, "filename": "pyflow-swf-1.4.2.tar.gz", "has_sig": false, "md5_digest": "0f00a3359407df690097e16fb328ea92", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17091, "upload_time": "2017-06-09T13:11:23", "url": "https://files.pythonhosted.org/packages/80/21/cef053e6174285d12c2e86b2d5285fd8a95af154851ab4b40780df10ad94/pyflow-swf-1.4.2.tar.gz" } ], "1.5.0": [ { "comment_text": "", "digests": { "md5": "e25ec38a518a53547682738ab2211131", "sha256": "bb023c28a01a0cd4d3f402fb6bc865e60bc2f666d0893a524a2a2f3358f617b6" }, "downloads": -1, "filename": "pyflow-swf-1.5.0.tar.gz", "has_sig": false, "md5_digest": "e25ec38a518a53547682738ab2211131", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17480, "upload_time": "2017-07-06T17:45:15", "url": "https://files.pythonhosted.org/packages/69/fc/0a58b7365ad661c24b3d4172e688625e79f3010634cd32a344bbcfcb45b1/pyflow-swf-1.5.0.tar.gz" } ], "1.6.0": [ { "comment_text": "", "digests": { "md5": "8b33413770194587d60834fa3e7433df", "sha256": "cf96f2e8323217fed3cd95521acc30cbc0b15731e6741119d37a2d2218dc2477" }, "downloads": -1, "filename": "pyflow-swf-1.6.0.tar.gz", "has_sig": false, "md5_digest": "8b33413770194587d60834fa3e7433df", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17630, "upload_time": "2017-07-10T15:35:37", "url": "https://files.pythonhosted.org/packages/02/85/3cf54b833dce689686ad0d3756024272e7932a0ec510a38c2ba669e04248/pyflow-swf-1.6.0.tar.gz" } ], "1.6.1": [ { "comment_text": "", "digests": { "md5": "c35d6d6434f4461e8aa286b729e68814", "sha256": "2babfb842d0d55bc10ebb9abd624367ff57fad131ac609844d3dc0e6ff1d099e" }, "downloads": -1, "filename": "pyflow-swf-1.6.1.tar.gz", "has_sig": false, "md5_digest": "c35d6d6434f4461e8aa286b729e68814", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17649, "upload_time": "2017-07-11T00:38:37", "url": "https://files.pythonhosted.org/packages/3d/23/a65a65fc309030fe0a2256c2b94e11463606da674ff1c12e80ee6fe5dec4/pyflow-swf-1.6.1.tar.gz" } ], "1.6.2": [ { "comment_text": "", "digests": { "md5": "f2a1ef813fa5edbe2af6ad540f93dc88", "sha256": "4a3e560a9eda8429100163eae4ff5e9d2144ab40bde39cd93db6504daac686c7" }, "downloads": -1, "filename": "pyflow-swf-1.6.2.tar.gz", "has_sig": false, "md5_digest": "f2a1ef813fa5edbe2af6ad540f93dc88", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18041, "upload_time": "2018-01-24T19:17:36", "url": "https://files.pythonhosted.org/packages/c2/b2/794e975c483fea545fd0da10864cd93e268bd58ae4c483a1955e485bd7ea/pyflow-swf-1.6.2.tar.gz" } ], "1.6.3": [ { "comment_text": "", "digests": { "md5": "34f88f230c0cc7db32ac6920b06832fc", "sha256": "ce17be1fb88ab78a794194aa3a01d056b91e285aa0d9e48ecb5fbf620437a715" }, "downloads": -1, "filename": "pyflow-swf-1.6.3.tar.gz", "has_sig": false, "md5_digest": "34f88f230c0cc7db32ac6920b06832fc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18116, "upload_time": "2018-01-24T20:45:29", "url": "https://files.pythonhosted.org/packages/ad/f6/a4c14e11f9f66e6c8dc8ff36bbb561e67d95461b51ed10e6c7d6268d7656/pyflow-swf-1.6.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "34f88f230c0cc7db32ac6920b06832fc", "sha256": "ce17be1fb88ab78a794194aa3a01d056b91e285aa0d9e48ecb5fbf620437a715" }, "downloads": -1, "filename": "pyflow-swf-1.6.3.tar.gz", "has_sig": false, "md5_digest": "34f88f230c0cc7db32ac6920b06832fc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18116, "upload_time": "2018-01-24T20:45:29", "url": "https://files.pythonhosted.org/packages/ad/f6/a4c14e11f9f66e6c8dc8ff36bbb561e67d95461b51ed10e6c7d6268d7656/pyflow-swf-1.6.3.tar.gz" } ] }