{ "info": { "author": "Guojian Li", "author_email": "guojianlee@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Programming Language :: Python :: 3" ], "description": "=======\nBotflow\n=======\n0.2.0 alpha\n\n\n* Data-driven programming framework\n* Paralleled in coroutines and ThreadPool\n* Type- and content-based route function\n* Interactive programming with Jupyter Notebook\n\n\nInstalling\n----------\n\nInstall and update using ``pip``:\n\n`pip install -U botflow`\n\nDocumentation\n------------\n\nhttp://botflow.readthedocs.io\n\n\n\n\nWhat's data-driven programming?\n===============================\n\nAll functions are connected by pipes (queues) and communicate by data. \n\nWhen data come in, the function will be called and return the result.\n\nThink about the pipeline operation in unix: ``ls|grep|sed``.\n\nBenefits:\n\n#. Decouple data and functionality\n#. Easy to reuse \n\nBotflow provides pipe and route. It makes data-driven programming and powerful data flow processes easier.\n\n\nBotflow is...\n=============\n\n- **Simple**\n\nBotflow is easy to use and maintain, *does not need configuration files*, and knows about ``asyncio`` and how to parallelize computation.\n\nHere's one of the simple applications you can make:\n\n_Load the price of Bitcoin every 2 seconds. Advantage price aggregator sample can be found `here `_.\n\n\n.. code-block:: python\n\n from botflow import *\n\n\n def main():\n Pipe(\n\n Timer(delay=2), # send timer data to pipe every 2 seconds\n \"http://api.coindesk.com/v1/bpi/currentprice.json\", # send url to pipe when timer trigger\n HttpLoader(), # read url and load http response\n lambda r: r.json['bpi']['USD']['rate_float'], # read http response and parse as json\n print, # print out\n\n )\n\n Bot.render('ex_output/simple_bitcoin_price')\n Bot.run()\n main()\n\n**Or write in chain style**\n\n\n.. code-block:: python\n\n\n\n from botflow import *\n p_cd_bitcoin=Pipe().Timer(delay=2).Loop(\"http://api.coindesk.com/v1/bpi/currentprice.json\")\\\n .HttpLoader().Map(lambda r: r.json['bpi']['USD']['rate_float']).Map(print)\n\n p_cd_bitcoin.run()\n\n\n\n- **Http server Support OOB** Publish your data pipe to public quickly.\n\n.. code-block:: python\n\n from botflow import *\n from aiohttp import web\n\n\n\n p = Pipe(\n\n {\"msg\":\"hello world!\"}\n )\n\n\n\n app = web.Application()\n\n app.add_routes([\n web.get('/', p.aiohttp_json_handle)\n ])\n\n\n Bot.run_app(app)\n #BotFlow start web server http://0.0.0.0:8080\n\n- **Flow Graph**\nWith render function:\n`Bot.render('bitcoin_arbitrage')`\nBotflow will render the data flow network into a graphviz image.\nbelow is the flow graph generated by Botflow.Aggreate 6 exchanges bitcoin price for trading.\n\n\n.. image:: docs/bitcoin_arbitrage.png\n :width: 400 \n\n\n\n\n- **Fast**\nNodes will be run in parallel, and they will perform well when processing stream data.\n:Web Crawle: Botflow is 10x fatter than Scrapy\n\n\n\n\n- **Replay-able**\n\nWith replay mode enabled:\n``config.replay_mode=True``\nwhen an exception is raised at step N, you don't need to run from setup 1 to N.\nBotflow will replay the data from nearest completed node, usually step N-1.\nIt will save a lot of time in the development phase.\n\nRelease\n\n:**0.2.0**: Milestone release.:\n\n # Jupyter support. Able to run inside Jupyter note book.\n\n # pipe can be nest in another Pipe.\n\n\n p1=Pipe(get_image)\n p2=Pipe(get_price)\n p_get_all=Pipe(Zip(p1,p2)).Filter\n\n # Support Chain style pipe line creating.\n\n Pipe(range(1,10)).Map(lambda x:x+1).Fiter(lambda x:x>2)\n\n same as :\n\n Pipe(range(1,10),lambda x:x+1,Filter(lambda x:x>2))\n\n\n\n:**0.1.9**: Major change see below .:\n\n # Backpressure rate limit support\n\n # Httpserver support\n\n # new Node support. *Zip*, *SendTo* *Flat* for make loop and redirect the flow\n\n # Type hints support .for function type route\n\n # reorge the source code for readable.\n\n\n:**0.1.8**: http://docs.botflow.org/en/latest/change/0.1.8.html .:\n\n #. Support parallel in ThreadPool for slow function.\n\n #. Loop Node is deprecated. raw value and Iterable value can be used directly.\n\n #. improve performance of BlockedJoin\n\n:**0.1.7**: \n\n\nRoadMap\n=======\n- Will add Httpserver support(REST,Websocket). \n- Will support server machine learning Model online.\n- Finshe the api reference doc.\n- Rename project to Botflow.?\n\nMore about Botflow\n===============\n\nData-driven programming is typically applied to streams of structured data for filtering, transforming, aggregating (such as computing statistics), or calling other programs.\n\nBotflow has a few basic concepts to implement Data-driven programming .\n\n- **Source**\n It is feed stream data to the pipe.\n\n * **Timer**: It will send a message in the pipe by timer param. **delay**, **max_time** **until** some finished\n * **Pipe.run**: you can use Pipe.run to trigger the data into pipe. By default it will feed int **0**\n\n\n\n- **Function**\n It is callable unit.Any callable function and object can work as Node. It is driven by data. Custom functions work as Map unit.\n There are some built-in nodes:\n\n\n\n * **Fetch**: (Alias:HttpLoader) Get a url and return the HTTP response\n * **AioFile**: for file I/O.\n * **SpeedLimit**: limit the stream speed limit\n * **Delay**: delay in special second.\n * **Map** : Work ad Convert unit.\n * **Filter** : Drop data from pipe if it does not match some condition\n * **Flat** : Drop data from pipe if it does not match some condition\n\n\n- **Route**\n It will be used to create a complex data flow network, not just one main process. Botflow can nest Routes inside Routes.\n It is a powerful concept.\n There are some pre built-in Route:\n * **Pipe**: It is the main stream process of the program. All units will work inside.\n * **Tee** : (Alias:Branch) Duplicate data from parent pipe to a child pipe as branch.\n * **Zip** : Combine multi pipes result to list.\n * **Link**: (Alias: LinkTo) Route flow to any Node or Route for making loop , circle\n\n\nAll units (Pipe, Node, Route) communicate via queues and perform parallel computation in coroutines.\nThis is abstracted so that Botflow can be used with only limited knowledge of ``asyncio``.\n\n\n\n\nContributing\n------------\n\n\nDonate\n------\n\n\nLinks\n-----\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/kkyx/botflow", "keywords": "", "license": "BSD", "maintainer": "", "maintainer_email": "", "name": "botflow", "package_url": "https://pypi.org/project/botflow/", "platform": "", "project_url": "https://pypi.org/project/botflow/", "project_urls": { "Homepage": "https://github.com/kkyx/botflow" }, "release_url": "https://pypi.org/project/botflow/0.2.0/", "requires_dist": [ "aiohttp (>=3.3.0)", "graphviz", "beautifulsoup4", "lxml" ], "requires_python": ">=3.6.5", "summary": "Data-driven and Reactive programming framework", "version": "0.2.0" }, "last_serial": 4272610, "releases": { "0.1.8": [ { "comment_text": "", "digests": { "md5": "459a06caeb30c4c8106d2dda91adf81e", "sha256": "dae0ce20c7eb870115cb1510e3482c47725f57cc00728fd537f2b9fe4dc911c2" }, "downloads": -1, "filename": "botflow-0.1.8-py3-none-any.whl", "has_sig": false, "md5_digest": "459a06caeb30c4c8106d2dda91adf81e", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.5", "size": 18377, "upload_time": "2018-09-03T14:41:48", "url": "https://files.pythonhosted.org/packages/7c/9a/4991d98ed40a0f01eab14b92ebe9091a74e717f77d22ce33e9ed3e064055/botflow-0.1.8-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9dee9a6524e16ab1ec34d6c75b6ae21e", "sha256": "8049c5c4c91c766318d255437a9e038b548401e26f038528da7ac2ac6e6b6fe0" }, "downloads": -1, "filename": "botflow-0.1.8.tar.gz", "has_sig": false, "md5_digest": "9dee9a6524e16ab1ec34d6c75b6ae21e", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.5", "size": 17323, "upload_time": "2018-09-03T14:41:49", "url": "https://files.pythonhosted.org/packages/0a/de/36fd5731e41ac0040c2959576fbc2e124e8d954f6cf43375fd72dbb4fe03/botflow-0.1.8.tar.gz" } ], "0.1.9.1": [ { "comment_text": "", "digests": { "md5": "95039d537cf5fbebdfe984d392e84257", "sha256": "07dd028bdd94dae7b70af10e9500a643c5e53565e0ae3153f864897aa6214ef6" }, "downloads": -1, "filename": "botflow-0.1.9.1-py3-none-any.whl", "has_sig": false, "md5_digest": "95039d537cf5fbebdfe984d392e84257", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.5", "size": 25184, "upload_time": "2018-09-07T15:19:15", "url": "https://files.pythonhosted.org/packages/7b/d8/9313b382212b5ae2fa55a32273ef2a6e0b1bef7b394e35f8fdcbf72cf6c4/botflow-0.1.9.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "56c6d11b9a39a14371a14c28a0ba84b8", "sha256": "157c7fbcdd8a31fdc21786f0ec03548ec6071372aa9aaa2367b54da821cfb3d7" }, "downloads": -1, "filename": "botflow-0.1.9.1.tar.gz", "has_sig": false, "md5_digest": "56c6d11b9a39a14371a14c28a0ba84b8", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.5", "size": 21411, "upload_time": "2018-09-07T15:19:17", "url": "https://files.pythonhosted.org/packages/5b/be/6b081b2a333abb5ec4bf704442828c537a65414a1503be9556f540474e6d/botflow-0.1.9.1.tar.gz" } ], "0.1.9.2": [ { "comment_text": "", "digests": { "md5": "725ced0c5ecb24559cb46f55f1757b3b", "sha256": "4e6aad9cba6bfe2cc870be54add3b30357279495e3b93ec189e710557692d242" }, "downloads": -1, "filename": "botflow-0.1.9.2-py3-none-any.whl", "has_sig": false, "md5_digest": "725ced0c5ecb24559cb46f55f1757b3b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.5", "size": 27193, "upload_time": "2018-09-09T13:42:08", "url": "https://files.pythonhosted.org/packages/08/d4/6f51de746522a7b3b69b0934920d0a97a0c015388563703a5ec35dfdc473/botflow-0.1.9.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "7fcb187c985c815f35611303636f24d7", "sha256": "87ccf5667d16b8b589d13a6d0e5141ef729340c19e2e872bb4bc81deb3cd3bf6" }, "downloads": -1, "filename": "botflow-0.1.9.2.tar.gz", "has_sig": false, "md5_digest": "7fcb187c985c815f35611303636f24d7", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.5", "size": 23544, "upload_time": "2018-09-09T13:42:09", "url": "https://files.pythonhosted.org/packages/c7/5c/abd619a6017756d108b523053f2b9d0d66b190be8540d4e6d075f47d1cad/botflow-0.1.9.2.tar.gz" } ], "0.1.9.3": [ { "comment_text": "", "digests": { "md5": "255603703399380be9bb0ce4f0b11649", "sha256": "da822cbb9b20f52323df21d4ec4e6a01887d4ac8142a992cf7345d10c0dfc26a" }, "downloads": -1, "filename": "botflow-0.1.9.3-py3-none-any.whl", "has_sig": false, "md5_digest": "255603703399380be9bb0ce4f0b11649", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.5", "size": 27558, "upload_time": "2018-09-10T07:53:41", "url": "https://files.pythonhosted.org/packages/6b/a2/74ed445ccdf211d31c2a40c2fc057c47442252cbffce30f32b57e8c7d486/botflow-0.1.9.3-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "89610d30d8dc2fe271a4e3bf1455280b", "sha256": "0693111d217b423f0c5983f6ebb49f31247d726cd70b4a128bd5408ae0f45d0c" }, "downloads": -1, "filename": "botflow-0.1.9.3.tar.gz", "has_sig": false, "md5_digest": "89610d30d8dc2fe271a4e3bf1455280b", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.5", "size": 23936, "upload_time": "2018-09-10T07:53:44", "url": "https://files.pythonhosted.org/packages/e1/4b/189b4778a0d3b30ed933d902b07129ae9af9bd30d3b360618c1fe7feb664/botflow-0.1.9.3.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "5c577dc31faad8d6cab1c8eb278f7f67", "sha256": "d438de659e96f1c18871b2e3e3c614afc49d30f3c4defdff3972aa7cc3a0665c" }, "downloads": -1, "filename": "botflow-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "5c577dc31faad8d6cab1c8eb278f7f67", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.5", "size": 31626, "upload_time": "2018-09-14T14:38:32", "url": "https://files.pythonhosted.org/packages/17/3f/f0d2a8bfbde38777c77e243217d7b65eb1da1f2feb4b9024517dd042e025/botflow-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6d6578137167c40ec6ba7a7c00d964e6", "sha256": "fddbffb3d0abd3f6a4004c654ef00dd253d823d7669998de99b2b66dfa076b9a" }, "downloads": -1, "filename": "botflow-0.2.0.tar.gz", "has_sig": false, "md5_digest": "6d6578137167c40ec6ba7a7c00d964e6", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.5", "size": 26228, "upload_time": "2018-09-14T14:38:37", "url": "https://files.pythonhosted.org/packages/f5/eb/b4e606390a0d1e5d2387a4596dbacdd226bfbbf60850d0453aedf4b768eb/botflow-0.2.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5c577dc31faad8d6cab1c8eb278f7f67", "sha256": "d438de659e96f1c18871b2e3e3c614afc49d30f3c4defdff3972aa7cc3a0665c" }, "downloads": -1, "filename": "botflow-0.2.0-py3-none-any.whl", "has_sig": false, "md5_digest": "5c577dc31faad8d6cab1c8eb278f7f67", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.6.5", "size": 31626, "upload_time": "2018-09-14T14:38:32", "url": "https://files.pythonhosted.org/packages/17/3f/f0d2a8bfbde38777c77e243217d7b65eb1da1f2feb4b9024517dd042e025/botflow-0.2.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6d6578137167c40ec6ba7a7c00d964e6", "sha256": "fddbffb3d0abd3f6a4004c654ef00dd253d823d7669998de99b2b66dfa076b9a" }, "downloads": -1, "filename": "botflow-0.2.0.tar.gz", "has_sig": false, "md5_digest": "6d6578137167c40ec6ba7a7c00d964e6", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.6.5", "size": 26228, "upload_time": "2018-09-14T14:38:37", "url": "https://files.pythonhosted.org/packages/f5/eb/b4e606390a0d1e5d2387a4596dbacdd226bfbbf60850d0453aedf4b768eb/botflow-0.2.0.tar.gz" } ] }