{
    "info": {
        "author": "Paulo Porto",
        "author_email": "cesarpaulomp@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# Batchflows for Python 3\n\nThis tool will help you create and process a lot of data in an organized manner.\nYou can create batches of processing synchronously and asynchronously.\n\n*remember it's in BETA :D*\n\n### Get Started\n\n```python\nfrom batchflows.Batch import Batch, Step\n\n#First extend Step class and implement method execute\nclass SaveValueStep(Step):\n    def __init__(self, value_name, value):\n        #Remember name is required if you\n        super().__init__()\n        self.value_name = value_name\n        self.value = value\n\n    # \"_context\" is a dict you can use to store values that will be used in other steps.\n    # If you intend to use a ContextManager (see below) to save progress, I recommend that you use only primitive values.\n    def execute(self, _context):\n        #do what u have to do here!\n        _context[self.value_name] = self.value\n\n#creating a second step just to make the explanation richer\nclass SumCalculatorStep(Step):\n    def __init__(self, attrs):\n        super().__init__()\n        self.attrs = attrs\n\n    def execute(self, _context):\n        calc = 0.0\n        for attr in self.attrs:\n            calc += _context[attr]\n\n        _context[self.name] = calc\n\n#Here we create our batch!\nbatch = Batch()\nbatch.add_step(SaveValueStep('value01', 1))\nbatch.add_step(SaveValueStep('value02', 4))\nbatch.add_step(SumCalculatorStep(['value01', 'value02', 'other_value']))\n\n#You can add something useful to your steps before starting bath!\nbatch.add_to_context('other_value', 5)\n\n#than execute your batch and be happy ;)\nbatch.execute()\n```\n\n### Let's try run some parallel code\n\n```python\nfrom batchflows.Batch import Batch, Step, ParallelFlows\nimport time\n\n\nclass SomeStep(Step):\n    def execute(self, _context):\n        #count to 10 slowly\n        c = 0\n        while c < 10:\n            c += 1\n            print(c)\n            time.sleep(1)\n\n#Create your AsyncFlow\nlazy_counter = ParallelFlows('LazySteps01')\n#add steps so they run in parallel\nlazy_counter.add_step(SomeStep('lazy01'))\nlazy_counter.add_step(SomeStep('lazy02'))\n\nlazy_counter2 = ParallelFlows('LazySteps02')\nlazy_counter2.add_step(SomeStep('lazy03'))\nlazy_counter2.add_step(SomeStep('lazy04'))\n\nbatch = Batch()\nbatch.add_step(lazy_counter)\nbatch.add_step(lazy_counter2)\n\n#batchfllows will wait for each step to finish before executing the next one.\n#In this example lazy_counter will be called first and execute steps \"lazy01\" and \"lazy02\" in parallel.\n#Only when both steps finish ,the batch will star lazy_counter2\nbatch.execute()\n```\n\n### ContextManager\n\nStarting with version 1.0.1-beta, you can use FileContextManager.\nThis feature allows you to save batch progress to a file.\nThis allows you to resume the process from the step that had a runtime error.\nYou can also extend the ContextManager class and create your way of saving progress.\n\n```python\nfrom batchflows.Batch import Batch, Step, ParallelFlows\nfrom batchflows.contextmanager.LocalContextManager import FileContextManager\n\n\nclass DownloadVideoStep(Step):\n    def __init__(self, name, file_url):\n        # Remember if you using ContextManager, name is required and need be unique.\n        # required (default=False) tells the batch that this step is required.\n        # So if the process is resumed, even if this step was successfully executed, it will be executed again.\n        super().__init__(name=name, required=True)\n        self.file_url = file_url\n\n    def execute(self, _context):\n        # download file\n        pass\n\nclass AddWaterMarkStep(Step):\n  #do something\n  pass\n\n\nclass UploadFile(Step):\n  #do something\n  pass\n\n\ncm = FileContextManager('path\\\\to\\\\storage\\\\file.name')\nbatch = Batch(context_manager=cm)\n\nbatch.add_step(DownloadVideoStep('downloadVideo', 'https://somevideo.url/example'))\nbatch.add_step(DownloadVideoStep(AddWaterMarkStep()))\nbatch.add_step(DownloadVideoStep(UploadFile()))\n\n\nbatch.execute()\n```\n\n# Next features:\n  * AsyncFlow thread pool size\n  * AsyncFlow timeout\n  * Remote Flow\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://bitbucket.org/pcmporto/batchflows/src/master",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "batchflows",
        "package_url": "https://pypi.org/project/batchflows/",
        "platform": "",
        "project_url": "https://pypi.org/project/batchflows/",
        "project_urls": {
            "Homepage": "https://bitbucket.org/pcmporto/batchflows/src/master"
        },
        "release_url": "https://pypi.org/project/batchflows/1.0.1b0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "library for executing batches of data processing sequentially or asynchronously to python 3",
        "version": "1.0.1b0"
    },
    "last_serial": 5817600,
    "releases": {
        "1.0.1b0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f39a843b389f9ba6a0e698c9e426b88c",
                    "sha256": "6712dbac62d4f969326e3ca09063d8be03d3df51d9c970e9c9cf07691e47897f"
                },
                "downloads": -1,
                "filename": "batchflows-1.0.1b0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f39a843b389f9ba6a0e698c9e426b88c",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 12812,
                "upload_time": "2019-09-12T00:58:59",
                "url": "https://files.pythonhosted.org/packages/eb/c2/b34dc7140b3faa4fc60d15039332fe24e75b66db160cc4e3c4a6c48e8b4f/batchflows-1.0.1b0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "d021a7894ec2e32ac26a8757fc47ca0b",
                    "sha256": "da84ee50866d521ac3813d42d9413db9d1db321267a4c12f276547458b939b75"
                },
                "downloads": -1,
                "filename": "batchflows-1.0.1b0.tar.gz",
                "has_sig": false,
                "md5_digest": "d021a7894ec2e32ac26a8757fc47ca0b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8015,
                "upload_time": "2019-09-12T00:59:02",
                "url": "https://files.pythonhosted.org/packages/ed/47/5e9330c20215dc73d900570d1350b9289ac2b86198794f3d92f06ce08d95/batchflows-1.0.1b0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "f39a843b389f9ba6a0e698c9e426b88c",
                "sha256": "6712dbac62d4f969326e3ca09063d8be03d3df51d9c970e9c9cf07691e47897f"
            },
            "downloads": -1,
            "filename": "batchflows-1.0.1b0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "f39a843b389f9ba6a0e698c9e426b88c",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 12812,
            "upload_time": "2019-09-12T00:58:59",
            "url": "https://files.pythonhosted.org/packages/eb/c2/b34dc7140b3faa4fc60d15039332fe24e75b66db160cc4e3c4a6c48e8b4f/batchflows-1.0.1b0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "d021a7894ec2e32ac26a8757fc47ca0b",
                "sha256": "da84ee50866d521ac3813d42d9413db9d1db321267a4c12f276547458b939b75"
            },
            "downloads": -1,
            "filename": "batchflows-1.0.1b0.tar.gz",
            "has_sig": false,
            "md5_digest": "d021a7894ec2e32ac26a8757fc47ca0b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 8015,
            "upload_time": "2019-09-12T00:59:02",
            "url": "https://files.pythonhosted.org/packages/ed/47/5e9330c20215dc73d900570d1350b9289ac2b86198794f3d92f06ce08d95/batchflows-1.0.1b0.tar.gz"
        }
    ]
}