{
    "info": {
        "author": "Simon Schmidt",
        "author_email": "schmidt.simon@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: ISC License (ISCL)",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5"
        ],
        "description": "Multi-worker pipelines with gevent.\n\nInstallation\n~~~~~~~~~~~~\n\n.. code:: bash\n\n    pip install gevent-pipeline\n\nExample\n-------\n\n.. code:: python\n\n\n    import gevent\n    import random\n\n    from itertools import repeat\n    from gevent_pipeline import Pipeline\n\n    def sample(b):\n        r = random.uniform(0, b)\n        gevent.sleep(r)\n        return r\n\n    (Pipeline()\n        .from_iter(repeat(1, times=200))\n        .map(sample, n_workers=100)\n        .filter(lambda x: x < 0.5)\n        .fold(max, x0=0, n_workers=50))\n\nPipeline\n--------\n\nChain together operations with multiple workers for each layer.\n\nExample:\n\n.. code:: python\n\n    >>> import operator\n    >>> def only_odd(x):\n    ...     return x & 1\n    ...\n    >>> def double(x):\n    ...     return 2 * x\n    ...\n    >>> (Pipeline()\n    ...         .from_iter(range(100))\n    ...         .filter(only_odd, n_workers=10)\n    ...         .map(double, n_workers=8)\n    ...         .fold(operator.add, x0=0, n_workers=5))\n    5000\n\nThe above is functionally equivalent to:\n\n.. code:: python\n\n    >>> @worker(discard_none=True)\n    ... def only_odd(x):\n    ...     '''Forward only odd numbers to the next layer'''\n    ...     if x & 1:\n    ...        return x\n    ...\n    >>> @worker()\n    ... def double(x):\n    ...     return 2 * x\n    ...\n    >>> def load_numbers(q_in, q_out, q_done):\n    ...     for i in range(100):\n    ...         q_out.put(i)\n    ...     q_done.put(None)\n    ...\n    >>> q_out = ClosableQueue()\n    >>> p = (Pipeline()\n    ...     .chain_workers(load_numbers)\n    ...     .chain_workers(only_odd, n_workers=10)\n    ...     .chain_workers(double, n_workers=8, q_out=q_out))\n    >>> sum(i for i in q_out)\n    5000\n\nThere is no guarantee of order:\n\n.. code:: python\n\n    >>> def f(x):\n    ...     gevent.sleep(random.uniform(0, 0.001))\n    ...     return x\n    >>> p = Pipeline().from_iter(range(10)).map(f, n_workers=5)\n    >>> list(p)\n    [2, 1, 4, 0, 3, 5, 8, 6, 7, 9]\n\nExceptions in workers\n~~~~~~~~~~~~~~~~~~~~~\n\nThere is a predefined ``forward_input`` exception handler, in the event\nthe function raises an exception the handler takes the input to the\nfunction and passes it along as if it was the output.\n\n.. code:: python\n\n    from gevent_pipeline import Pipeline, worker, forward_input\n\n    @worker(exception_handler=forward_input)\n    def f(x):\n        if x & 1:\n            raise ValueError(\"oh no!\")\n            # Will be treated as if it were:\n            # return x\n        else:\n            return 2 * x\n\n    p = (Pipeline()\n         .from_iter(range(100))\n         .chain_workers(f, n_workers=10))\n\n    s_odd = sum(range(1, 100, 2))\n    s_even = sum(2*i for i in range(0, 100, 2))\n    assert sum(p) == s_odd + s_even\n\nClosableQueue\n-------------\n\nActs like ``gevent.queue.Queue`` but in addition has a ``.close()``\nmethod which invokes following behavior:\n\n-  Calling ``.put(item)`` becomes an error\n-  Successive calls to ``.get()`` will return whatever remains in the\n   queue and after that StopIteration is returned for every subsequent\n   call\n\n.. code:: python\n\n    >>> from gevent_pipeline import ClosableQueue\n    >>> q = ClosableQueue()\n    >>> q.put('hello')\n    >>> q.close()\n    >>> q.get()\n    'hello'\n    >>> q.get() is StopIteration\n    True",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/simonschmidt/gevent-pipeline",
        "keywords": "gevent pipelines chaining closablequeue",
        "license": "ISC",
        "maintainer": "",
        "maintainer_email": "",
        "name": "gevent-pipeline",
        "package_url": "https://pypi.org/project/gevent-pipeline/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/gevent-pipeline/",
        "project_urls": {
            "Homepage": "https://github.com/simonschmidt/gevent-pipeline"
        },
        "release_url": "https://pypi.org/project/gevent-pipeline/0.0.5/",
        "requires_dist": [
            "gevent"
        ],
        "requires_python": "",
        "summary": "Multi-worker pipeline and closable queue",
        "version": "0.0.5"
    },
    "last_serial": 2907282,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "772eef0981005719f78430b917530556",
                    "sha256": "a3a1cfad59f2db91c5d1e4f099fd05fb023b5f35cab86d92a14b51c4da4e9b8b"
                },
                "downloads": -1,
                "filename": "gevent_pipeline-0.0.1-py34-none-any.whl",
                "has_sig": false,
                "md5_digest": "772eef0981005719f78430b917530556",
                "packagetype": "bdist_wheel",
                "python_version": "py34",
                "requires_python": null,
                "size": 8741,
                "upload_time": "2016-09-23T18:55:08",
                "url": "https://files.pythonhosted.org/packages/80/50/a022aa8bd60bed216ce41e7b926c28f604a298da48509677189c5351ef00/gevent_pipeline-0.0.1-py34-none-any.whl"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "88b686735f6610e66807e4b9fc7a70cc",
                    "sha256": "acab561a780e243b70e3f1037bc48377b98b7c82516121bd090ab9d082b216ed"
                },
                "downloads": -1,
                "filename": "gevent_pipeline-0.0.2-py34-none-any.whl",
                "has_sig": false,
                "md5_digest": "88b686735f6610e66807e4b9fc7a70cc",
                "packagetype": "bdist_wheel",
                "python_version": "py34",
                "requires_python": null,
                "size": 8563,
                "upload_time": "2016-09-23T21:17:10",
                "url": "https://files.pythonhosted.org/packages/fd/39/77d83736a8af4c40d051992a0ed4e71b3c73d36359fb54d9c7aed618fbd8/gevent_pipeline-0.0.2-py34-none-any.whl"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0956a6eaf2d2b1a6fad189c60040c313",
                    "sha256": "5bf32e30d67fc784eedeb54a5e55e9bda4d76b27b59b583422ed7041b53b1feb"
                },
                "downloads": -1,
                "filename": "gevent_pipeline-0.0.3-py34-none-any.whl",
                "has_sig": false,
                "md5_digest": "0956a6eaf2d2b1a6fad189c60040c313",
                "packagetype": "bdist_wheel",
                "python_version": "py34",
                "requires_python": null,
                "size": 8560,
                "upload_time": "2016-09-25T06:41:30",
                "url": "https://files.pythonhosted.org/packages/c0/c8/ded891ce2140c55ebf335853695fda1ca39e195f1cf1df0775a9c16fc6a8/gevent_pipeline-0.0.3-py34-none-any.whl"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6038635a6fc33083823e7bb5c53e7a2f",
                    "sha256": "80f8cd59600f452d6556c8931e4743a6d7c19a26565a204346b7e76fb857fff6"
                },
                "downloads": -1,
                "filename": "gevent_pipeline-0.0.4-py34-none-any.whl",
                "has_sig": false,
                "md5_digest": "6038635a6fc33083823e7bb5c53e7a2f",
                "packagetype": "bdist_wheel",
                "python_version": "py34",
                "requires_python": null,
                "size": 8553,
                "upload_time": "2016-09-27T13:33:52",
                "url": "https://files.pythonhosted.org/packages/66/34/d1d20bec8ff94afe62d7a541bc477934cb720b8fea851a4896e4f2c47ea4/gevent_pipeline-0.0.4-py34-none-any.whl"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9cacf22bbaf681941abe0ee4af875a37",
                    "sha256": "4979b1e238857d0236442a6401de7d42d5f43aaae8310423271b1ed903a1579a"
                },
                "downloads": -1,
                "filename": "gevent_pipeline-0.0.5-py34-none-any.whl",
                "has_sig": false,
                "md5_digest": "9cacf22bbaf681941abe0ee4af875a37",
                "packagetype": "bdist_wheel",
                "python_version": "py34",
                "requires_python": null,
                "size": 9054,
                "upload_time": "2017-05-29T18:44:08",
                "url": "https://files.pythonhosted.org/packages/56/4e/17d9824bff97aa21c6677090bfe174b24be228a631d5955d39d3fb589d8c/gevent_pipeline-0.0.5-py34-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "9cacf22bbaf681941abe0ee4af875a37",
                "sha256": "4979b1e238857d0236442a6401de7d42d5f43aaae8310423271b1ed903a1579a"
            },
            "downloads": -1,
            "filename": "gevent_pipeline-0.0.5-py34-none-any.whl",
            "has_sig": false,
            "md5_digest": "9cacf22bbaf681941abe0ee4af875a37",
            "packagetype": "bdist_wheel",
            "python_version": "py34",
            "requires_python": null,
            "size": 9054,
            "upload_time": "2017-05-29T18:44:08",
            "url": "https://files.pythonhosted.org/packages/56/4e/17d9824bff97aa21c6677090bfe174b24be228a631d5955d39d3fb589d8c/gevent_pipeline-0.0.5-py34-none-any.whl"
        }
    ]
}