{
    "info": {
        "author": "Guillaume Plique",
        "author_email": "kropotkinepiotr@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "[![Build Status](https://travis-ci.org/medialab/quenouille.svg)](https://travis-ci.org/medialab/quenouille)\n\n# Quenouille\n\nA library of multithreaded iterator workflows for python.\n\n## Installation\n\nYou can install `quenouille` with pip with the following command:\n\n```\npip install quenouille\n```\n\n## Usage\n* [imap](#imap)\n* [imap_unordered](#imapunordered)\n\n### imap\n\nFunction lazily consuming an iterator and applying the desired function over the yielded items in a multithreaded fashion. The function will yield results in an order consistent with the provided iterator.\n\nFurthermore, it's possible to tweak options regarding group parallelism if you ever need to ensure that a limited number of threads may perform their tasks over the same group, e.g. a domain name when fetching urls: you can give a function extracting the group from the current task, you can tweak the maximum number of threads working on a same group and finally you can edit a group's buffer size to let the function load more values into memory in hope of finding next ones it can process without needing to wait.\n\nIf you don't care about output order and want snappier performance, the library also exports an [`imap_unordered`](#imap_unordered) method.\n\n```python\nimport csv\nfrom quenouille import imap\n\n# Example fetching urls from a CSV file\nwith open(csv_path, 'r') as f:\n  reader = csv.DictReader(f)\n\n  urls = (line['url'] for line in reader)\n\n  # The `fetch` function remains to be implemented by the reader\n  for html in imap(urls, fetch, 10):\n\n    # Results will be yielded in lines order\n    print(html)\n```\n\n*Arguments*\n\n* **iterable** *iterable*: Any python iterable.\n* **func** *callable*: Function used to perform desired tasks. The function takes any item yielded from the given iterable as sole argument. Note that since this function will be dispatched in a multithreaded environment, it should be thread-safe.\n* **threads** *int*: Number of threads to use.\n* **group** *?callable* [`None`]: Function taking a single item yielded by the provided iterable and returning its group.\n* **group_parallelism** *?int* [`Infinity`]: Maximum number of threads that can work on the same group at once. Defaults to no limit. This option requires that you give a function as the `group` argument.\n* **group_buffer_size** *?int* [`1`]: Maximum number of values that will be loaded into memory from the iterable before waiting for other relevant threads to be available.\n* **group_throttle** *?float|?callable* [`0`]: throttle time to wait (in seconds) between two tasks on the same group. Can also be a function taking the group and item and returning throttle time.\n* **group_throttle_entropy** *?float* [`0`]: additional random throttle time between `0` and given value. Useful to simulate erratic behavior.\n* **listener** *callable* [`None`]: A function called on certain events with the name of the event and the related item.\n\n*Events*\n\n* **start**: Emitted when the given function actually starts to work on a yielded item.\n\n### imap_unordered\n\nFunction lazily consuming an iterator and applying the desired function over the yielded items in a multithreaded fashion. The function will yield results in arbitrary order based on thread completion.\n\nFurthermore, it's possible to tweak options regarding group parallelism if you ever need to ensure that a limited number of threads may perform their tasks over the same group, e.g. a domain name when fetching urls: you can give a function extracting the group from the current task, you can tweak the maximum number of threads working on a same group and finally you can edit a group's buffer size to let the function load more values into memory in hope of finding next ones it can process without needing to wait.\n\nIf output order is important to you, the library also exports an [`imap`](#imap) method.\n\n```python\nimport csv\nfrom quenouille import imap_unordered\n\n# Example fetching urls from a CSV file\nwith open(csv_path, 'r') as f:\n  reader = csv.DictReader(f)\n\n  urls = (line['url'] for line in reader)\n\n  # The `fetch` function remains to be implemented by the reader\n  for html in imap_unordered(urls, fetch, 10):\n\n    # Results will be yielded in arbitrary order as soon as tasks complete\n    print(html)\n```\n\n*Arguments*\n\n* **iterable** *iterable*: Any python iterable.\n* **func** *callable*: Function used to perform desired tasks. The function takes any item yielded from the given iterable as sole argument. Note that since this function will be dispatched in a multithreaded environment, it should be thread-safe.\n* **threads** *int*: Number of threads to use.\n* **group** *?callable* [`None`]: Function taking a single item yielded by the provided iterable and returning its group.\n* **group_parallelism** *?int* [`Infinity`]: Maximum number of threads that can work on the same group at once. Defaults to no limit. This option requires that you give a function as the `group` argument.\n* **group_buffer_size** *?int* [`1`]: Maximum number of values that will be loaded into memory from the iterable before waiting for other relevant threads to be available.\n* **group_throttle** *?float|?callable* [`0`]: throttle time to wait (in seconds) between two tasks on the same group. Can also be a function taking the group and item and returning throttle time.\n* **group_throttle_entropy** *?float* [`0`]: additional random throttle time between `0` and given value. Useful to simulate erratic behavior.\n* **listener** *callable* [`None`]: A function called on certain events with the name of the event and the related item.\n\n*Events*\n\n* **start**: Emitted when the given function actually starts to work on a yielded item.\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://github.com/medialab/quenouille",
        "keywords": "url",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "quenouille",
        "package_url": "https://pypi.org/project/quenouille/",
        "platform": "",
        "project_url": "https://pypi.org/project/quenouille/",
        "project_urls": {
            "Homepage": "http://github.com/medialab/quenouille"
        },
        "release_url": "https://pypi.org/project/quenouille/0.6.0/",
        "requires_dist": null,
        "requires_python": ">=3.0",
        "summary": "A library of multithreaded iterator workflows.",
        "version": "0.6.0"
    },
    "last_serial": 5988689,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0931620af0b7aa1c83cf26edb27886ff",
                    "sha256": "328605eccdf295e262456637976c411b6028566dbc8d209c8df7cd810aa589aa"
                },
                "downloads": -1,
                "filename": "quenouille-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0931620af0b7aa1c83cf26edb27886ff",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.0",
                "size": 7204,
                "upload_time": "2019-02-22T17:32:23",
                "url": "https://files.pythonhosted.org/packages/ce/86/406edfdd95cf88b35d8f4af26bcdc872af959620af68da3d3bbde191d128/quenouille-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "4bd9d5911f4dfea5b74ba0a78c26b20e",
                    "sha256": "e4e75a1b877c4dc9507f58198307e6202e5a117b562ee6074ad457cc1e10bc27"
                },
                "downloads": -1,
                "filename": "quenouille-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "4bd9d5911f4dfea5b74ba0a78c26b20e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.0",
                "size": 5466,
                "upload_time": "2019-02-22T17:32:25",
                "url": "https://files.pythonhosted.org/packages/44/20/899f92d8325f2cc813074d463cad3f083e689880e9d0e958bced4590da74/quenouille-0.1.0.tar.gz"
            }
        ],
        "0.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1f78cb07d26412f65eb12e9b75e964ee",
                    "sha256": "1a80c437e9c4911cec7c64de7e724888ee95a942d7733db4f12c5210bacaca0f"
                },
                "downloads": -1,
                "filename": "quenouille-0.2.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1f78cb07d26412f65eb12e9b75e964ee",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.0",
                "size": 7943,
                "upload_time": "2019-02-25T17:15:12",
                "url": "https://files.pythonhosted.org/packages/d2/95/c5d9c211bfbcb8318e25d11f4ab6c277f9e3a79a091c937b2675277df58d/quenouille-0.2.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7fd2c8b5dc450a1a99cb1389301e8d03",
                    "sha256": "98f9b713cd818cbcd4169d6eded09486afae4fa9b8cc71b3c11d80205ec76b3c"
                },
                "downloads": -1,
                "filename": "quenouille-0.2.0.tar.gz",
                "has_sig": false,
                "md5_digest": "7fd2c8b5dc450a1a99cb1389301e8d03",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.0",
                "size": 6222,
                "upload_time": "2019-02-25T17:15:13",
                "url": "https://files.pythonhosted.org/packages/64/4c/dfaf1b273f6c21ead4e226b44d330b3f14ed5765e9655f41108b5bc81ec3/quenouille-0.2.0.tar.gz"
            }
        ],
        "0.3.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "3620ec8af759ef669c42003d24644dc5",
                    "sha256": "2b427596f3c38de7fe962fffad4118ee7556985b8ccaaf25cc1924bca987a0f2"
                },
                "downloads": -1,
                "filename": "quenouille-0.3.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "3620ec8af759ef669c42003d24644dc5",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.0",
                "size": 8232,
                "upload_time": "2019-03-18T11:29:02",
                "url": "https://files.pythonhosted.org/packages/9e/95/8dce1f00777c228e656080e6bffc9e27e928c9135409355279f471efb915/quenouille-0.3.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8eaf552ec2abd44098ce6046ca03066d",
                    "sha256": "438d78faa8a6244694cd5e4199345bd98e7de9ec5f4d546c71fcb9f3413267fb"
                },
                "downloads": -1,
                "filename": "quenouille-0.3.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8eaf552ec2abd44098ce6046ca03066d",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.0",
                "size": 6499,
                "upload_time": "2019-03-18T11:29:03",
                "url": "https://files.pythonhosted.org/packages/8a/5f/772aeb2985574d8646006e507c7615d61635533c62400a3f469a60f0cf77/quenouille-0.3.0.tar.gz"
            }
        ],
        "0.4.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4f8b37962df440cf9ea186bd8c39c9a7",
                    "sha256": "80794a54c6c00d124b76fc7a210b4986cce7c240ca3a3e7bd89e1965ccf88c95"
                },
                "downloads": -1,
                "filename": "quenouille-0.4.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "4f8b37962df440cf9ea186bd8c39c9a7",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.0",
                "size": 8530,
                "upload_time": "2019-10-02T12:36:52",
                "url": "https://files.pythonhosted.org/packages/5a/98/333daf8e8de96e312748a5c53f9dbaca746c58421beecc070e6cf3e32003/quenouille-0.4.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1a3774af5e8ad70759fa4964a1d560ac",
                    "sha256": "6a86dac147eaac9c8cfa8193c98af182a949e3ffa3f5eeb6e1989730a2465ca6"
                },
                "downloads": -1,
                "filename": "quenouille-0.4.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1a3774af5e8ad70759fa4964a1d560ac",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.0",
                "size": 7072,
                "upload_time": "2019-10-02T12:36:53",
                "url": "https://files.pythonhosted.org/packages/d2/38/977fe38855609ee19433b0e49816c0bd99bb5f0af7274cdd49795da16509/quenouille-0.4.0.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "aadb003e8a384d3d9b281e55045ba07e",
                    "sha256": "caeb6903a6093152205c28357a2cc02bafbb73796416c32fecb26504f4d70608"
                },
                "downloads": -1,
                "filename": "quenouille-0.5.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "aadb003e8a384d3d9b281e55045ba07e",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.0",
                "size": 9410,
                "upload_time": "2019-10-10T18:53:52",
                "url": "https://files.pythonhosted.org/packages/66/d8/1f1cb0711fcca45f729b5cd1e8cfb70a1a2bf4759582745511a30b4bcc9e/quenouille-0.5.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "947fa1aa552a6e22b5a12b7ee27125a1",
                    "sha256": "8f4eacaa0c304db165689806855eda46880e1a64197e7d25f45b4948aaa4e7f7"
                },
                "downloads": -1,
                "filename": "quenouille-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "947fa1aa552a6e22b5a12b7ee27125a1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.0",
                "size": 7243,
                "upload_time": "2019-10-10T18:53:55",
                "url": "https://files.pythonhosted.org/packages/be/83/8d45706cde63b7be4ecc057f2d43311da817aa844667a0ed83bdbd520e4e/quenouille-0.5.0.tar.gz"
            }
        ],
        "0.6.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7571476743ab9dbc307db57f0aa0c132",
                    "sha256": "13a72fe74046373266a724f2aa88077c4533136f2ebf57a6c7aa3c8aedc7b46f"
                },
                "downloads": -1,
                "filename": "quenouille-0.6.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7571476743ab9dbc307db57f0aa0c132",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.0",
                "size": 9630,
                "upload_time": "2019-10-17T09:33:24",
                "url": "https://files.pythonhosted.org/packages/b5/39/218af3a9f4a2e9b3ec884de40c67aa3ea38795d1e17855c762bd0d62ed48/quenouille-0.6.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "127f33ddfd37130e2fb2fddc5c8ec5fc",
                    "sha256": "78e1ccba4a9d08774236575095c698118a3bc3b06e956be1b309579b97f1d7b0"
                },
                "downloads": -1,
                "filename": "quenouille-0.6.0.tar.gz",
                "has_sig": false,
                "md5_digest": "127f33ddfd37130e2fb2fddc5c8ec5fc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.0",
                "size": 7326,
                "upload_time": "2019-10-17T09:33:26",
                "url": "https://files.pythonhosted.org/packages/c6/bd/2410c3bc6bae03a015c3fb06463dfddfcecb554ddfaf6f34c90354457340/quenouille-0.6.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "7571476743ab9dbc307db57f0aa0c132",
                "sha256": "13a72fe74046373266a724f2aa88077c4533136f2ebf57a6c7aa3c8aedc7b46f"
            },
            "downloads": -1,
            "filename": "quenouille-0.6.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "7571476743ab9dbc307db57f0aa0c132",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.0",
            "size": 9630,
            "upload_time": "2019-10-17T09:33:24",
            "url": "https://files.pythonhosted.org/packages/b5/39/218af3a9f4a2e9b3ec884de40c67aa3ea38795d1e17855c762bd0d62ed48/quenouille-0.6.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "127f33ddfd37130e2fb2fddc5c8ec5fc",
                "sha256": "78e1ccba4a9d08774236575095c698118a3bc3b06e956be1b309579b97f1d7b0"
            },
            "downloads": -1,
            "filename": "quenouille-0.6.0.tar.gz",
            "has_sig": false,
            "md5_digest": "127f33ddfd37130e2fb2fddc5c8ec5fc",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.0",
            "size": 7326,
            "upload_time": "2019-10-17T09:33:26",
            "url": "https://files.pythonhosted.org/packages/c6/bd/2410c3bc6bae03a015c3fb06463dfddfcecb554ddfaf6f34c90354457340/quenouille-0.6.0.tar.gz"
        }
    ]
}