{
    "info": {
        "author": "Mowshon",
        "author_email": "mowshon@yandex.ru",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "# Bounded Process&Thread Pool Executor\nBoundedSemaphore for [ProcessPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor) & [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor) from [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html)\n\n## Installation\n```bash\npip install bounded-pool-executor\n```\n\n# What is the main problem?\nIf you use the standard module \"**concurrent.futures**\" and want to simultaneously process several million data, then a queue of workers will take up all the free memory.\n\nIf the script is run on a weak VPS, this will lead to a **memory leak**.\n\n\n\n## BoundedProcessPoolExecutor VS ProcessPoolExecutor\n\n# BoundedProcessPoolExecutor\n**BoundedProcessPoolExecutor** will put a new worker in queue only when another worker has finished his work.\n\n```python\nfrom bounded_pool_executor import BoundedProcessPoolExecutor\nfrom time import sleep\nfrom random import randint\n\ndef do_job(num):\n    sleep_sec = randint(1, 10)\n    print('value: %d, sleep: %d sec.' % (num, sleep_sec))\n    sleep(sleep_sec)\n\nwith BoundedProcessPoolExecutor(max_workers=5) as worker:\n    for num in range(10000):\n        print('#%d Worker initialization' % num)\n        worker.submit(do_job, num)\n\n```\n### Result:\n![BoundedProcessPoolExecutor](https://python-scripts.com/wp-content/uploads/2018/12/bounded.gif)\n\n# Classic concurrent.futures.ProcessPoolExecutor\n**ProcessPoolExecutor** inserts all workers into the queue and expects tasks to be performed as the new worker is released, depending on the value of `max_workers`.\n\n```python\nimport concurrent.futures\nfrom time import sleep\nfrom random import randint\n\ndef do_job(num):\n    sleep_sec = randint(1, 3)\n    print('value: %d, sleep: %d sec.' % (num, sleep_sec))\n    sleep(sleep_sec)\n\nwith concurrent.futures.ProcessPoolExecutor(max_workers=5) as worker:\n    for num in range(100000):\n        print('#%d Worker initialization' % num)\n        worker.submit(do_job, num)\n```\n\n### Result:\n![concurrent.futures.ProcessPoolExecutor](https://python-scripts.com/wp-content/uploads/2018/12/future-ProcessPoolExecutor.gif)\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://github.com/mowshon/bounded_pool_executor",
        "keywords": "concurrent futures ProcessPoolExecutor ThreadPoolExecutor Semaphore memory leak",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "bounded-pool-executor",
        "package_url": "https://pypi.org/project/bounded-pool-executor/",
        "platform": "",
        "project_url": "https://pypi.org/project/bounded-pool-executor/",
        "project_urls": {
            "Homepage": "http://github.com/mowshon/bounded_pool_executor"
        },
        "release_url": "https://pypi.org/project/bounded-pool-executor/0.0.3/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Bounded Process&Thread Pool Executor",
        "version": "0.0.3"
    },
    "last_serial": 5359303,
    "releases": {
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "9363c2a4cd8d5f8f2d476a6de1a29d33",
                    "sha256": "19807a6ea4e2cac322bb69bfcf19949c9e33ee3b43e06056e6b1a3e488e7df89"
                },
                "downloads": -1,
                "filename": "bounded_pool_executor-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "9363c2a4cd8d5f8f2d476a6de1a29d33",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2242,
                "upload_time": "2018-12-16T19:57:39",
                "url": "https://files.pythonhosted.org/packages/d8/d6/2ca7a52fd5e741a6127118d3f997d7685f29f728f7db711a3aca65278640/bounded_pool_executor-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "e8646f3884ea0bd0d75411f0775bc39f",
                    "sha256": "6f164d64919db1e6a5c187cce281f62bc559a5fed4ce064942e650c227aef190"
                },
                "downloads": -1,
                "filename": "bounded_pool_executor-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "e8646f3884ea0bd0d75411f0775bc39f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 3371,
                "upload_time": "2019-06-04T19:29:05",
                "url": "https://files.pythonhosted.org/packages/bc/23/72ecfe284a1da711257ff310b29c6667d0187a608322d58bf1c7a927c7b2/bounded_pool_executor-0.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "15bf4b1c6c69a6d06f49d886d972c7bf",
                    "sha256": "e092221bc38ade555e1064831f9ed800580fa34a4b6d8e9dd3cd961549627f6e"
                },
                "downloads": -1,
                "filename": "bounded_pool_executor-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "15bf4b1c6c69a6d06f49d886d972c7bf",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2238,
                "upload_time": "2019-06-04T19:29:06",
                "url": "https://files.pythonhosted.org/packages/23/f1/e34501c1228415e9fbcac8cb9c81098900e78331b30eeee1816176324bab/bounded_pool_executor-0.0.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "e8646f3884ea0bd0d75411f0775bc39f",
                "sha256": "6f164d64919db1e6a5c187cce281f62bc559a5fed4ce064942e650c227aef190"
            },
            "downloads": -1,
            "filename": "bounded_pool_executor-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e8646f3884ea0bd0d75411f0775bc39f",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 3371,
            "upload_time": "2019-06-04T19:29:05",
            "url": "https://files.pythonhosted.org/packages/bc/23/72ecfe284a1da711257ff310b29c6667d0187a608322d58bf1c7a927c7b2/bounded_pool_executor-0.0.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "15bf4b1c6c69a6d06f49d886d972c7bf",
                "sha256": "e092221bc38ade555e1064831f9ed800580fa34a4b6d8e9dd3cd961549627f6e"
            },
            "downloads": -1,
            "filename": "bounded_pool_executor-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "15bf4b1c6c69a6d06f49d886d972c7bf",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 2238,
            "upload_time": "2019-06-04T19:29:06",
            "url": "https://files.pythonhosted.org/packages/23/f1/e34501c1228415e9fbcac8cb9c81098900e78331b30eeee1816176324bab/bounded_pool_executor-0.0.3.tar.gz"
        }
    ]
}