{ "info": { "author": "Martin Bammer", "author_email": "mrbm74@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Software Development :: Build Tools" ], "description": "An efficient and lightweight thread pool\n========================================\n\nExisting implementations of thread pools have a relatively high overhead in certain\nsituations. Especially ``apply_async`` in ``multiprocessing.pool.ThreadPool`` and\n``concurrent.futures.ThreadPoolExecutor`` at all (see benchmarks).\nIn case of ``ThreadPoolExecutor`` don't use the ``wait``. It can be *extremely* slow!\nIf you've only a small number of jobs and the jobs have a relatively long processing\ntime, then these overheads don't count. But in case of high number of jobs with\nshort processing time the overhead of the above implementations will noticeably\nslow down the processing speed.\nThe fastthreadpool module solves this issue, because it has a very small overhead in\nall situations.\n\nThe API is described `here `_.\n\nExamples\n========\n\n::\n\n pool = fastthreadpool.Pool()\n pool.map(worker, iterable)\n pool.shutdown()\n\nResults with successful execution were saved in the **done** queue, with failed execution in the **failed** queue.\n\n::\n\n pool = fastthreadpool.Pool()\n pool.map(worker, iterable, done_cb)\n pool.shutdown()\n\nFor every successful execution of the worker the done_cb callback function is called. Results with failed execution in the **failed** queue.\n\n::\n\n pool = fastthreadpool.Pool(result_id = True)\n job_id1 = pool.submit(worker, foo1)\n pool.shutdown()\n\nResults with successful execution were saved in the **done** queue, with failed execution in the **failed** queue. Each entry in the queues is a tuple with the job_id as the first argument and the result as the second argument.\n\n::\n\n pool = fastthreadpool.Pool(result_id = True)\n for i in range(100):\n jobid = pool.submit(worker, foo1, i)\n pool.submit_first(worker, foo2)\n pool.cancel(jobid)\n pool.submit_later(0.1, delayed_worker, foo3)\n pool.schedule(1.0, scheduled_worker, foo4)\n time.sleep(1.0)\n pool.cancel(None, True)\n pool.shutdown()\n\nThis is a more complex example which shows some of the features of fastthreadpool. First 100 jobs with foo1 and a counter are submitted. Then a job is submitted to the beginning of the job queue. Then the job with foo1 and i=99 is cancelled. Then a job is scheduled for a one time execution in 0.1 seconds. Finally a job is scheduled for repeated execution in a 1 second interval.\n\nNext example shows a use case of an initialization callback function::\n\n def worker(compressed_data):\n return current_thread().Z.decompress(compressed_data)\n\n def cbInit(ctx):\n ctx.Z = zstd.ZstdDecompressor()\n\n pool = fastthreadpool.Pool(init_callback = cbInit)\n for data in iterable:\n pool.submit(worker, data)\n\nNext example shows a simple echo server. The echo server is extremely fast is the buffer size is big enough.\nResults have shown on a Ryzen 7 and Linux that this simple server can handle more than 400000 messages / second::\n\n def pool_echo_server(address, threads, size):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)\n sock.bind(address)\n sock.listen(threads)\n with sock:\n while True:\n client, addr = sock.accept()\n pool.submit(pool_echo_client, client, size)\n\n def pool_echo_client(client, size):\n client.setsockopt(IPPROTO_TCP, TCP_NODELAY, 1)\n b = bytearray(size)\n bl = [ b ]\n with client:\n try:\n while True:\n client.recvmsg_into(bl)\n client.sendall(b)\n except:\n pass\n\n pool = fastthreadpool.Pool(8)\n pool.submit(pool_echo_server, addr, 8, 4096)\n pool.join()\n\n\n**Benchmarks**\n==============\n\nExample ``ex_semaphore.py`` results on a Celeron N3160 are:\n\n::\n\n 1.8018 seconds for threading.Semaphore\n 0.083 seconds for fasthreadpool.Semaphore\n\nfastthreadpool.Semaphore is **21.7** x faster.\n\n\nExample ``ex_simple_sum.py`` results on a Celeron N3160 are:\n\n::\n\n 0.019 seconds for simple for loop.\n 0.037 seconds for simple for loop. Result is saved in class variable.\n 0.048 seconds for fastthreadpool.map. Results are save in done queue.\n 0.494 seconds for fastthreadpool.submit. Results are save in done queue.\n 0.111 seconds for multiprocessing.pool.ThreadPool.map_async.\n 21.280 seconds for multiprocessing.pool.ThreadPool.apply_async.\n\nfastthreadpool.map is **2,3** x faster than multiprocessing.pool.ThreadPool.map_async.\nfastthreadpool.submit is **43** x faster than multiprocessing.pool.ThreadPool.apply_async.", "description_content_type": "text/x-rst", "docs_url": null, "download_url": "https://github.com/brmmm3/fastthreadpool/releases/download/1.5.2/fastthreadpool-1.5.2.tar.gz", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/brmmm3/fastthreadpool", "keywords": "fast threading thread pool", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "fastthreadpool", "package_url": "https://pypi.org/project/fastthreadpool/", "platform": "", "project_url": "https://pypi.org/project/fastthreadpool/", "project_urls": { "Download": "https://github.com/brmmm3/fastthreadpool/releases/download/1.5.2/fastthreadpool-1.5.2.tar.gz", "Homepage": "https://github.com/brmmm3/fastthreadpool" }, "release_url": "https://pypi.org/project/fastthreadpool/1.5.2/", "requires_dist": null, "requires_python": "", "summary": "An efficient and leightweight thread pool.", "version": "1.5.2" }, "last_serial": 5566052, "releases": { "1.1.0": [ { "comment_text": "", "digests": { "md5": "dec77018e3836fcb5a147272d2969a2d", "sha256": "4231aab970ec4529d6a0e991558ed857a284e375e51b5bf970ddd6a954eb9b60" }, "downloads": -1, "filename": "fastthreadpool-1.1.0.tar.gz", "has_sig": false, "md5_digest": "dec77018e3836fcb5a147272d2969a2d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7064, "upload_time": "2018-03-17T16:34:53", "url": "https://files.pythonhosted.org/packages/d5/ca/f62955f86342afafd6f09687c691787d36827f1378769bd69785f90fc68c/fastthreadpool-1.1.0.tar.gz" } ], "1.2.1": [ { "comment_text": "", "digests": { "md5": "f8ca94f32df9b3b2f8d0599dcd67d4e6", "sha256": "840a8145901e949bd5e0a588b35f3ba492ffdfb86ffe6b2a6a6d2011cef24a7f" }, "downloads": -1, "filename": "fastthreadpool-1.2.1.tar.gz", "has_sig": false, "md5_digest": "f8ca94f32df9b3b2f8d0599dcd67d4e6", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7601, "upload_time": "2018-03-19T08:18:13", "url": "https://files.pythonhosted.org/packages/d7/87/e7e4f15c145825c1a4e2e24b3b982a6180b931c2ee8fc2a724b625e3dbc4/fastthreadpool-1.2.1.tar.gz" } ], "1.2.10": [ { "comment_text": "", "digests": { "md5": "6d4f94ff1677b4499e5b24d884524741", "sha256": "cd050103a1730cbc1b653766be1a15e277e10fa4841ab4883e82694fd66d9b6c" }, "downloads": -1, "filename": "fastthreadpool-1.2.10.tar.gz", "has_sig": false, "md5_digest": "6d4f94ff1677b4499e5b24d884524741", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18398, "upload_time": "2018-03-31T06:08:50", "url": "https://files.pythonhosted.org/packages/b5/42/9c50da49143c932bb1532de2e5b91b2c8d2b00bf9b5acc241b320803302d/fastthreadpool-1.2.10.tar.gz" } ], "1.2.11": [ { "comment_text": "", "digests": { "md5": "2bce58982999b8b8d30fb6ddd658282b", "sha256": "1908170c18b8c1e466106252018382874e39e4a959f112ca951555ea8487bbae" }, "downloads": -1, "filename": "fastthreadpool-1.2.11.tar.gz", "has_sig": false, "md5_digest": "2bce58982999b8b8d30fb6ddd658282b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13999, "upload_time": "2018-07-22T18:31:10", "url": "https://files.pythonhosted.org/packages/31/f8/b3ca1178925aaffda47180ca9cd53135d9970cad1379b94db75960358bbc/fastthreadpool-1.2.11.tar.gz" } ], "1.2.2": [ { "comment_text": "", "digests": { "md5": "ed2d7a4211681260087349f792599b47", "sha256": "9677f676779fd7f06ab4ba743436c9c364ad0fc2d07bab5c2d1e58f0bdde60f8" }, "downloads": -1, "filename": "fastthreadpool-1.2.2.tar.gz", "has_sig": false, "md5_digest": "ed2d7a4211681260087349f792599b47", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11853, "upload_time": "2018-03-19T09:10:21", "url": "https://files.pythonhosted.org/packages/b9/8d/ca1062702b1763f22d3a5353bd854301738f744d14bd2de5ace58de26fce/fastthreadpool-1.2.2.tar.gz" } ], "1.2.3": [ { "comment_text": "", "digests": { "md5": "7cf435ed673b41af8b738e08859f4387", "sha256": "58997c36ca5ed5b20dc7b3785a8b352970474eea3279172972af2260a2300b5c" }, "downloads": -1, "filename": "fastthreadpool-1.2.3.tar.gz", "has_sig": false, "md5_digest": "7cf435ed673b41af8b738e08859f4387", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13040, "upload_time": "2018-03-19T09:16:30", "url": "https://files.pythonhosted.org/packages/1b/a7/ced82d7a07ef5db3949ed7cb8a3f631920ad0182cff4578c6a3e3be69283/fastthreadpool-1.2.3.tar.gz" } ], "1.2.4": [ { "comment_text": "", "digests": { "md5": "269b6bae6350ca4e115638c3f5398b9f", "sha256": "aed8895f5470afc38977dffea131d6bc73ac7b5559b955f9017cd087b7827391" }, "downloads": -1, "filename": "fastthreadpool-1.2.4.tar.gz", "has_sig": false, "md5_digest": "269b6bae6350ca4e115638c3f5398b9f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 12850, "upload_time": "2018-03-19T11:06:43", "url": "https://files.pythonhosted.org/packages/ff/c4/4b2a2d89a15a5c4cfb34a37912033e78ad4246636df45f990a23ead55508/fastthreadpool-1.2.4.tar.gz" } ], "1.2.5": [ { "comment_text": "", "digests": { "md5": "d5b50838806cc788e20dfea6084565bd", "sha256": "0f93736db7600338b4dc0bf20dde44d5d3c55dc205d368ac4eae6faaf8b8d049" }, "downloads": -1, "filename": "fastthreadpool-1.2.5.tar.gz", "has_sig": false, "md5_digest": "d5b50838806cc788e20dfea6084565bd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17084, "upload_time": "2018-03-21T18:30:16", "url": "https://files.pythonhosted.org/packages/24/20/5c6cf2a5254e93bb6e547ca0b3227182c74cc06dadb0b93cb30bdf25a343/fastthreadpool-1.2.5.tar.gz" } ], "1.2.6": [ { "comment_text": "", "digests": { "md5": "4ee254dbe7f655dc1d3b67e25f3e5616", "sha256": "bcaa875594bc6b87dfc76607e818befef45fd1d46caff3f4bc6661b9dad18f58" }, "downloads": -1, "filename": "fastthreadpool-1.2.6.tar.gz", "has_sig": false, "md5_digest": "4ee254dbe7f655dc1d3b67e25f3e5616", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17067, "upload_time": "2018-03-21T18:32:21", "url": "https://files.pythonhosted.org/packages/20/2b/90e6b4fb425e0827d90d7d21ffe00dcfd4744176680793c41775f8a5583a/fastthreadpool-1.2.6.tar.gz" } ], "1.2.7": [ { "comment_text": "", "digests": { "md5": "dd2adfd3f996b1bd5098d25a8c9b50b0", "sha256": "52e53f92cc6998199d837a8029dda27f0124571e6e31e78b7169d0b5d4d80eac" }, "downloads": -1, "filename": "fastthreadpool-1.2.7.tar.gz", "has_sig": false, "md5_digest": "dd2adfd3f996b1bd5098d25a8c9b50b0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17096, "upload_time": "2018-03-25T14:17:08", "url": "https://files.pythonhosted.org/packages/43/2d/e27173376ee7421e173de55bf96f0d3e0ee895a35975e0f5d6fb5e76a882/fastthreadpool-1.2.7.tar.gz" } ], "1.2.8": [ { "comment_text": "", "digests": { "md5": "733e98bc8e339b46360200c6bb52344f", "sha256": "2feb648d38a9033fe2493e1cf8d14c085ce807ed01cf66fa3fa809cfdbd2c65f" }, "downloads": -1, "filename": "fastthreadpool-1.2.8.tar.gz", "has_sig": false, "md5_digest": "733e98bc8e339b46360200c6bb52344f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13259, "upload_time": "2018-03-27T18:53:31", "url": "https://files.pythonhosted.org/packages/76/5e/e3b9742712b8632e7f27c7028f28334feb5e483bf129e24d0e9276c09abf/fastthreadpool-1.2.8.tar.gz" } ], "1.2.9": [ { "comment_text": "", "digests": { "md5": "919a6fcd3f8459a1bb9faf225ce5348b", "sha256": "aab2ddc88fd3a1bd1ecf56e501bbeece6ab0651bab31892c1f111f5fb0e06d41" }, "downloads": -1, "filename": "fastthreadpool-1.2.9.tar.gz", "has_sig": false, "md5_digest": "919a6fcd3f8459a1bb9faf225ce5348b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17473, "upload_time": "2018-03-30T21:31:56", "url": "https://files.pythonhosted.org/packages/ca/d0/38631340d65e4c3916f292b72efcb1b8b461938b752e403c2c727a886d66/fastthreadpool-1.2.9.tar.gz" } ], "1.3.0": [ { "comment_text": "", "digests": { "md5": "576d45cfe77b10b851bdbbf9a95b3100", "sha256": "435e5e8a247a7e2b18231645eeee59fe600247bf24ec5eb96e43ee0bcb6dd004" }, "downloads": -1, "filename": "fastthreadpool-1.3.0.tar.gz", "has_sig": false, "md5_digest": "576d45cfe77b10b851bdbbf9a95b3100", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14007, "upload_time": "2018-07-27T16:36:59", "url": "https://files.pythonhosted.org/packages/88/55/7e024daddd3363ca991e45f7abd4755d87a0df774e4c8529ee314414bd2e/fastthreadpool-1.3.0.tar.gz" } ], "1.3.1": [ { "comment_text": "", "digests": { "md5": "524bbc9bcae10c36bbd8f9566e222fa5", "sha256": "327b5f8cc85cb44b12641ea126d4c6476570c9a57041a220c9e5f316168077f5" }, "downloads": -1, "filename": "fastthreadpool-1.3.1.tar.gz", "has_sig": false, "md5_digest": "524bbc9bcae10c36bbd8f9566e222fa5", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10594, "upload_time": "2018-08-25T20:57:59", "url": "https://files.pythonhosted.org/packages/12/07/aa70e97c9f0e102a51787d3d55adb7c9698a954877776b27b37b99863d5e/fastthreadpool-1.3.1.tar.gz" } ], "1.3.2": [ { "comment_text": "", "digests": { "md5": "89f7d6a015de6e8c88b5a831639fd12b", "sha256": "a48bcbef3e00d3803713109f8340d724b6aa6db9b3fc6a21e121d89ac6e69c4a" }, "downloads": -1, "filename": "fastthreadpool-1.3.2.tar.gz", "has_sig": false, "md5_digest": "89f7d6a015de6e8c88b5a831639fd12b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13940, "upload_time": "2018-08-28T06:07:12", "url": "https://files.pythonhosted.org/packages/11/a0/9383cd8aa81366e15722976c80e4886d0e09608c9218a1ae0cec04947ea9/fastthreadpool-1.3.2.tar.gz" } ], "1.3.3": [ { "comment_text": "", "digests": { "md5": "1121db3b413293a08ecbcae48dbf75ba", "sha256": "86b2a1c4af4a651837dc0c5816d0a6cfbcb2cdf95e96abbd2ab7cfed4ac03526" }, "downloads": -1, "filename": "fastthreadpool-1.3.3.tar.gz", "has_sig": false, "md5_digest": "1121db3b413293a08ecbcae48dbf75ba", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13984, "upload_time": "2018-08-28T06:14:59", "url": "https://files.pythonhosted.org/packages/eb/0b/c31f5fe36955be58126d020b962388332432c7110c585177052790726ab4/fastthreadpool-1.3.3.tar.gz" } ], "1.3.4": [ { "comment_text": "", "digests": { "md5": "76181efe1b323b3c46f509381cbe3fb8", "sha256": "457ed43dda681576bdc1bda5228312302052c2c3a4c765df973a953c8db2fd34" }, "downloads": -1, "filename": "fastthreadpool-1.3.4.tar.gz", "has_sig": false, "md5_digest": "76181efe1b323b3c46f509381cbe3fb8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13993, "upload_time": "2018-08-28T06:20:11", "url": "https://files.pythonhosted.org/packages/c2/f2/7e77fd64c1b660ba4ce5800676e366f219b0bc217e375b1f472929399226/fastthreadpool-1.3.4.tar.gz" } ], "1.3.5": [ { "comment_text": "", "digests": { "md5": "f62b1a92d2c3ed5f7f6750a1ebe665f0", "sha256": "a41e7e88a70d1a5a1d1e328002f585db86722d0fa6a661e28a93f5a3aa0a25bb" }, "downloads": -1, "filename": "fastthreadpool-1.3.5.tar.gz", "has_sig": false, "md5_digest": "f62b1a92d2c3ed5f7f6750a1ebe665f0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13903, "upload_time": "2018-08-28T06:36:07", "url": "https://files.pythonhosted.org/packages/eb/ae/8f6a2119331fa9621d957d71de703e08fbb6b9660a8df8640a3254f63f16/fastthreadpool-1.3.5.tar.gz" } ], "1.4.1": [ { "comment_text": "", "digests": { "md5": "5be73ea0406455dfc23c30f79e686119", "sha256": "be0fc03a92758fc63301919462e7db3a2d11301dc6331d9d7d300d8f677ddbe6" }, "downloads": -1, "filename": "fastthreadpool-1.4.1.tar.gz", "has_sig": false, "md5_digest": "5be73ea0406455dfc23c30f79e686119", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14103, "upload_time": "2018-09-24T12:15:16", "url": "https://files.pythonhosted.org/packages/29/bc/4b31e1d89760a06f2b45c81657fd3a4c0db5c6f881cd2e3184b16694f60d/fastthreadpool-1.4.1.tar.gz" } ], "1.5.1": [ { "comment_text": "", "digests": { "md5": "6ad5810dcaddf082fdca97a58174f5b3", "sha256": "6993455c6d7417baa8e06a3c9db709dc32462b177d14f65901bd49d433a23936" }, "downloads": -1, "filename": "fastthreadpool-1.5.1.tar.gz", "has_sig": false, "md5_digest": "6ad5810dcaddf082fdca97a58174f5b3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13480, "upload_time": "2019-07-08T03:32:07", "url": "https://files.pythonhosted.org/packages/7c/34/f5cbd8ad844e7f12745c8c8d8578de164dbb21590796d57252519092139e/fastthreadpool-1.5.1.tar.gz" } ], "1.5.2": [ { "comment_text": "", "digests": { "md5": "12713bf534d83abd38b36114261af7e0", "sha256": "9b0f0c9d449faa982e1ee416658ff2884f9b7ed974f219d28a50c080844df48d" }, "downloads": -1, "filename": "fastthreadpool-1.5.2.tar.gz", "has_sig": false, "md5_digest": "12713bf534d83abd38b36114261af7e0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13501, "upload_time": "2019-07-22T07:24:36", "url": "https://files.pythonhosted.org/packages/7d/88/aa72ec6fcbf37c81e6546f1f7febe5cba7a6a783db6bf643d03e20649c3f/fastthreadpool-1.5.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "12713bf534d83abd38b36114261af7e0", "sha256": "9b0f0c9d449faa982e1ee416658ff2884f9b7ed974f219d28a50c080844df48d" }, "downloads": -1, "filename": "fastthreadpool-1.5.2.tar.gz", "has_sig": false, "md5_digest": "12713bf534d83abd38b36114261af7e0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13501, "upload_time": "2019-07-22T07:24:36", "url": "https://files.pythonhosted.org/packages/7d/88/aa72ec6fcbf37c81e6546f1f7febe5cba7a6a783db6bf643d03e20649c3f/fastthreadpool-1.5.2.tar.gz" } ] }