{ "info": { "author": "Robert Lechte", "author_email": "robert.lechte@dpc.vic.gov.au", "bugtrack_url": null, "classifiers": [ "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.7" ], "description": "# ratemate\n\nThere's a bunch of python rate limiting modules out there, but they all seem to suffer from similar problems:\n\n- Weird APIs, usually inflexible decorators that you need to wrap your calls in\n- Lack of `multiprocessing` support (eg, two processes will be unaware of each other, and thus double the intended rate)\n- Unnecessary coupling to other libraries\n\n`ratemate`, meanwhile, gives you a simple `RateLimit` object that avoids all these problems.\n\nIt works like this. Declare a `RateLimit` as follows:\n\n from ratemate import RateLimit\n\n rate_limit = RateLimit(max_count=2, per=5) # 2 requests per 5 seconds\n\nThen call `.wait()` appropriately when you need to limit the rate.\n\nFor instance, here's an example when creating multiple threads with `concurrent.futures`. First the original rate-unlimited code:\n\n```python\n\nfrom concurrent.futures import ThreadPoolExecutor, as_completed\n\n\ndef task(n):\n print(f\" task {n} called\")\n return n\n\nfutures = []\n\nwith ThreadPoolExecutor() as executor:\n for i in range(20):\n future = executor.submit(task, i)\n futures.append(future)\n\n for completed in as_completed(futures):\n result = completed.result()\n print('completed')\n```\n\nAdd rate-limiting simply by adding a wait at the appropriate time, either at task creation:\n\n```python\nfor i in range(20):\n rate_limit.wait() # wait before creating the task\n future = executor.submit(task, i)\n futures.append(future)\n```\n\nOr at the start of the task itself:\n\n```python\ndef task(n):\n waited_time = rate_limit.wait() # wait at start of task\n print(f\" task {n}: waited for {waited_time} secs\")\n return n\n```\n\nBecause `ratemate` uses multi-process-aware shared memory to track its state, you can also use `ProcessPoolExecutor` and everything will still work nicely.\n\n\n## Greedy mode\n\nThe default (aka non-greedy aka patient) rate limiting mode spaces out calls evenly. First instance, max_count=10 and per=60 will result in one call every 6 seconds.\n\nYou may instead wish for calls to happen as fast as possible, only slowing down if the limit would be exceeded. Enable this with greedy=True, eg:\n\n```\nrate_limit = RateLimit(max_count=20, per=60, greedy=True)\n```\n\n## Further enhancements\n\nRate limit coordination between truly independent processes (not just subprocesses), possibly using Python 3.8's new shared memory or Redis or PostgreSQL or whatever.\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "", "maintainer": "Robert Lechte", "maintainer_email": "robert.lechte@dpc.vic.gov.au", "name": "ratemate", "package_url": "https://pypi.org/project/ratemate/", "platform": "", "project_url": "https://pypi.org/project/ratemate/", "project_urls": null, "release_url": "https://pypi.org/project/ratemate/0.1.0/", "requires_dist": null, "requires_python": ">=3.7", "summary": "A python rate limiter module with multi-process support and a simple, intuitive API", "version": "0.1.0" }, "last_serial": 5958054, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "d1a526379dce5686de960a5e939eebd9", "sha256": "5660bfced1a39682fa98103ed4103d46c237e4064256dfde2f29a9e58549d59e" }, "downloads": -1, "filename": "ratemate-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d1a526379dce5686de960a5e939eebd9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.7", "size": 4143, "upload_time": "2019-10-09T03:42:39", "url": "https://files.pythonhosted.org/packages/32/46/71693a0bf8eb4aa329d60e3c16c0add85872b8cdb853475a58134d61ff19/ratemate-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3c2621d544d876e2ad49c037279c4afe", "sha256": "681e36441152f100480ec5e3223ea1a18335094a55abe36e952fa067bfd63352" }, "downloads": -1, "filename": "ratemate-0.1.0.tar.gz", "has_sig": false, "md5_digest": "3c2621d544d876e2ad49c037279c4afe", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.7", "size": 4097, "upload_time": "2019-10-09T03:42:42", "url": "https://files.pythonhosted.org/packages/f1/a6/a639f4ba6432c7b56a0f0bf7528f20ef5e73ac4706475ba18f798d353fa2/ratemate-0.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "d1a526379dce5686de960a5e939eebd9", "sha256": "5660bfced1a39682fa98103ed4103d46c237e4064256dfde2f29a9e58549d59e" }, "downloads": -1, "filename": "ratemate-0.1.0-py3-none-any.whl", "has_sig": false, "md5_digest": "d1a526379dce5686de960a5e939eebd9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": ">=3.7", "size": 4143, "upload_time": "2019-10-09T03:42:39", "url": "https://files.pythonhosted.org/packages/32/46/71693a0bf8eb4aa329d60e3c16c0add85872b8cdb853475a58134d61ff19/ratemate-0.1.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3c2621d544d876e2ad49c037279c4afe", "sha256": "681e36441152f100480ec5e3223ea1a18335094a55abe36e952fa067bfd63352" }, "downloads": -1, "filename": "ratemate-0.1.0.tar.gz", "has_sig": false, "md5_digest": "3c2621d544d876e2ad49c037279c4afe", "packagetype": "sdist", "python_version": "source", "requires_python": ">=3.7", "size": 4097, "upload_time": "2019-10-09T03:42:42", "url": "https://files.pythonhosted.org/packages/f1/a6/a639f4ba6432c7b56a0f0bf7528f20ef5e73ac4706475ba18f798d353fa2/ratemate-0.1.0.tar.gz" } ] }