{ "info": { "author": "Ross McFarland", "author_email": "rwmcfa1@neces.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5" ], "description": "Asynchronous Python HTTP Requests for Humans\n============================================\n\n.. image:: https://travis-ci.org/ross/requests-futures.png?branch=master\n :target: https://travis-ci.org/ross/requests-futures\n\nSmall add-on for the python requests_ http library. Makes use of python 3.2's\n`concurrent.futures`_ or the backport_ for prior versions of python.\n\nThe additional API and changes are minimal and strives to avoid surprises.\n\nThe following synchronous code:\n\n.. code-block:: python\n\n from requests import Session\n\n session = Session()\n # first requests starts and blocks until finished\n response_one = session.get('http://httpbin.org/get')\n # second request starts once first is finished\n response_two = session.get('http://httpbin.org/get?foo=bar')\n # both requests are complete\n print('response one status: {0}'.format(response_one.status_code))\n print(response_one.content)\n print('response two status: {0}'.format(response_two.status_code))\n print(response_two.content)\n\nCan be translated to make use of futures, and thus be asynchronous by creating\na FuturesSession and catching the returned Future in place of Response. The\nResponse can be retrieved by calling the result method on the Future:\n\n.. code-block:: python\n\n from requests_futures.sessions import FuturesSession\n\n session = FuturesSession()\n # first request is started in background\n future_one = session.get('http://httpbin.org/get')\n # second requests is started immediately \n future_two = session.get('http://httpbin.org/get?foo=bar')\n # wait for the first request to complete, if it hasn't already\n response_one = future_one.result()\n print('response one status: {0}'.format(response_one.status_code))\n print(response_one.content)\n # wait for the second request to complete, if it hasn't already\n response_two = future_two.result()\n print('response two status: {0}'.format(response_two.status_code))\n print(response_two.content)\n\nBy default a ThreadPoolExecutor is created with 2 workers. If you would like to\nadjust that value or share a executor across multiple sessions you can provide\none to the FuturesSession constructor.\n\n.. code-block:: python\n\n from concurrent.futures import ThreadPoolExecutor\n from requests_futures.sessions import FuturesSession\n\n session = FuturesSession(executor=ThreadPoolExecutor(max_workers=10))\n # ...\n\nAs a shortcut in case of just increasing workers number you can pass\n`max_workers` straight to the `FuturesSession` constructor:\n\n.. code-block:: python\n\n from requests_futures.sessions import FuturesSession\n session = FuturesSession(max_workers=10)\n\nFutureSession will use an existing session object if supplied:\n\n.. code-block:: python\n\n from requests import session\n from requests_futures.sessions import FuturesSession\n my_session = session()\n future_session = FuturesSession(session=my_session)\n\nThat's it. The api of requests.Session is preserved without any modifications\nbeyond returning a Future rather than Response. As with all futures exceptions\nare shifted (thrown) to the future.result() call so try/except blocks should be\nmoved there.\n\nCanceling queued requests (a.k.a cleaning up after yourself)\n=========================\n\nIf you know that you won't be needing any additional responses from futures that \nhaven't yet resolved, it's a good idea to cancel those requests. You can do this \nby using the session as a context manager:\n\n.. code-block:: python\n\n from requests_futures.sessions import FuturesSession\n with FuturesSession(max_workers=1) as session:\n future = session.get('https://httpbin.org/get')\n future2 = session.get('https://httpbin.org/delay/10')\n future3 = session.get('https://httpbin.org/delay/10')\n response = future.result()\n\nIn this example, the second or third request will be skipped, saving time and \nresources that would otherwise be wasted.\n\nWorking in the Background\n=========================\n\nThere is one additional parameter to the various request functions,\nbackground_callback, which allows you to work with the Response objects in the\nbackground thread. This can be useful for shifting work out of the foreground,\nfor a simple example take json parsing.\n\n.. code-block:: python\n\n from pprint import pprint\n from requests_futures.sessions import FuturesSession\n\n session = FuturesSession()\n\n def bg_cb(sess, resp):\n # parse the json storing the result on the response object\n resp.data = resp.json()\n\n future = session.get('http://httpbin.org/get', background_callback=bg_cb)\n # do some other stuff, send some more requests while this one works\n response = future.result()\n print('response status {0}'.format(response.status_code))\n # data will have been attached to the response object in the background\n pprint(response.data)\n\n\n\nUsing ProcessPoolExecutor\n=========================\n\nSimilarly to `ThreadPoolExecutor`, it is possible to use an instance of\n`ProcessPoolExecutor`. As the name suggest, the requests will be executed\nconcurrently in separate processes rather than threads.\n\n.. code-block:: python\n\n from concurrent.futures import ProcessPoolExecutor\n from requests_futures.sessions import FuturesSession\n\n session = FuturesSession(executor=ProcessPoolExecutor(max_workers=10))\n # ... use as before\n\n.. HINT::\n Using the `ProcessPoolExecutor` is useful, in cases where memory\n usage per request is very high (large response) and cycling the interpretor\n is required to release memory back to OS.\n\nA base requirement of using `ProcessPoolExecutor` is that the `Session.request`,\n`FutureSession` and (the optional) `background_callback` all be pickle-able.\n\nThis means that only Python 3.5 is fully supported, while Python versions\n3.4 and above REQUIRE an existing `requests.Session` instance to be passed\nwhen initializing `FutureSession`. Python 2.X and < 3.4 are currently not\nsupported.\n\n.. code-block:: python\n\n # Using python 3.4\n from concurrent.futures import ProcessPoolExecutor\n from requests import Session\n from requests_futures.sessions import FuturesSession\n\n session = FuturesSession(executor=ProcessPoolExecutor(max_workers=10),\n session=Session())\n # ... use as before\n\nIn case pickling fails, an exception is raised pointing to this documentation.\n\n.. code-block:: python\n\n # Using python 2.7\n from concurrent.futures import ProcessPoolExecutor\n from requests import Session\n from requests_futures.sessions import FuturesSession\n\n session = FuturesSession(executor=ProcessPoolExecutor(max_workers=10),\n session=Session())\n Traceback (most recent call last):\n ...\n RuntimeError: Cannot pickle function. Refer to documentation: https://github.com/ross/requests-futures/#using-processpoolexecutor\n\n.. IMPORTANT::\n * Python >= 3.4 required\n * A session instance is required when using Python < 3.5\n * If sub-classing `FuturesSession` it must be importable (module global)\n * If using `background_callback` it too must be importable (module global)\n\n\nInstallation\n============\n\n pip install requests-futures\n\n.. _`requests`: https://github.com/kennethreitz/requests\n.. _`concurrent.futures`: http://docs.python.org/dev/library/concurrent.futures.html\n.. _backport: https://pypi.python.org/pypi/futures\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/GainCompliance/gain-requests-futures", "keywords": "", "license": "Apache License v2", "maintainer": "", "maintainer_email": "", "name": "gain-requests-futures", "package_url": "https://pypi.org/project/gain-requests-futures/", "platform": "", "project_url": "https://pypi.org/project/gain-requests-futures/", "project_urls": { "Homepage": "https://github.com/GainCompliance/gain-requests-futures" }, "release_url": "https://pypi.org/project/gain-requests-futures/0.9.7/", "requires_dist": [ "requests (>=1.2.0)" ], "requires_python": "", "summary": "Asynchronous Python HTTP for Humans.", "version": "0.9.7" }, "last_serial": 3695823, "releases": { "0.9.7": [ { "comment_text": "", "digests": { "md5": "8ac6c831d150c4ec63caa66a2bb9d508", "sha256": "848377c2f829c933fcdb40e7f1f07ac2c13a4f6ce1c1ab724d77658049f5b312" }, "downloads": -1, "filename": "gain_requests_futures-0.9.7-py2-none-any.whl", "has_sig": false, "md5_digest": "8ac6c831d150c4ec63caa66a2bb9d508", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 8925, "upload_time": "2018-03-22T15:35:57", "url": "https://files.pythonhosted.org/packages/6a/8e/40ba97ddf918c3a9196b20bd6855cfa3fea67452e305f92e8f6fe5d466f5/gain_requests_futures-0.9.7-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4930e31ebde0984364b334602e6c0957", "sha256": "0e1b42f01967affa960da792200ba248dce5712e3656fc121b387dc3743c794d" }, "downloads": -1, "filename": "gain_requests_futures-0.9.7-py3-none-any.whl", "has_sig": false, "md5_digest": "4930e31ebde0984364b334602e6c0957", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 8908, "upload_time": "2018-03-22T15:34:39", "url": "https://files.pythonhosted.org/packages/b2/c3/11d8a5ebbc6d1a4710f242440a4f9c7b4b96ec6d89d4593ceaa6f647af20/gain_requests_futures-0.9.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "89bb05904555d7204a59ed3795df70ab", "sha256": "8c4e4ec67866479eb99fd931396e5de34714ec9b05c68889a79516c400e2f8e6" }, "downloads": -1, "filename": "gain-requests-futures-0.9.7.tar.gz", "has_sig": false, "md5_digest": "89bb05904555d7204a59ed3795df70ab", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8219, "upload_time": "2018-03-22T15:34:40", "url": "https://files.pythonhosted.org/packages/7b/f0/4aefb44d04b507a1ac4fce82f0cffe82eb71d1c932e6761f3a3814e134b4/gain-requests-futures-0.9.7.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "8ac6c831d150c4ec63caa66a2bb9d508", "sha256": "848377c2f829c933fcdb40e7f1f07ac2c13a4f6ce1c1ab724d77658049f5b312" }, "downloads": -1, "filename": "gain_requests_futures-0.9.7-py2-none-any.whl", "has_sig": false, "md5_digest": "8ac6c831d150c4ec63caa66a2bb9d508", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 8925, "upload_time": "2018-03-22T15:35:57", "url": "https://files.pythonhosted.org/packages/6a/8e/40ba97ddf918c3a9196b20bd6855cfa3fea67452e305f92e8f6fe5d466f5/gain_requests_futures-0.9.7-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "4930e31ebde0984364b334602e6c0957", "sha256": "0e1b42f01967affa960da792200ba248dce5712e3656fc121b387dc3743c794d" }, "downloads": -1, "filename": "gain_requests_futures-0.9.7-py3-none-any.whl", "has_sig": false, "md5_digest": "4930e31ebde0984364b334602e6c0957", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 8908, "upload_time": "2018-03-22T15:34:39", "url": "https://files.pythonhosted.org/packages/b2/c3/11d8a5ebbc6d1a4710f242440a4f9c7b4b96ec6d89d4593ceaa6f647af20/gain_requests_futures-0.9.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "89bb05904555d7204a59ed3795df70ab", "sha256": "8c4e4ec67866479eb99fd931396e5de34714ec9b05c68889a79516c400e2f8e6" }, "downloads": -1, "filename": "gain-requests-futures-0.9.7.tar.gz", "has_sig": false, "md5_digest": "89bb05904555d7204a59ed3795df70ab", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8219, "upload_time": "2018-03-22T15:34:40", "url": "https://files.pythonhosted.org/packages/7b/f0/4aefb44d04b507a1ac4fce82f0cffe82eb71d1c932e6761f3a3814e134b4/gain-requests-futures-0.9.7.tar.gz" } ] }