{ "info": { "author": "Alexander Grechin", "author_email": "infinum@mail.ru", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: BSD License", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Software Development", "Topic :: Utilities" ], "description": "====================================\nYandex-tank input/output file parser\n====================================\n\n.. image:: https://travis-ci.org/travis-ci/travis-web.svg?branch=master\n :target: https://travis-ci.org/travis-ci/travis-web\n\n.. image:: https://codecov.io/gh/gaainf/tanktools/branch/master/graph/badge.svg\n :target: https://codecov.io/gh/gaainf/tanktools/\n\n.. image:: https://img.shields.io/badge/python-2.7-blue.svg\n :target: https://www.python.org/downloads/release/python-270/\n\n.. image:: https://img.shields.io/badge/python-3.5-blue.svg\n :target: https://www.python.org/downloads/release/python-350/\n\n.. image:: https://img.shields.io/badge/python-3.6-blue.svg\n :target: https://www.python.org/downloads/release/python-360/\n\n.. image:: https://img.shields.io/pypi/l/tanktools.svg\n :target: https://github.com/gaainf/tanktools/blob/master/LICENSE\n\n\nYandex-tank prepare `phout` file with statistics after load testing.\n**tanktools** module helps to parse such files and convert to DataFrame.\nYou can use **pandas** module in manual mode to handle DataFrame\nor use build-in functions.\n\nGenerate Yandex-tank ammo from pcap or har files using `pcap2ammo`\nor `har2ammo` scrips. HTTP requests are extracted completely\nwith headers and body.\n\nSo you can:\n\n- calc quantiles\n\n- get information about timings, latency, status codes\n\n- extract requests by timestamp, tag and other columns\n\n- group and analyze specific data like total/partial RPS,\n average request/response size\n\n- calc statistical metrics\n\n- convert pcap file to ammo\n\n- convert har file to ammo\n\n- filter out and modify requests on ammo generating\n\n\n************\nInstallation\n************\n.. code:: python\n\n pip install tanktools\n\n********\nExamples\n********\n\nSelect DataFrame by timestamp\n*****************************\n\nIt is possible to parse a part of staistics by time and overal count\n\n.. code:: python\n\n from tanktools import phout\n flags = {\n 'from_date': '2018-01-18 20:09:50.123',\n 'to_date' : '2018-01-18 20:10:00.456',\n 'limit': 100\n }\n data = phout.parse_phout('phout.log', flags)\n print(\"Total count: %d\" % phout.size(data))\n\n.. code::\n\n Total count: 100\n\nPrint percentiles\n*****************\n.. code:: python\n\n data = phout.parse_phout('phout.log')\n phout.print_quantiles(data, 'receive_time')\n\n.. code::\n\n Percentiles for 150030 requests\n from 2018-01-18 20:09:42.983\n to 2018-01-18 20:10:55.108:\n quantile (%) receive_time (mks)\n 10.0 9\n 20.0 9\n 30.0 10\n 40.0 10\n 50.0 10\n 60.0 10\n 70.0 11\n 80.0 12\n 90.0 13\n 95.0 14\n 98.0 16\n 99.0 17\n 100.0 716\n\n\n.. note::\n\n Pay attention, timings are calculated in microseconds.\n\nPrint latency median\n************************\n\n.. code:: python\n\n data = phout.parse_phout('phout.log')\n # Convert and print timing in milliseconds\n print(\"\\n\\nLatency median: %d ms\" % int(data.latency.median() / 1000))\n\n.. code::\n\n Latency median: 30 ms\n\nGet RPS\n*******\n\n.. code:: python\n\n data = phout.parse_phout('phout.log')\n rps = phout.get_rps(data)\n\nPrint HTTP response statistics\n*******************************\n\n.. code:: python\n\n data = phout.parse_phout('phout.log')\n phout.print_http_reponses(data)\n\n.. code::\n\n HTTP code count percent (%)\n 500 83429 56.38\n 200 61558 41.60\n 502 2944 1.99\n 0 41 0.03\n\nSelect 200 OK responses and print latency median\n************************************************\n\n.. code:: python\n\n data = phout.parse_phout('phout.log')\n selected_http_responses = data[data.proto_code == 200]\n print(\"Latency median for 200 OK: %d\" %\n selected_http_responses.latency.median())\n\n.. code::\n\n Latency median for 200 OK: 3539\n\nPrint average request/response size\n***********************************\n\n.. code:: python\n\n print(\"Avg. Request / Response: %d / %d bytes\" % (\n data.size_in.astype(float).mean(),\n data.size_out.astype(float).mean()\n ))\n\n.. code::\n\n Avg. Request / Response: 364 / 26697 bytes\n\n.. note::\n\n Pay attention it is required to convert data to float for correct work of ``mean`` function\n\nPrint RPS at Nth request\n************************\n\n.. code:: python\n\n print(\"RPS at request:\")\n chunk_size = int(phout.size(data) / 2)\n for start in range(0, phout.size(data), chunk_size):\n data_subset = phout.subset(data, start, chunk_size)\n print(\"\\t%s: %.2f\" %\n (start + chunk_size, phout.get_rps(data_subset)))\n\n.. code::\n\n RPS at request:\n 73986: 2062.50\n 147972: 2530.56\n\n\n*********\npcap2ammo\n*********\n\nConvert pcap file to Yandex-tank ammo\n*************************************\n\n.. code:: bash\n\n pcap2ammo file.pcap\n\n.. code::\n\n 73\n GET https://rambler.ru/ HTTP/1.1\\r\\n\n Host: rambler.ru\\r\\n\n Content-Length: 0\\r\\n\\r\\n\n\nCount statistics about HTTP requests\n***************************************\n\n.. code:: bash\n\n pcap2ammo -S file.pcap\n\n Stats:\n total: 1\n complete: 1\n incorrect: 0\n incomplete: 0\n\nPrint to file\n*************************************\n\n.. code:: bash\n\n pcap2ammo -o out.ammo file.pcap\n\nAdd or delete headers\n*********************\nApplyed for all requests, containing specified headers\n\n.. code:: bash\n\n pcap2ammo --add-header 'Referer: http://domain.com' --add-header 'X-Ip: 1.1.1.1' file.pcap\n\n.. code:: bash\n\n pcap2ammo --delete-header 'Content-Length' file.pcap\n pcap2ammo --delete-header 'Connection' --add-header 'Connection: keep-alive' file.pcap\n\nFilter TCP/IP packets\n*********************\n\n.. code:: bash\n\n pcap2ammo -f 'ip.src==10.10.10.10 and tcp.dport==8080' file.pcap\n\nFilter HTTP packets\n*********************\n\n.. code:: bash\n\n pcap2ammo -F '\"rambler.ru\" in http.uri' file.pcap\n\nYou can use logical expressions in filters\n\n.. code:: bash\n\n pcap2ammo -F '\"keep-alive\" in http.headers[\"connection\"] or \"Keep-alive\" in http.headers[\"connection\"]' file.pcap\n\nString functions over HTTP headers\n\n.. code:: bash\n\n pcap2ammo -F '\"keep-alive\" in http.headers[\"connection\"].lower()' file.pcap\n\nUse excluding filters also\n\n.. code:: bash\n\n pcap2ammo -F '\"rambler.ru\" != http.headers[\"host\"]' file.pcap\n\n*********\nhar2ammo\n*********\n\nConvert pcap file to Yandex-tank ammo\n*************************************\n\n.. code:: bash\n\n har2ammo file.har\n\n.. code::\n\n 73\n GET https://rambler.ru/ HTTP/1.1\\r\\n\n Host: rambler.ru\\r\\n\n Content-Length: 0\\r\\n\\r\\n\n\nCount statistics about HTTP requests\n***************************************\n\n.. code:: bash\n\n har2ammo -S file.har\n\n Stats:\n total: 1\n complete: 1\n incorrect: 0\n incomplete: 0\n\nPrint to file\n*************************************\n\n.. code:: bash\n\n har2ammo -o out.ammo file.har\n\nAdd or delete headers\n*********************\nApplyed for all requests, containing specified headers\n\n.. code:: bash\n\n har2ammo --add-header 'Referer: http://domain.com' --add-header 'X-Ip: 1.1.1.1' file.har\n\n.. code:: bash\n\n har2ammo --delete-header 'Content-Length' file.har\n har2ammo --delete-header 'Connection' --add-header 'Connection: keep-alive' file.har\n\nFilter HTTP packets\n*********************\n\n.. code:: bash\n\n har2ammo -F '\"rambler.ru\" in http.uri' file.har\n\nYou can use logical expressions and python functions in filters\n\n.. code:: bash\n\n har2ammo -F '\"keep-alive\" in http.headers[\"connection\"] or \"Keep-alive\" in http.headers[\"connection\"]' file.har\n har2ammo -F '\"keep-alive\" not in http.headers[\"connection\"].lower()' file.har\n\nPlease, see more information about filters in `pcaper `_ package description.\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "https://github.com/gaainf/tanktools", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/gaainf/tanktools", "keywords": "yandextank yandex-tank statistics tools utilities", "license": "BSD-3-Clause", "maintainer": "", "maintainer_email": "", "name": "tanktools", "package_url": "https://pypi.org/project/tanktools/", "platform": "", "project_url": "https://pypi.org/project/tanktools/", "project_urls": { "Download": "https://github.com/gaainf/tanktools", "Homepage": "https://github.com/gaainf/tanktools" }, "release_url": "https://pypi.org/project/tanktools/1.0.9/", "requires_dist": [ "python-dateutil (>=2.8.0)", "pandas (>=0.23.4)", "flake8 (>=3.5.0)", "pcaper (>=1.0.2)" ], "requires_python": "", "summary": "Yandex-tank tools", "version": "1.0.9" }, "last_serial": 4934111, "releases": { "1.0.2": [ { "comment_text": "", "digests": { "md5": "7adcd7e7cfc3c8b50eecb3e325d99ff3", "sha256": "b7894db572c092be0ee273b3279abc82c655bda6c19cdfd31fca2f1e9f3a3db3" }, "downloads": -1, "filename": "tanktools-1.0.2-py2-none-any.whl", "has_sig": false, "md5_digest": "7adcd7e7cfc3c8b50eecb3e325d99ff3", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 5454, "upload_time": "2018-09-23T14:43:56", "url": "https://files.pythonhosted.org/packages/2c/d4/87d058d5d08594ac630ede70642a29347908b0674c5ab69182e6d3563c27/tanktools-1.0.2-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "26c943c0ea3b2e734b877b034ad9a673", "sha256": "d38b94f903a9c543cb1163763f2d1cbbeb16e330b5172f5b358f0dbeb2c63911" }, "downloads": -1, "filename": "tanktools-1.0.2.tar.gz", "has_sig": false, "md5_digest": "26c943c0ea3b2e734b877b034ad9a673", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5410, "upload_time": "2018-09-23T14:43:57", "url": "https://files.pythonhosted.org/packages/26/8f/8638b883323df622ee6106f1254a44aceb5ff13794dcd3d633e86e3274d2/tanktools-1.0.2.tar.gz" } ], "1.0.3": [ { "comment_text": "", "digests": { "md5": "9e8a0eee676ef66b1d0356d0cdf585cd", "sha256": "883295386816e19b0d546eef97837ef1bdd3779f4b1c9bc4259fa85360f49890" }, "downloads": -1, "filename": "tanktools-1.0.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9e8a0eee676ef66b1d0356d0cdf585cd", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8102, "upload_time": "2018-11-14T13:15:25", "url": "https://files.pythonhosted.org/packages/b0/ad/91944819f229f66f2966f34f1a9b75698a3412cd4af6d369d056bf8ea4d8/tanktools-1.0.3-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2dc275e8bf46865e4bed355c347d18d8", "sha256": "6f69e3a6acdb966eb0520db0a7cc7ff303d6750ec7ef9f2407f92452b1f1cba2" }, "downloads": -1, "filename": "tanktools-1.0.3.tar.gz", "has_sig": false, "md5_digest": "2dc275e8bf46865e4bed355c347d18d8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7539, "upload_time": "2018-11-14T13:15:27", "url": "https://files.pythonhosted.org/packages/19/4a/52e5cda87402676d7d95040a0c8a2cefcad734ec6362103d2c8249eebfeb/tanktools-1.0.3.tar.gz" } ], "1.0.4": [ { "comment_text": "", "digests": { "md5": "a85478873eba6994fa2a04b0abcf08be", "sha256": "037d32bb5e98d0b37ddbfbee167ec9798d589b52b06f96859b35f74a69e394d7" }, "downloads": -1, "filename": "tanktools-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a85478873eba6994fa2a04b0abcf08be", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8377, "upload_time": "2018-12-25T22:44:01", "url": "https://files.pythonhosted.org/packages/22/fe/74dc70a64fd27a41c52fc331d0c82e18797a13ba7d4e094f3e49ccc6f5b5/tanktools-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3a94b347e2d5fe6db777fd7ff19c956b", "sha256": "c2ddb723fa91e1e351bb09e5384d1a5fa21464173802b96a3bf224b8192b1939" }, "downloads": -1, "filename": "tanktools-1.0.4.tar.gz", "has_sig": false, "md5_digest": "3a94b347e2d5fe6db777fd7ff19c956b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7826, "upload_time": "2018-12-25T22:44:03", "url": "https://files.pythonhosted.org/packages/ce/27/3f1a68d717fdddedb904d648f075526d5f77ec9501aa8c72c68667caa626/tanktools-1.0.4.tar.gz" } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "08180489297269484a5848082e97fd1e", "sha256": "b57adc0bed6058d31b993140139b13fd05d6e663024b0e332654dba44e063d6d" }, "downloads": -1, "filename": "tanktools-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "08180489297269484a5848082e97fd1e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 8268, "upload_time": "2019-01-13T16:58:13", "url": "https://files.pythonhosted.org/packages/1f/0e/50cb2113ce2359f0fce71212ce36bd5c7a39872e9a8fe5c4d34471d1142a/tanktools-1.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "72e17eaaa5f77816aad47060084ca2dd", "sha256": "79ea8d4f91f2b37c07dbf81a981e85415cb2d16c1f5d60fe65a4b271d2f48af2" }, "downloads": -1, "filename": "tanktools-1.0.5.tar.gz", "has_sig": false, "md5_digest": "72e17eaaa5f77816aad47060084ca2dd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7735, "upload_time": "2019-01-13T16:58:14", "url": "https://files.pythonhosted.org/packages/c5/4f/3a610e3fee9de9efdfeab1d18ca3419821c86b5ebcd941697d4f78a9069a/tanktools-1.0.5.tar.gz" } ], "1.0.6": [ { "comment_text": "", "digests": { "md5": "72f3aad4c3d592f3cc53fdc52184256e", "sha256": "528666dab702b6f6fbf1baab2d5ca34ba9d7f8e055dba8253281a36b72095fc6" }, "downloads": -1, "filename": "tanktools-1.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "72f3aad4c3d592f3cc53fdc52184256e", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10069, "upload_time": "2019-03-13T03:48:08", "url": "https://files.pythonhosted.org/packages/ee/68/5e30ec809eccf3e024a6963fe3fec1de933c889dd17873bc363b481e20a3/tanktools-1.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c338aa98df0b618a5159ecf81ce90d6c", "sha256": "edb89e7ad4b910086f1038d627cc139b6f5a0d7309be7a9c1aa0105214d9fd77" }, "downloads": -1, "filename": "tanktools-1.0.6.tar.gz", "has_sig": false, "md5_digest": "c338aa98df0b618a5159ecf81ce90d6c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10179, "upload_time": "2019-03-13T03:48:09", "url": "https://files.pythonhosted.org/packages/d0/d0/4d181cf14aaf18395d6e0e0c9f6b8221943b4c771294a8bcfd48eb555b8b/tanktools-1.0.6.tar.gz" } ], "1.0.7": [ { "comment_text": "", "digests": { "md5": "74076e8396e41a9df2d9a4deae7898b6", "sha256": "20ef8b108d970c91fb03dc6a31991c00ba941743f2be50b2519c64bd3716af91" }, "downloads": -1, "filename": "tanktools-1.0.7-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "74076e8396e41a9df2d9a4deae7898b6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10072, "upload_time": "2019-03-13T10:24:39", "url": "https://files.pythonhosted.org/packages/5c/d2/07deb20a52cfaf94fd5f97ac527f36e8b9abf9c2f322321bd574334d6d66/tanktools-1.0.7-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "47ed1612d6e7925aada2a8da63fd2367", "sha256": "cc8c738ede7d96181b8ececce33e96342950cdf0edc4259f513adf536010a5ba" }, "downloads": -1, "filename": "tanktools-1.0.7.tar.gz", "has_sig": false, "md5_digest": "47ed1612d6e7925aada2a8da63fd2367", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10193, "upload_time": "2019-03-13T10:24:40", "url": "https://files.pythonhosted.org/packages/cd/28/d2b7f8de244b643dd8c48b6603c73bb5f23dba8ea75d9ed0848d3aabd54d/tanktools-1.0.7.tar.gz" } ], "1.0.8": [ { "comment_text": "", "digests": { "md5": "e195ac342ca3ec9d8b49be7ce0b92c7f", "sha256": "1735908bcd4dc2b67296636efc7f2219597fc0af1fc7bdb8e57ce611c41acb8b" }, "downloads": -1, "filename": "tanktools-1.0.8-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "e195ac342ca3ec9d8b49be7ce0b92c7f", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10065, "upload_time": "2019-03-13T10:59:32", "url": "https://files.pythonhosted.org/packages/43/da/3b7024c4017d2560bf82f8748d9eec74be38e24d3b2caa2eaa7fd39b67f1/tanktools-1.0.8-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "67cfb123d919a38cb50ab36796c34639", "sha256": "fcce68c60807f515878a2596545d2f67f9d491a117fc8c67555beaf93ef9c1e8" }, "downloads": -1, "filename": "tanktools-1.0.8.tar.gz", "has_sig": false, "md5_digest": "67cfb123d919a38cb50ab36796c34639", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10185, "upload_time": "2019-03-13T10:59:34", "url": "https://files.pythonhosted.org/packages/19/65/f9968a564110e784f7ea66cae733702e702ba4972ed4a493256cd307ac13/tanktools-1.0.8.tar.gz" } ], "1.0.9": [ { "comment_text": "", "digests": { "md5": "c16a091de04be80f206eef6d18c71575", "sha256": "5f372f89693c7e2321e3e54de9a810b1151a81d99d92877a7c978f094e3d8a8d" }, "downloads": -1, "filename": "tanktools-1.0.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c16a091de04be80f206eef6d18c71575", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10065, "upload_time": "2019-03-13T11:03:12", "url": "https://files.pythonhosted.org/packages/ed/28/008100dbd0cd4c64551cb382625dc208d586e3ed4fb34f705994da736682/tanktools-1.0.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ef5ff355bae7d5a964040d735c59469d", "sha256": "187ea15b7a0f7b49c2d2e8d281adda506d2350d45ec6a8927ef99ca8db55d107" }, "downloads": -1, "filename": "tanktools-1.0.9.tar.gz", "has_sig": false, "md5_digest": "ef5ff355bae7d5a964040d735c59469d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10186, "upload_time": "2019-03-13T11:03:13", "url": "https://files.pythonhosted.org/packages/ef/34/2b1e3a66b2923ae1fd95af03b86a8ec67c0ad92ab02070a72b71ce94b505/tanktools-1.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "c16a091de04be80f206eef6d18c71575", "sha256": "5f372f89693c7e2321e3e54de9a810b1151a81d99d92877a7c978f094e3d8a8d" }, "downloads": -1, "filename": "tanktools-1.0.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "c16a091de04be80f206eef6d18c71575", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10065, "upload_time": "2019-03-13T11:03:12", "url": "https://files.pythonhosted.org/packages/ed/28/008100dbd0cd4c64551cb382625dc208d586e3ed4fb34f705994da736682/tanktools-1.0.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ef5ff355bae7d5a964040d735c59469d", "sha256": "187ea15b7a0f7b49c2d2e8d281adda506d2350d45ec6a8927ef99ca8db55d107" }, "downloads": -1, "filename": "tanktools-1.0.9.tar.gz", "has_sig": false, "md5_digest": "ef5ff355bae7d5a964040d735c59469d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10186, "upload_time": "2019-03-13T11:03:13", "url": "https://files.pythonhosted.org/packages/ef/34/2b1e3a66b2923ae1fd95af03b86a8ec67c0ad92ab02070a72b71ce94b505/tanktools-1.0.9.tar.gz" } ] }