{
"info": {
"author": "Raul Gutierrez Segales",
"author_email": "rgs@pinterest.com",
"bugtrack_url": null,
"classifiers": [
"Development Status :: 3 - Alpha",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Topic :: System :: Distributed Computing",
"Topic :: System :: Networking"
],
"description": "thrift-tools |Build Status| |Coverage Status| |PyPI version|\n============================================================\n\n**Table of Contents**\n\n- `tl;dr <#tldr>`__\n- `Installing <#installing>`__\n- `Tools <#tools>`__\n- `Library <#library>`__\n- `Tests <#tests>`__\n\ntl;dr\n~~~~~\n\nthrift-tools is a library and a set of tools to introspect `Apache\nThrift `__ traffic.\n\nInstalling\n~~~~~~~~~~\n\nYou can install thrift-tools via pip:\n\n::\n\n $ pip install thrift-tools\n\nOr run it from source (if you have the dependencies installed, see\nbelow):\n\n::\n\n $ git clone ...\n $ cd thrift-tools\n $ sudo FROM_SOURCE=1 bin/thrift-tool --iface=eth0 --port 9091 dump --show-all --pretty\n ...\n $ FROM_SOURCE=1 bin/thrift-tool --port 9090 \\\n --pcap-file thrift_tools/tests/resources/calc-service-binary.pcap \\\n dump --show-all --pretty --color \\\n --idl-file=thrift_tools/tests/resources/tutorial.thrift\n\nTools\n~~~~~\n\nthrift-tool can be used in interactive mode to analyze live thrift\nmessages:\n\n::\n\n $ sudo thrift-tool --iface eth0 --port 9091 dump --show-all --pretty\n [00:39:42:850848] 10.1.8.7:49858 -> 10.1.2.20:3636: method=dosomething, type=call, seqid=1120\n header: ()\n fields: [ ( 'struct',\n 1,\n [ ('string', 1, 'something to do'),\n ('i32', 3, 0),\n ( 'struct',\n 9,\n [ ('i32', 3, 2),\n ('i32', 14, 0),\n ('i32', 16, 0),\n ('i32', 18, 25)])])]\n ------>[00:39:42:856204] 10.1.2.20:3636 -> 10.1.8.7:49858: method=dosomething, type=reply, seqid=1120\n header: ()\n fields: [ ( 'struct',\n 0,\n [ ('string', 1, 'did something'),\n ('string', 2, 'did something else'),\n ('string', 3, 'did some other thing'),\n ('string', 4, 'did the last thing'),\n ('i32', 6, 3),\n ('i32', 7, 11),\n ('i32', 8, 0),\n ('i32', 9, 0),\n ('list', 10, [0]),\n ...\n\nAlternatively, offline pcap files may be introspected:\n\n::\n\n $ sudo thrift-tool --port 9091 --pcap-file /path/to/myservice.pcap dump\n ...\n\nNote that you still need to set the right port.\n\nIf you are using `Finagle `__, try\nsomething like:\n\n::\n\n $ sudo thrift-tool --iface eth0 --port 9091 dump --show-all --pretty --finagle-thrift\n ...\n\nJSON output is available for easy filtering & querying via jq. For\ninstance, you can enumerate all the IPs calling the method 'search' via:\n\n::\n\n $ sudo thrift-tool --port 3030 dump --unpaired --json | jq 'select(.method == \"search\" and .type == \"call\") | .src'\n \"10.1.18.5:48534\"\n \"10.1.60.2:52008\"\n \"10.1.10.27:49856\"\n \"10.1.23.24:48116\"\n \"10.1.26.7:60462\"\n \"10.1.11.10:41895\"\n \"10.1.15.13:35285\"\n \"10.1.7.17:39759\"\n \"10.1.1.19:35481\"\n ...\n\nGathering per method latency stats is available via the ``stats``\ncommand:\n\n::\n\n $ sudo thrift-tool --port 6666 stats --count 100\n method count avg min max p90 p95 p99 p999\n -------- ------- ---------- ---------- ---------- ---------- ---------- ---------- ----------\n search2 61 0.00860996 0.00636292 0.0188479 0.010778 0.015192 0.0174422 0.0187074\n doc 39 0.00134846 0.00099802 0.00274897 0.00177183 0.00199242 0.00256242 0.00273031\n 287 unmatched calls\n\nYou can also specify .thrift file for nicer output:\n\n::\n\n $ sudo thrift-tool --port 9091 dump --show-all --pretty --color --idl-file /path/to/myidl.thrift\n ...\n\nTo list all the available options:\n\n::\n\n $ thrift-tool --help\n\nNote that for servers with high throughput (i.e.: > couple Ks packets\nper second), it might be hard for thrift-tools to keep up because start\nof message detection is a bit expensive (and you can only go so fast\nwith Python). For these cases, you are better off saving a pcap file\n(i.e.: via tcpdump) and then post-processing it, i.e.:\n\n::\n\n $ tcpdump -nn -t port 3030 -w dump.pcap\n $ sudo thrift-tool --port 3030 --pcap-file dump.pcap stats --count 40000\n method count avg min max p90 p95 p99 p999\n -------- ------- ---------- ---------- ---------- ---------- ---------- ---------- ----------\n resize 40000 0.00850996 0.00336091 0.0101364 0.008071 0.009132 0.009890 0.01005665\n\nLibrary\n~~~~~~~\n\nTo use thrift-tools from another (Python) application, you can import it\nvia:\n\n::\n\n from thrift_tools.message_sniffer import MessageSnifferOptions, MessageSniffer\n\n options = MessageSnifferOptions(\n iface='eth0',\n port='3636',\n ip=None, # include msgs from all IPs\n pcap_file=None, # don't read from a pcap file, live sniff\n protocol=None, # auto detect protocol\n finagle_thrift=False, # apache thrift (not twitter's finagle)\n read_values=True, # read the values of each msg/struct\n max_queued=20000, # decent sized queue\n max_message_size=2000, # 2k messages to keep mem usage frugal\n debug=False # don't print parsing errors, etc\n )\n\n def printer(timestamp, src, dst, msg):\n print '%s %s %s %s' % (timestamp, src, dst, msg)\n\n message_sniffer = MessageSniffer(options, printer)\n\n # loop forever\n message_sniffer.join()\n\nOf if you want to use a pcap file:\n\n::\n\n options = MessageSnifferOptions(\n iface='eth0',\n port='3636',\n ip=None,\n pcap_file=\"/tmp/myservice.pcap\",\n protocol=None,\n finagle_thrift=False,\n read_values=True,\n max_queued=20000,\n max_message_size=2000,\n debug=False\n )\n\n ...\n\nIf you want to filter messages for specific IPs:\n\n::\n\n options = MessageSnifferOptions(\n iface='eth0',\n port='3636',\n ip=['172.16.24.3', '172.16.24.4'], # ignores everyone else\n pcap_file=\"/tmp/myservice.pcap\",\n protocol=None,\n finagle_thrift=False,\n read_values=True,\n max_queued=20000,\n max_message_size=2000,\n debug=False\n )\n\n ...\n\nSee examples/ for more ways to use this library!\n\nTests\n~~~~~\n\nTo run the tests:\n\n::\n\n $ python setup.py nosetests\n\n.. |Build Status| image:: https://travis-ci.org/pinterest/thrift-tools.svg?branch=master\n :target: https://travis-ci.org/pinterest/thrift-tools\n.. |Coverage Status| image:: https://coveralls.io/repos/pinterest/thrift-tools/badge.svg?branch=master&service=github\n :target: https://coveralls.io/github/pinterest/thrift-tools?branch=master\n.. |PyPI version| image:: https://badge.fury.io/py/thrift-tools.svg\n :target: http://badge.fury.io/py/thrift-tools\n\n\n",
"description_content_type": "",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/pinterest/thrift-tools",
"keywords": "thrift pcap",
"license": "Apache",
"maintainer": "",
"maintainer_email": "",
"name": "thrift-tools",
"package_url": "https://pypi.org/project/thrift-tools/",
"platform": "",
"project_url": "https://pypi.org/project/thrift-tools/",
"project_urls": {
"Homepage": "https://github.com/pinterest/thrift-tools"
},
"release_url": "https://pypi.org/project/thrift-tools/0.0.6/",
"requires_dist": [
"ansicolors",
"dpkt (==1.9.2)",
"ptsd (==0.2.0)",
"scapy (==2.4.2)",
"thrift (==0.11.0)",
"tabulate",
"ansicolors ; extra == 'test'",
"dpkt ; extra == 'test'",
"nose ; extra == 'test'",
"ptsd (==0.2.0) ; extra == 'test'",
"scapy (==2.4.2) ; extra == 'test'",
"six (==1.12.0) ; extra == 'test'",
"thrift (==0.11.0) ; extra == 'test'",
"tabulate ; extra == 'test'"
],
"requires_python": "",
"summary": "Thrift protocol analyzer",
"version": "0.0.6"
},
"last_serial": 5746044,
"releases": {
"0.0.1": [
{
"comment_text": "",
"digests": {
"md5": "cc614c7fe88ba85590aea119e908ed5b",
"sha256": "f3547aa5711f3d60f6054c6ceff9cdd524e8b6e38029daf3c6403365c12b910a"
},
"downloads": -1,
"filename": "thrift-tools-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "cc614c7fe88ba85590aea119e908ed5b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 24440,
"upload_time": "2015-09-05T18:22:37",
"url": "https://files.pythonhosted.org/packages/87/41/b0e6e8b0650a08bc84b31267621d9daa654207f571089d5366296576929f/thrift-tools-0.0.1.tar.gz"
}
],
"0.0.2": [
{
"comment_text": "",
"digests": {
"md5": "f46c098ea40b5507f92e9cf984ba49fc",
"sha256": "020af0f2fd27e7adcee7beababf88db0b9fb607d2040df7d2340093b78a8a3e4"
},
"downloads": -1,
"filename": "thrift-tools-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "f46c098ea40b5507f92e9cf984ba49fc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 27424,
"upload_time": "2015-09-26T16:21:30",
"url": "https://files.pythonhosted.org/packages/12/ba/56bb5520fe0894ae19268d1ee58fa9c550edb5d3ee71c519b7adf6a04fe4/thrift-tools-0.0.2.tar.gz"
}
],
"0.0.3": [
{
"comment_text": "",
"digests": {
"md5": "3814d673497039159c1df70e7c70d5c6",
"sha256": "959e61ccd6fdb9ec291c0d2260590b77db1a5599ab56288bf9bb1d5228eba992"
},
"downloads": -1,
"filename": "thrift-tools-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "3814d673497039159c1df70e7c70d5c6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 29223,
"upload_time": "2015-11-04T19:33:27",
"url": "https://files.pythonhosted.org/packages/b8/22/232bcf1f8ba75fa7bc3673bc61f64e8cdb95af7270f3e476e810597f7751/thrift-tools-0.0.3.tar.gz"
}
],
"0.0.4": [
{
"comment_text": "",
"digests": {
"md5": "7a5ddef160adfd63e19b714eee8e418c",
"sha256": "95a0a4cfa2b1de788328566ba4d4b9c5919ff0428af3d2e0b934513729087a0d"
},
"downloads": -1,
"filename": "thrift-tools-0.0.4.tar.gz",
"has_sig": false,
"md5_digest": "7a5ddef160adfd63e19b714eee8e418c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 26590,
"upload_time": "2019-02-01T20:33:17",
"url": "https://files.pythonhosted.org/packages/b1/42/a88d17717a2353fddf10ed6dcdfd3410d091c1357a67e7b79ad1945ff459/thrift-tools-0.0.4.tar.gz"
}
],
"0.0.5": [
{
"comment_text": "",
"digests": {
"md5": "ecb9416e97cf3b8aa6293bc574511ddf",
"sha256": "ed2cda080363fc82c6a9ebc3d3cc0c12de1bd36fdf45f98214a7c4f38b9ed56f"
},
"downloads": -1,
"filename": "thrift_tools-0.0.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ecb9416e97cf3b8aa6293bc574511ddf",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 33561,
"upload_time": "2019-08-25T23:43:49",
"url": "https://files.pythonhosted.org/packages/89/c8/7afd1fbe33f74ec8e325bc0273d1e7d942c1cbef527cc2065da62700c7da/thrift_tools-0.0.5-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "7b41d01212b838b7ea2b89225c2b4bb9",
"sha256": "e9426c614faae57d7fbfa89f01a7333be0374e474e7916f7f946313cb9b2fead"
},
"downloads": -1,
"filename": "thrift-tools-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "7b41d01212b838b7ea2b89225c2b4bb9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 29384,
"upload_time": "2019-08-25T23:43:51",
"url": "https://files.pythonhosted.org/packages/5d/64/b9671f3d7a37695ea6ac042701028645f6aedd6f6b486417ad9190d0f3da/thrift-tools-0.0.5.tar.gz"
}
],
"0.0.6": [
{
"comment_text": "",
"digests": {
"md5": "1d6a9993651d0ab0d8f605727aab865e",
"sha256": "24dcf14705254ba09ccfb7bf781af69c37407313ce15d829eab1f0dc863ca19b"
},
"downloads": -1,
"filename": "thrift_tools-0.0.6-py2-none-any.whl",
"has_sig": false,
"md5_digest": "1d6a9993651d0ab0d8f605727aab865e",
"packagetype": "bdist_wheel",
"python_version": "py2",
"requires_python": null,
"size": 37403,
"upload_time": "2019-08-28T21:45:53",
"url": "https://files.pythonhosted.org/packages/52/5a/25c0eeac32ffc5c3a4facf186301e5c4a82ae9b48334d0cf58b909aca954/thrift_tools-0.0.6-py2-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "a9412e6bc3eff429a1bc2071dae17fc8",
"sha256": "4ad18aa7a7747b748d191eddac6e40007d41deac4251a431d72a4d3dcce3b254"
},
"downloads": -1,
"filename": "thrift-tools-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "a9412e6bc3eff429a1bc2071dae17fc8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 32234,
"upload_time": "2019-08-28T21:45:56",
"url": "https://files.pythonhosted.org/packages/0a/4c/0965dd293d990b356d38dbc7e072d890828ab71735f5818bc63e0e3b2d56/thrift-tools-0.0.6.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "1d6a9993651d0ab0d8f605727aab865e",
"sha256": "24dcf14705254ba09ccfb7bf781af69c37407313ce15d829eab1f0dc863ca19b"
},
"downloads": -1,
"filename": "thrift_tools-0.0.6-py2-none-any.whl",
"has_sig": false,
"md5_digest": "1d6a9993651d0ab0d8f605727aab865e",
"packagetype": "bdist_wheel",
"python_version": "py2",
"requires_python": null,
"size": 37403,
"upload_time": "2019-08-28T21:45:53",
"url": "https://files.pythonhosted.org/packages/52/5a/25c0eeac32ffc5c3a4facf186301e5c4a82ae9b48334d0cf58b909aca954/thrift_tools-0.0.6-py2-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "a9412e6bc3eff429a1bc2071dae17fc8",
"sha256": "4ad18aa7a7747b748d191eddac6e40007d41deac4251a431d72a4d3dcce3b254"
},
"downloads": -1,
"filename": "thrift-tools-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "a9412e6bc3eff429a1bc2071dae17fc8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 32234,
"upload_time": "2019-08-28T21:45:56",
"url": "https://files.pythonhosted.org/packages/0a/4c/0965dd293d990b356d38dbc7e072d890828ab71735f5818bc63e0e3b2d56/thrift-tools-0.0.6.tar.gz"
}
]
}