{
    "info": {
        "author": "c0nch0b4r",
        "author_email": "lp1.on.fire@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Environment :: Console",
            "Intended Audience :: Information Technology",
            "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
            "Natural Language :: English",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7"
        ],
        "description": "======\nREADME\n======\n\nPichi is a small, cross-platform, fast pcap indexer using only standard libraries.\n\nWhere pichi shines is in pulling select traffic out of indexed pcaps. This is done by specifying one or more filters\n(see below) during extraction. On average, pichi is 10 times as fast as sancp, and 5 times as fast as using tshark.\n\nPichi can write index data in two forms:\n\n- text\n   slower to write and read and larger, but easily parsable with standard command line tools and human-readable\n- binary\n   faster and smaller, but unusable without Pichi's tools\n\nAdditionally, pichi has two output modes. These don't change the format of the files written, only the number:\n\n- individual\n   Write one index file per input pcap, and use the ``output_path`` name directly\n- combined\n   Write one index file for all input pcaps, with index names based on the input file name and ``output_path`` as the\n   output directory\n\nAdditionally, since version 0.3.5 pichi can optionally output a bloom filter generated from IP addresses after indexing.\nThis allows for fast determination of whether a pcap contains traffic from an IP without having to parse the entire\nindex.\n\n\n-----\nUsage\n-----\n\nPichi can be invoked from the command line, or it can be imported and used in python scripts as a library. The input is\none or more pcap files. If they are gzipped, Pichi will handle decompression transparently.\n\n\n~~~~~~~~~~~~\nAs a Library\n~~~~~~~~~~~~\n\n\nIndexing a pcap\n***************\n\n>>> from pichi import PichiBinaryIndexer\n>>> indexer = PichiBinaryIndexer(input_pcaps=['demo_traffic.pcap'], index_name='Demo Pichi Index')\n>>> indexer.index()\nMode is set to combined\nIndexing `demo_traffic.pcap` . . .\nIndexing completed: 1 file with 514 packets\n>>>\n\n\nReading Index Data\n******************\n\n>>> from pichi import PichiParser\n>>> parser = PichiParser(index_file='pichi.pi')\n>>> parser.parse_whole()\nParsing index `Demo Pichi Index`\nFound file `demo_traffic.pcap`\nFound 514 packets\n>>>\n\n``parser`` is also an iterable that yields a ``PichiBinaryFileIndex`` or ``PichiTextFileIndex`` object describing the\noriginally indexed file. This object is itself an iterable which yields a ``PichiTextRecordRow`` object for each record.\n\nWhen using ``parser`` as an iterable, the files and record rows are NOT saved in memory, so it is the preferred method\nwhen dealing with large indexes. However, if the PichiParser is provided the ``store=True`` argument, this is\noverwritten and already parsed files will be kept in ``parser.input_files``, and rows will be kept in\n``input_file.rows``. Once an index has been parsed in it's entirety, ``parser.completed_index`` is set to ``True``. When\ncalling PichiParser.parse_whole(), ``store`` is always set to ``True``.\n\n\nExtracting Traffic with an Index\n********************************\n\n>>> from pichi import PichiExtractor\n>>> extractor = PichiExtractor(input_index='pichi.pi', output_pcap='pichi_demo.pcap')\n>>> extractor.extract()\nExtracting packets from index `Demo Pichi Index`\nWriting to `pichi_demo.pcap`\nExtracting from file 1: `demo_traffic.pcap`\nWorking . . .\nExtracted 514 packets\n>>>\n\n\nChecking a Bloom Filter\n***********************\n\n>>> from pichi.bloom import BloomFilter\n>>> bloom_filter = BloomFilter.from_file(filename='testing/20190430_22:57:35.pcap.bf')\n>>> bloom_filter.bulk_check(items=['8.8.8.8', '1.1.1.1', '149.20.1.66'])\n{'8.8.8.8': True, '1.1.1.1': False, '149.20.1.66': False}\n>>>\n\n\n~~~~~~~~~~~~~~~~~~~~~\nFrom the Command Line\n~~~~~~~~~~~~~~~~~~~~~\n\n\nIndexing a pcap\n***************\n::\n\n  $ pichi index -i demo_traffic.pcap -o pib.pi -f bin -m combined\n  Format is set to binary\n  Mode is set to combined\n  Indexing `demo_traffic.pcap` . . .\n  Indexing completed: 1 file(s) with 514 packets\n  $\n\nNote: To generate a bloom filter along with the index, pass the ``-B`` argument to pichi.\n\n\nExtracting Traffic with an Index\n********************************\n::\n\n  $ pichi extract -i pib.pi -o pichi_demo.pcap\n  Extracting packets from index `Demo Pichi Index`\n  Writing to `pichi_demo.pcap`\n  Extracting from file 1: `demo_traffic.pcap`\n  Working . . .\n  Extracted 514 packets\n  $\n\n\nChecking a Bloom Filter\n***********************\n::\n\n  $ pichi bloom -b pib.bf -i '8.8.8.8 1.1.1.1 149.20.1.66'\n  8.8.8.8: True\n  1.1.1.1: False\n  149.20.1.66: False\n  $\n\n\n\n~~~~~~~~~~~~~\nUsing Filters\n~~~~~~~~~~~~~\n\nWhen extracting packets using an index, one or more filter statements (a filter set) can be provided to limit the\npackets written to those matching the statements. The 'language' is very basic, and a packet only gets written if it\npasses ALL filter statements. Statements take the form of::\n\n  {variable}{comparator}{value}\n\nVariables refer to fields in the index rows (outlined below).\nComparator must be one of:\n\n- ``==``\n   Equal. The values must match, or the value in the index must be a subset of the value given (i.e., when value is\n   10.0.0.0/8 and the index value is 10.0.0.41, this will match true)\n- ``!=``\n   Not equal. The opposite of the above.\n- ``>=``\n   Greater than or equal to. mostly useful for ports, but can be applied to any numeric variable.\n- ``<=``\n   Less than or equal to, opposite of the above.\n\nValid variables and values are:\n\n- ``host``\n   An IPv4 or IPv6 host or CIDR format network, or a domain name. If EITHER the source or destination host matches this\n   value, the statement is true. For a CIDR format network, the statement is true if EITHER source or destination host\n   falls within the given network.\n- ``src_host``\n   The same as ``host``, but only looking at the source host.\n- ``dst_host``\n   The same as ``host``, but only looking at the destination host.\n- ``port``\n   A port number or service name (from /etc/services or your OS's equivalent). Note that for EtherTypes where there is\n   no concept of a port (ICMP, ARP, etc.), this field is set to 0. If EITHER the source or destination port matches this\n   value, the statement is true.\n- ``src_port``\n   The same as ``port``, but only looking at the source port.\n- ``dst_port``\n   The same as ``port``, but only looking at the destination port.\n- ``eth_type``\n   The EtherType of the packet, must be a number. See\n   https://www.iana.org/assignments/ieee-802-numbers/ieee-802-numbers.xhtml for reference.\n- ``l2_proto``\n   The L2 protocol number or name of the packet. See\n   https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml for reference.\n\n\n-----------\nFile Format\n-----------\n\nAs stated above, Pichi can write in two formats: text and binary. Both formats can also be compressed using gzip\non-the-fly by passing the indexer the ``output_compressed=True`` option.\n\n\n~~~~~~~~~~~\nText Format\n~~~~~~~~~~~\n\nThe text format is very simple and easy to send to tools like ``awk``, ``sed``, ``cut``, etc.\n\nThere is no header or footer, and every packet record is contained on its own line with fields pipe-delimited::\n\n   {epoch}.{ms}|{in_filename}|{start}|{end}|{eth_proto}|{ip_proto}|{src_host}|{dst_host}|{src_port}|{dst_port}\\n\n\n- ``{in_filename}`` is the name of the input pcap\n- ``{start}`` is the first byte of the packet\n- ``{end}`` is the last byte of the packet\n\n\n~~~~~~~~~~~~~\nBinary Format\n~~~~~~~~~~~~~\n\nThe binary file format is also relatively simple. It was created to make writing as fast as possible, and parsing easy.\n\nRemember that indexes may or may not be compressed with Gzip.\n\nFor an in-depth look at the binary format, please see `the format spec`_. One small, handy trick to note, though:\n\n>>> import struct\n>>> with open('pichi.pi', 'rb') as fp:\n...     fp.seek(-5, 2)\n...     file_count = struct.unpack('B', fp.read(1))[0]\n...     packet_count = struct.unpack('I', fp.read(4))[0]\n\n\n-----\nTo-Do\n-----\n\n- Potentially store field values that have already passed a filter with text records and check against them first, to\n  speed up filter testing?\n- Have ``PichiIndexerBase`` objects optionally yield a ``PichiParser`` object when ``.index()`` is completed\n- PCAPNG Support (eek)\n- multithreaded indexing and extraction\n- Utilize mmap for index writing?\n- Allow specifying alternate pcap for extraction\n\n\n----------------\nAcknowledgements\n----------------\n\nThe original idea comes from SANCP, which is a fantastic project that died too early:\nhttp://sancp.sourceforge.net/\n\n\n.. _the format spec: docs/FILE_FORMAT.rst\n\n",
        "description_content_type": "text/x-rst",
        "docs_url": null,
        "download_url": "https://bitbucket.org/c0nch0b4r/pichi/get/0.3.6.tar.gz",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://bitbucket.org/c0nch0b4r/pichi",
        "keywords": "pcap indexer",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "pichi",
        "package_url": "https://pypi.org/project/pichi/",
        "platform": "",
        "project_url": "https://pypi.org/project/pichi/",
        "project_urls": {
            "Download": "https://bitbucket.org/c0nch0b4r/pichi/get/0.3.6.tar.gz",
            "Homepage": "https://bitbucket.org/c0nch0b4r/pichi",
            "Source": "https://bitbucket.org/c0nch0b4r/pichi/src"
        },
        "release_url": "https://pypi.org/project/pichi/0.3.6/",
        "requires_dist": [
            "typing"
        ],
        "requires_python": ">=3.6, <4",
        "summary": "A utility for generating simple pcap indexes",
        "version": "0.3.6"
    },
    "last_serial": 5919163,
    "releases": {
        "0.2.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7a84fec39c8428f1f3c3e05e2a9ad611",
                    "sha256": "98ab34f157696583d722a5659078153982f8d9ac63a32130e042a102ad00df3b"
                },
                "downloads": -1,
                "filename": "pichi-0.2.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7a84fec39c8428f1f3c3e05e2a9ad611",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6, <4",
                "size": 27453,
                "upload_time": "2019-02-20T06:37:28",
                "url": "https://files.pythonhosted.org/packages/cd/f6/e1ae8345cd2b80ef49cecadb0c37aaccb695fe07e28b8d82bbdee62c28ac/pichi-0.2.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "747c194f693d407519d8462230b67fa3",
                    "sha256": "8270897876b53e864dd162805016f05b6a055aa75903070633470cd408842163"
                },
                "downloads": -1,
                "filename": "pichi-0.2.4.tar.gz",
                "has_sig": false,
                "md5_digest": "747c194f693d407519d8462230b67fa3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6, <4",
                "size": 32133,
                "upload_time": "2019-02-20T06:37:30",
                "url": "https://files.pythonhosted.org/packages/8a/7f/30ed44951afac57de123b349992f6572ee9128857fd2d6de535e66456b08/pichi-0.2.4.tar.gz"
            }
        ],
        "0.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "8e4d93cc844ae8134858a1e9ac0ace37",
                    "sha256": "460aa35b6f89f18037da740859443e0a8922fedcada1831b9c10f98c198cc59b"
                },
                "downloads": -1,
                "filename": "pichi-0.3.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "8e4d93cc844ae8134858a1e9ac0ace37",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6, <4",
                "size": 28018,
                "upload_time": "2019-09-24T04:08:43",
                "url": "https://files.pythonhosted.org/packages/c9/43/22e96a18af59393d6426aeccd1ebcad71f06928b9f49777ed1b91de5e074/pichi-0.3.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "36e43d63fd3982d6f4ac38baed7b9fd4",
                    "sha256": "fc972ac823252f6ea91a50176275d4d39f5c2c5a40282f2da516d89a25f75bc4"
                },
                "downloads": -1,
                "filename": "pichi-0.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "36e43d63fd3982d6f4ac38baed7b9fd4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6, <4",
                "size": 32705,
                "upload_time": "2019-09-24T04:08:46",
                "url": "https://files.pythonhosted.org/packages/df/21/b0aa8b4f85eeae6703df24c9e820cbf34e653178f83ffcc80496a52a5ee5/pichi-0.3.1.tar.gz"
            }
        ],
        "0.3.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "b3a8610dc077fb244a2e3a394a8c06bb",
                    "sha256": "2bf7e922e6601eeb7edffde3668a591f6d006ecd684b700c5cf27f2616b651d8"
                },
                "downloads": -1,
                "filename": "pichi-0.3.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "b3a8610dc077fb244a2e3a394a8c06bb",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6, <4",
                "size": 28096,
                "upload_time": "2019-09-24T06:36:27",
                "url": "https://files.pythonhosted.org/packages/cd/bf/5fccbab6d624e2221885002bd35767feabe97b5c9e1d2f43ccad26f8d908/pichi-0.3.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "d3d1eb628c319a83d3aeb03e46ffc6a6",
                    "sha256": "5b3ac3191f151046a3a8e5adaa5798a1e101faa98e428664f0587c61a22718dc"
                },
                "downloads": -1,
                "filename": "pichi-0.3.3.tar.gz",
                "has_sig": false,
                "md5_digest": "d3d1eb628c319a83d3aeb03e46ffc6a6",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6, <4",
                "size": 32783,
                "upload_time": "2019-09-24T06:36:29",
                "url": "https://files.pythonhosted.org/packages/1e/e2/0301b477dc2e577f0e58200cd48d478aa6998ed46d3a6a3977cb4ee47b9e/pichi-0.3.3.tar.gz"
            }
        ],
        "0.3.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6d09d1c033ed45a21496a2deca8aa5f0",
                    "sha256": "eff4c91f8adc20f2b5c0c66e6309c89b2a7c0555f458275e7913bed0d2f361db"
                },
                "downloads": -1,
                "filename": "pichi-0.3.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6d09d1c033ed45a21496a2deca8aa5f0",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6, <4",
                "size": 28437,
                "upload_time": "2019-09-25T04:18:14",
                "url": "https://files.pythonhosted.org/packages/46/5c/062566d4886a18828e1738220d1d163a8fc7acc448f7b9ce6207c95783c9/pichi-0.3.4-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "90c892e299905c9fb1e5d6aace045282",
                    "sha256": "549e4e2f884307a2faf8fffc4d3205e02b935c6aa26e658d47500a769015183a"
                },
                "downloads": -1,
                "filename": "pichi-0.3.4.tar.gz",
                "has_sig": false,
                "md5_digest": "90c892e299905c9fb1e5d6aace045282",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6, <4",
                "size": 33328,
                "upload_time": "2019-09-25T04:18:16",
                "url": "https://files.pythonhosted.org/packages/5e/70/e6a0da503eb16aea3a9558c2c66f2bfdd32fdff10b9e9fdde3bcb89b60a1/pichi-0.3.4.tar.gz"
            }
        ],
        "0.3.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5fe17f898038b5391e2860e5c1e04fc6",
                    "sha256": "7ac88c11873e1a51dcb3d6163d00c8e29203ade2495e09871a0f2a79534c4d05"
                },
                "downloads": -1,
                "filename": "pichi-0.3.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "5fe17f898038b5391e2860e5c1e04fc6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6, <4",
                "size": 30741,
                "upload_time": "2019-09-28T22:30:36",
                "url": "https://files.pythonhosted.org/packages/81/45/ce1cdc6f9703eb16be3f602495f36881a412feeac766c61095b85225da9e/pichi-0.3.5-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8cf3e9ab287c54f293de11b13e107154",
                    "sha256": "eff26f188dfb9f18d0080c11d24a1c1713c58fd11f3901796f4094c2fc1d1d50"
                },
                "downloads": -1,
                "filename": "pichi-0.3.5.tar.gz",
                "has_sig": false,
                "md5_digest": "8cf3e9ab287c54f293de11b13e107154",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6, <4",
                "size": 35508,
                "upload_time": "2019-09-28T22:30:37",
                "url": "https://files.pythonhosted.org/packages/5c/70/f847a4614f19bcdc91a2c5befb989007dd73726d52381bd61b93052db3e5/pichi-0.3.5.tar.gz"
            }
        ],
        "0.3.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "cb8444b189c48d479f057912b93fb73f",
                    "sha256": "92201515da07d9ab30164d6c3b036af2567e456d00753cf87e655456e910f75b"
                },
                "downloads": -1,
                "filename": "pichi-0.3.6-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "cb8444b189c48d479f057912b93fb73f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": ">=3.6, <4",
                "size": 30874,
                "upload_time": "2019-10-02T17:22:35",
                "url": "https://files.pythonhosted.org/packages/c9/39/4c7a57a14e3cd47773232fdc14702d4da4d1c692ea5e87eb9930b56363a6/pichi-0.3.6-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "5b2953557a2b439ac681e96fd0369d51",
                    "sha256": "def2be79a9991da06ad35f8b6e793d240897bd5ed271b8e8145e1144af028e1f"
                },
                "downloads": -1,
                "filename": "pichi-0.3.6.tar.gz",
                "has_sig": false,
                "md5_digest": "5b2953557a2b439ac681e96fd0369d51",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": ">=3.6, <4",
                "size": 35672,
                "upload_time": "2019-10-02T17:22:37",
                "url": "https://files.pythonhosted.org/packages/83/83/3aa147bbbf156222039268f4b13113cc026e50c48c19f4cfc8247854f975/pichi-0.3.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "cb8444b189c48d479f057912b93fb73f",
                "sha256": "92201515da07d9ab30164d6c3b036af2567e456d00753cf87e655456e910f75b"
            },
            "downloads": -1,
            "filename": "pichi-0.3.6-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "cb8444b189c48d479f057912b93fb73f",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6, <4",
            "size": 30874,
            "upload_time": "2019-10-02T17:22:35",
            "url": "https://files.pythonhosted.org/packages/c9/39/4c7a57a14e3cd47773232fdc14702d4da4d1c692ea5e87eb9930b56363a6/pichi-0.3.6-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "5b2953557a2b439ac681e96fd0369d51",
                "sha256": "def2be79a9991da06ad35f8b6e793d240897bd5ed271b8e8145e1144af028e1f"
            },
            "downloads": -1,
            "filename": "pichi-0.3.6.tar.gz",
            "has_sig": false,
            "md5_digest": "5b2953557a2b439ac681e96fd0369d51",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6, <4",
            "size": 35672,
            "upload_time": "2019-10-02T17:22:37",
            "url": "https://files.pythonhosted.org/packages/83/83/3aa147bbbf156222039268f4b13113cc026e50c48c19f4cfc8247854f975/pichi-0.3.6.tar.gz"
        }
    ]
}