{
    "info": {
        "author": "CSIRT Foundry / Chris Horsley",
        "author_email": "chris.horsley@csirtfoundry.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "=======\nnetgrep\n=======\n\nNetgrep is a command line tool which tells you which lines in a text file\ncontain network resources related to a particular country or Autonomous\nNetwork (AS).\n\nGiven input, it will:\n\n- locate domain names and IP addresses\n- resolve domains to IP addresses\n- geo-locate IP addresses to country codes and ASNs\n- extract country codes from domain names \n- output each line matching at least one country code or ASN specified.\n\nUse case\n--------\n\nYou have a big list of infected botnet clients, URLs, or IP addresses, and you\nwant to find out which ones are related to your country code or AS.\n\nExample usage\n-------------\n\n# a simple log file\n\n$ cat mylog.txt\nabc.net.au,Australian Broadcasting Corporation\nbbc.co.uk,British Broadcasting Corporation\n203.2.218.214,Australian Broadcasting Corporation IP address\nxs4all.com,Hosted in Netherlands with .com extension\n\n# match anything hosted in .nl\n\n$ netgrep NL mylog.txt\nxs4all.com,Hosted in Netherlands with .com extension\n\n# match Australian IPs and domain names\n\n$ netgrep AU mylog.txt\nabc.net.au,Australian Broadcasting Corporation\n203.2.218.214,Australian Broadcasting Corporation IP address\n\n# match IPs resolving to Autonomous System 2818, owned by BBC\n\n$ netgrep AS2818 mylog.txt\nbbc.co.uk,British Broadcasting Corporation\n\n# combination filter with both Australian IPs / domains and AS2818\n\n$ netgrep AU,AS2818 mylog.txt\nabc.net.au,Australian Broadcasting Corporation\nbbc.co.uk,British Broadcasting Corporation\n203.2.218.214,Australian Broadcasting Corporation IP address\n\nAdvanced usage\n--------------\n\n* Multiple files\n\nYou can use wildcards or pass in multiple files:\n\n$ netgrep AS444 logs/firstlog.txt logs/secondlog.txt\n...\n$ netgrep AS444 logs/*.txt\n...\n\n(Note that netgrep can't handle recursive subdirectories as yet.)\n\n* Piping standard input\n\nNetgrep supports piping from standard input like this:\n\n$ cat input1.txt | netgrep BR\n\nYou can use netgrep as a quick little assertion tool. For example, does\nakamai.com resolve to any boxes in Singapore?\n\n$ echo \"akamai.com\" | netgrep SG\nakamai.com  \n$\n# got output - assertion proven\n\n$ echo \"akamai.com\" | netgrep FI\n$ \n# no output - assertion failed\n\nInstallation\n============\n\nQuick install:\n--------------\n\nMethod 1\n\n1. Make sure you have adns installed. \n2. $ sudo easy_install netgrep\n\nMethod 2\n\n1. Make sure you have adns installed\n2. Extract the tarball from \nhttps://github.com/csirtfoundry/netgrep/tarball/master\n3. cd [directory it was unpacked to]\n4. $ sudo python setup.py install\n\nDetailed install:\n-----------------\n\nIf the above didn't work out of the box, here's what you can do.\n\nYou'll need:\n\nadns (http://www.chiark.greenend.org.uk/~ian/adns/)\nPython libraries:\n  BulkWhois\n  publicsuffix\n  adns-python\n\nHere's some OS-specific ways to install the prerequisites and netgrep,\nassuming a clean system:\n\n* 1: Linux install with apt-get:\n\n$ sudo apt-get install python-setuptools gcc python-dev python-adns\n$ sudo easy_install netgrep\n\n* 2: OS X install:\n\nUsing homebrew (or your choice of package manager):\n\nInstall Xcode (needed for building lots of Homebrew packages)\n$ brew install adns\n$ sudo easy_install pip\n$ sudo pip install netgrep\n\n* 3: Download and extract the tarball and then (with apt-get):\n\n$ sudo apt-get install gcc python-dev python-setuptools python-adns\n$ sudo python setup.py install\n\n* Windows\n\nUntested, and suspect it may not work. If you like to report how it did or\ndidn't work, please let me know.\n\nInstallation issues:\n--------------------\n\nWhen installing adns-python, you may receive:\n\nadnsmodule.c:8:20: fatal error: Python.h: No such file or directory\n\nsudo apt-get install python-dev\n\n\nImplementation notes\n====================\n\n1. Netgrep makes one pass of the logs, extracting any candidate domain name and \nIP addresses it finds.\n\n2. Domain names are checked to see if they resolve to a TLD present in the\nMozilla Public Suffix List. Anything not matching is ignored.\n\n3. IP addresses are checked to ensure they're valid IPv4. IPv6 is currently\nnot supported, but there are plans to do add this later.\n\n4. Domains are resolved to IP addresses asynchronously. This should be quite\nfast for hundreds of IP addresses, but may take a little time if you have\nthousands.\n\n5. All IPs gathered both directly from the log and via DNS resolution are\nsubmitted via bulk query to Team Cymru's bulk whois service, retrieving\ncountry code and ASN.\n\n6. The file is scanned, the country code and ASN filters applied, and matching\nlines are output.\n\nOther notes\n-----------\n\n* This initial release is focusing on functionality rather than performance for\nmatching. Because of the network lookups it uses, it's not going to be anywhere\nnear as fast as mighty grep: \nhttp://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html?\n* Pains have been taken to keep the memory footprint low and network calls\nasynchronous / bulk where possible.\n* netgrep only handles one record per line for now, so that means no multiline\nXML parsing. If this might be useful, let me know \n[chris.horsley at csirtfoundry dot com].\n\n\nAcknowledgements\n================\n\nRob Lowe for the concept\n\nPeteris Krumins (http://www.catonmat.net/blog/asynchronous-dns-resolution/)\nfor his no-fuss interface to adns. Made one very slight tweak to return \nmultiple A records rather than just the first.",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "https://github.com/csirtfoundry/netgrep/tarball/master",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://pypi.python.org/pypi/netgrep/",
        "keywords": null,
        "license": "LICENSE.txt",
        "maintainer": null,
        "maintainer_email": null,
        "name": "netgrep",
        "package_url": "https://pypi.org/project/netgrep/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/netgrep/",
        "project_urls": {
            "Download": "https://github.com/csirtfoundry/netgrep/tarball/master",
            "Homepage": "http://pypi.python.org/pypi/netgrep/"
        },
        "release_url": "https://pypi.org/project/netgrep/0.3.7/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Grep-like filter for files based on country codes and ASN",
        "version": "0.3.7"
    },
    "last_serial": 795319,
    "releases": {
        "0.3.0": [],
        "0.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ae1416f14a99e8f612881efea811990f",
                    "sha256": "4aee789c60abc9aff4227c25389253fde7bfc2c45087a71eecadfc54d1649167"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.1.tar.gz",
                "has_sig": false,
                "md5_digest": "ae1416f14a99e8f612881efea811990f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9481,
                "upload_time": "2011-08-12T02:00:54",
                "url": "https://files.pythonhosted.org/packages/17/d2/c71cb93400e6f259a13601070532a2f2c6de049082924c6fdbf5c756c136/netgrep-0.3.1.tar.gz"
            }
        ],
        "0.3.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "887cbc388851a7ad6ebb8166beeafca2",
                    "sha256": "1057ba6dddcaa9903fe75527252259e14ea287081a89db7a44021bf650748b77"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.2.tar.gz",
                "has_sig": false,
                "md5_digest": "887cbc388851a7ad6ebb8166beeafca2",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9506,
                "upload_time": "2011-08-12T02:13:22",
                "url": "https://files.pythonhosted.org/packages/ec/8b/f1dab5c4106c5b880d73e2cee0af0339b869c57e825b8eadd919b0e25965/netgrep-0.3.2.tar.gz"
            }
        ],
        "0.3.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2e65a1b2285ae67846f5bb60bcf9a5af",
                    "sha256": "7d3780bbb21446034eb2597ea9569d797347ecdb0409a9176e12bf266e87dd18"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.3.tar.gz",
                "has_sig": false,
                "md5_digest": "2e65a1b2285ae67846f5bb60bcf9a5af",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9510,
                "upload_time": "2011-08-12T02:16:42",
                "url": "https://files.pythonhosted.org/packages/77/5b/984d1c186fc711e9cc46347569b1a3bdcb144643c97cd06533310bd2f2aa/netgrep-0.3.3.tar.gz"
            }
        ],
        "0.3.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "02860f8d3570645db82899cf3072773f",
                    "sha256": "200df15bd5b5d4301dc3bd271fa4af01b396acfaaf55620e0c956997f89c9e71"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.4.tar.gz",
                "has_sig": false,
                "md5_digest": "02860f8d3570645db82899cf3072773f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9663,
                "upload_time": "2011-08-13T10:03:11",
                "url": "https://files.pythonhosted.org/packages/20/e2/1d44ac2e65b9756bc58540396b9e5da277ab8bbe847a4f24c2e4cdc22b6e/netgrep-0.3.4.tar.gz"
            }
        ],
        "0.3.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1477b862ad44b87235d3f7db261acb0c",
                    "sha256": "77599b788564086badaba95a0324db08080d07d6caaeff03f8ae74cf3a8b7392"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.5.tar.gz",
                "has_sig": false,
                "md5_digest": "1477b862ad44b87235d3f7db261acb0c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10036,
                "upload_time": "2011-08-18T01:54:34",
                "url": "https://files.pythonhosted.org/packages/c2/b4/ade93c7cf56b3dcf3ac69bd2c2ad2aca1b68cfb6a05466d462746e17c366/netgrep-0.3.5.tar.gz"
            }
        ],
        "0.3.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4387babb4d1846c49078b3cd0da90c5e",
                    "sha256": "2de1d2c5c3c47f169bcba2128922d19eb31362769af9e5332f8541c4afc71a37"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.6.tar.gz",
                "has_sig": false,
                "md5_digest": "4387babb4d1846c49078b3cd0da90c5e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10347,
                "upload_time": "2011-08-22T09:17:30",
                "url": "https://files.pythonhosted.org/packages/4c/3e/2b8df3f4a4d067721b76b15c6bc2a691e7484c0e875c2f18901f37b70535/netgrep-0.3.6.tar.gz"
            }
        ],
        "0.3.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c72daedd8b42e3a624dd1148452268d5",
                    "sha256": "458016675bb4e8ed5d3b3a3d9c586937238cbb9130ce85a9ebadebf15cf03eda"
                },
                "downloads": -1,
                "filename": "netgrep-0.3.7.tar.gz",
                "has_sig": false,
                "md5_digest": "c72daedd8b42e3a624dd1148452268d5",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10386,
                "upload_time": "2011-08-25T13:58:40",
                "url": "https://files.pythonhosted.org/packages/13/24/6ebc9fb1745187aa41aeec3a310a914aa9cdd3b9249db2e16cbf8e0b7ca6/netgrep-0.3.7.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "c72daedd8b42e3a624dd1148452268d5",
                "sha256": "458016675bb4e8ed5d3b3a3d9c586937238cbb9130ce85a9ebadebf15cf03eda"
            },
            "downloads": -1,
            "filename": "netgrep-0.3.7.tar.gz",
            "has_sig": false,
            "md5_digest": "c72daedd8b42e3a624dd1148452268d5",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10386,
            "upload_time": "2011-08-25T13:58:40",
            "url": "https://files.pythonhosted.org/packages/13/24/6ebc9fb1745187aa41aeec3a310a914aa9cdd3b9249db2e16cbf8e0b7ca6/netgrep-0.3.7.tar.gz"
        }
    ]
}