{
    "info": {
        "author": "Rui Zhou",
        "author_email": "quicksort@outlook.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3"
        ],
        "description": "Log2csv\n=======\nA fast tool to parse unstructured log files into structured csv, using a grok-like pattern.\n\n```\nlog2csv [-p custom_pattern.grok|custom_pattern_dir]  [-o output.csv] -e '%{NUMBER:size} (?P<custom_name>regexpression) (?:content to ignore but match) %{IP: client} %{UserAgent: agent} %{URL: request_url}' nginx.log\n```\n\n## Expression\n\n```\n%{PATTERN_NAME1: csv_field_name}\n%{PATTERN_NAME2}\n```\n\n## Grok File\n\n```\nPATTERN_NAME1 regexpression\nPATTERN_NAME2 %{SUB_PATTERN: field_name}\n# Comment\n```\n\n## Example\n\nsample.log\n```\n77.179.66.156 - - [25/Oct/2016:14:49:33 +0200] \"GET / HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36\"\n77.179.66.156 - - [25/Oct/2016:14:49:34 +0200] \"GET /favicon.ico HTTP/1.1\" 404 571 \"http://localhost:8080/\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36\"\n77.179.66.156 - - [25/Oct/2016:14:50:44 +0200] \"GET /adsasd HTTP/1.1\" 404 571 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36\"\n77.179.66.156 - - [07/Dec/2016:10:34:43 +0100] \"GET / HTTP/1.1\" 200 612 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156 - - [07/Dec/2016:10:34:43 +0100] \"GET /favicon.ico HTTP/1.1\" 404 571 \"http://localhost:8080/\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156 - - [07/Dec/2016:10:43:18 +0100] \"GET /test HTTP/1.1\" 404 571 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156 - - [07/Dec/2016:10:43:21 +0100] \"GET /test HTTP/1.1\" 404 571 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156 - - [07/Dec/2016:10:43:23 +0100] \"GET /test1 HTTP/1.1\" 404 571 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n127.0.0.1 - - [07/Dec/2016:11:04:37 +0100] \"GET /test1 HTTP/1.1\" 404 571 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n127.0.0.1 - - [07/Dec/2016:11:04:58 +0100] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0\"\n127.0.0.1 - - [07/Dec/2016:11:04:59 +0100] \"GET / HTTP/1.1\" 304 0 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0\"\n127.0.0.1 - - [07/Dec/2016:11:05:07 +0100] \"GET /taga HTTP/1.1\" 404 169 \"-\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0\"\n```\n\nlog2csv command\n```\nlog2csv -e '%{IP:ip} - - \\[%{HTTPDATE:date}\\] \"%{WORD:http_method} %{URIPATH:path} HTTP/1.1\" %{NUMBER:http_status} %{NUMBER:payload_bytes} \"-\" \"%{DATA:user_agent}\"' sample.log\n```\n\noutput csv format\n```\nip,date,http_method,path,http_status,payload_bytes,user_agent\n77.179.66.156,25/Oct/2016:14:49:33 +0200,GET,/,200,612,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36\"\n77.179.66.156,25/Oct/2016:14:50:44 +0200,GET,/adsasd,404,571,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36\"\n77.179.66.156,07/Dec/2016:10:34:43 +0100,GET,/,200,612,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156,07/Dec/2016:10:43:18 +0100,GET,/test,404,571,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156,07/Dec/2016:10:43:21 +0100,GET,/test,404,571,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n77.179.66.156,07/Dec/2016:10:43:23 +0100,GET,/test1,404,571,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n127.0.0.1,07/Dec/2016:11:04:37 +0100,GET,/test1,404,571,\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36\"\n127.0.0.1,07/Dec/2016:11:04:58 +0100,GET,/,304,0,Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0\n127.0.0.1,07/Dec/2016:11:04:59 +0100,GET,/,304,0,Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0\n127.0.0.1,07/Dec/2016:11:05:07 +0100,GET,/taga,404,169,Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:49.0) Gecko/20100101 Firefox/49.0\n```\n\n## Benchmark\nTested on Macbook Pro 2 cores 4 threads 8G memroy: 1.8G logs file costs 176s \n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/quick-sort/log2csv",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "log2csv",
        "package_url": "https://pypi.org/project/log2csv/",
        "platform": "",
        "project_url": "https://pypi.org/project/log2csv/",
        "project_urls": {
            "Homepage": "https://github.com/quick-sort/log2csv"
        },
        "release_url": "https://pypi.org/project/log2csv/1.0.0/",
        "requires_dist": [
            "regex"
        ],
        "requires_python": "",
        "summary": "A tool to parse log files into csv, using a grok-like pattern",
        "version": "1.0.0"
    },
    "last_serial": 4151861,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6692639c2b64373b7c8a5aa09f9a663f",
                    "sha256": "fd67b13681c36cd81bf6ed1bdac6072abf66c9654b1b166cdc4ae62594fdb087"
                },
                "downloads": -1,
                "filename": "log2csv-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6692639c2b64373b7c8a5aa09f9a663f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 23746,
                "upload_time": "2018-08-08T02:47:35",
                "url": "https://files.pythonhosted.org/packages/13/31/bcf293ffcf6138554d57af11678ef9e8e532fbcea842f2d8cee8ce9c8c7a/log2csv-0.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7eafe4be2531eafa6fe49820568d051b",
                    "sha256": "355f2b98f2a0098597ec20d9ebc29042d30a56caedbbc339d5faaabcb6ed2044"
                },
                "downloads": -1,
                "filename": "log2csv-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "7eafe4be2531eafa6fe49820568d051b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17048,
                "upload_time": "2018-08-08T02:47:38",
                "url": "https://files.pythonhosted.org/packages/c0/ff/aff601b0a9c36b9b5627a9f8ad31f9284b7a8fa8c2d73b90265ba0f10c36/log2csv-0.0.1.tar.gz"
            }
        ],
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d8c12904494a78f90961361eac8836bb",
                    "sha256": "8782ec07008b12f3844d6bfee98f97ec20c8e995f86dd7f67db451ffd25f3bbe"
                },
                "downloads": -1,
                "filename": "log2csv-1.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d8c12904494a78f90961361eac8836bb",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 24522,
                "upload_time": "2018-08-09T06:34:07",
                "url": "https://files.pythonhosted.org/packages/d2/9e/5d8ed4715b51050d9573a318b12a2a1eb74c13405938b666583ad541bb08/log2csv-1.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "1d9408695b27402b333b2967cf0cd942",
                    "sha256": "015f26e014116071e8b8df45497ddf167c7527aac8c4bb5d09326586cbe4bdec"
                },
                "downloads": -1,
                "filename": "log2csv-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "1d9408695b27402b333b2967cf0cd942",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18054,
                "upload_time": "2018-08-09T06:34:08",
                "url": "https://files.pythonhosted.org/packages/1b/9d/08990714d15a9d2e66d59be08d2695cea88f180df8515e0d1b690cd1cc80/log2csv-1.0.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d8c12904494a78f90961361eac8836bb",
                "sha256": "8782ec07008b12f3844d6bfee98f97ec20c8e995f86dd7f67db451ffd25f3bbe"
            },
            "downloads": -1,
            "filename": "log2csv-1.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d8c12904494a78f90961361eac8836bb",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 24522,
            "upload_time": "2018-08-09T06:34:07",
            "url": "https://files.pythonhosted.org/packages/d2/9e/5d8ed4715b51050d9573a318b12a2a1eb74c13405938b666583ad541bb08/log2csv-1.0.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "1d9408695b27402b333b2967cf0cd942",
                "sha256": "015f26e014116071e8b8df45497ddf167c7527aac8c4bb5d09326586cbe4bdec"
            },
            "downloads": -1,
            "filename": "log2csv-1.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "1d9408695b27402b333b2967cf0cd942",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 18054,
            "upload_time": "2018-08-09T06:34:08",
            "url": "https://files.pythonhosted.org/packages/1b/9d/08990714d15a9d2e66d59be08d2695cea88f180df8515e0d1b690cd1cc80/log2csv-1.0.0.tar.gz"
        }
    ]
}