{ "info": { "author": "glomex SRE Team", "author_email": "mark.fink@glomex.com", "bugtrack_url": null, "classifiers": [ "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.6" ], "description": "KORG\n====\n\n|Build Status| |License|\n\nWhy a logstash / grok port to Python?\n-------------------------------------\n\nI like the logstash ``grok`` approach to logfile parsing. So I want to\nuse this in Python.\n\nOne solution would be to use the C version of logstash / grok\n(https://github.com/jordansissel/grok) and to write a wrapper:\n\n- https://github.com/kiwi0530/python-grok\n- https://github.com/emgee/libgrok-py\n\nBasically grok assembles regular expressions. I already know that in\nPython file processing with regular expressions is blazingly fast so I\nchoose to port it to Python.\n\nUnfortunately a ``grok`` package already existed in Python for something\ncompletely different - consequently I had to \"reverse-engineer\" it. Thus\nthe name ``korg``.\n\nThe pattern files are updated from the logstash grok project:\nhttps://github.com/logstash-plugins/logstash-patterns-core\n\nA big thank you belongs to the logstash community for an awesome job\nmaintaining the regex pattern files!\n\nExamples using korg\n-------------------\n\n- extracting metrics from logfiles:\n https://github.com/finklabs/loganalyser\n\nStatus\n------\n\n- Base functionality is implemented including tests\n- Logstash patterns are included\n- Some grok features are still missing (not sure which ones are\n necessary)\n\nI made some first benchmarks to verify whether my performance\nrequirements can be realized with this approach. Please do not use this\nresults in any blog posts or articles since this is not a complete\nbenchmark (from a statistical view point the sample size is way too\nsmall).\n\n**Processing a 1.7MB apache access log with korg**\n\n::\n\n ..\n 200\n 200\n 404\n 200\n 200\n\n real 0m0.248s\n user 0m0.172s\n sys 0m0.040s\n\n**Processing the same logfile with logstash**\n\n::\n\n ..\n 200\n 200\n 404\n 200\n 200\n ^CSIGINT received, shutting down. {:level=>:warn}\n\n real 0m11.752s\n user 0m23.948s\n sys 0m0.528s\n\nNote: both implementations read in all available patterns from the\n'patterns' folder.\n\nResources\n---------\n\n- http://jpmens.net/2012/08/06/my-logstash-and-graylog2-notes/\n- http://grokdebug.herokuapp.com/\n\n.. |Build Status| image:: https://travis-ci.org/finklabs/korg.svg?branch=master\n :target: https://travis-ci.org/finklabs/korg\n.. |License| image:: http://img.shields.io/badge/license-MIT-yellowgreen.svg\n :target: MIT_LICENSE\n", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/finklabs/korg", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "korg", "package_url": "https://pypi.org/project/korg/", "platform": "", "project_url": "https://pypi.org/project/korg/", "project_urls": { "Homepage": "https://github.com/finklabs/korg" }, "release_url": "https://pypi.org/project/korg/0.0.11/", "requires_dist": null, "requires_python": "", "summary": "Fast logfile parsing. This is a port of Ruby logstash / grok to Python", "version": "0.0.11" }, "last_serial": 2995988, "releases": { "0.0.10": [ { "comment_text": "", "digests": { "md5": "f7482e49c4b231a759a886457383b847", "sha256": "36c4a2904a9a56ec4d5daa38ed043c2e1ab97fb7a557428791e613903166838c" }, "downloads": -1, "filename": "korg-0.0.10.tar.gz", "has_sig": false, "md5_digest": "f7482e49c4b231a759a886457383b847", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19394, "upload_time": "2017-06-28T15:21:45", "url": "https://files.pythonhosted.org/packages/16/a3/3b00e53bef45a85dcc3eaa2a9c238ac5c1a10b6744f855ee7e89587df26c/korg-0.0.10.tar.gz" } ], "0.0.11": [ { "comment_text": "", "digests": { "md5": "ada91ff65cd76879ebe5c88debe12d9b", "sha256": "7a3ae01c1ab0ef667c2fcdc5d01b12b4ead0e52563f9bb3d71701fc41f643a50" }, "downloads": -1, "filename": "korg-0.0.11.tar.gz", "has_sig": false, "md5_digest": "ada91ff65cd76879ebe5c88debe12d9b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19223, "upload_time": "2017-07-03T07:17:48", "url": "https://files.pythonhosted.org/packages/1f/79/58d8ddcca6441f96c288c51a5dbd43264ecd426388c96b28e6c3683d4a9c/korg-0.0.11.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "b982a1a2e23a6055fd433999bd4bda24", "sha256": "651f339d3b25cf40db54e49c72a646655cccd78648eb354aee7ad14cae4f5cc6" }, "downloads": -1, "filename": "korg-0.0.2.tar.gz", "has_sig": false, "md5_digest": "b982a1a2e23a6055fd433999bd4bda24", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9290, "upload_time": "2013-06-21T18:00:59", "url": "https://files.pythonhosted.org/packages/5b/ab/1322ea92e580b5ca79aac1ca49e734a8b45b551ec36ca549519f5245bb12/korg-0.0.2.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "bf1d329ef0745ddcf32c2513133baa60", "sha256": "301b33bfef4013a3f15f232590f8a3bb317db04fa65a643ab8646c4da8a398ad" }, "downloads": -1, "filename": "korg-0.0.3.tar.gz", "has_sig": false, "md5_digest": "bf1d329ef0745ddcf32c2513133baa60", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9286, "upload_time": "2013-06-22T05:45:31", "url": "https://files.pythonhosted.org/packages/a0/1b/95c26a95195248be6af47afe4d8051b88a3be31d382ab3c540b1236c6c51/korg-0.0.3.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "7f1178c19593468d32cbc717a22aebfe", "sha256": "89b58458e79a67df9225f31c37a7d09f6398b651a53d07e2282009860bcb6bff" }, "downloads": -1, "filename": "korg-0.0.5.tar.gz", "has_sig": false, "md5_digest": "7f1178c19593468d32cbc717a22aebfe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20128, "upload_time": "2016-06-18T14:26:46", "url": "https://files.pythonhosted.org/packages/04/a8/ba0f273ad120e2cdce4040a396d169c3acf716ca4f1bef032de3393f89fe/korg-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "65ac63385750e13d8560ea6bd39c994a", "sha256": "be003c75787bf9adac38faf57515c2baa26d50fcf5affde06d0aae4fddf9b0bd" }, "downloads": -1, "filename": "korg-0.0.6.tar.gz", "has_sig": false, "md5_digest": "65ac63385750e13d8560ea6bd39c994a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20126, "upload_time": "2016-10-04T11:08:05", "url": "https://files.pythonhosted.org/packages/a0/e0/41ecbb9fe7f19a58e3c2aeb8b4a225a92aace8ab5169aab33a4f23faeaa3/korg-0.0.6.tar.gz" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "89657c8e5472d8cba0b946647abdff04", "sha256": "533a1eea03304cd689c2554bcd6318a68b328a9c442ecab9c6630ee5d952f00a" }, "downloads": -1, "filename": "korg-0.0.8.tar.gz", "has_sig": false, "md5_digest": "89657c8e5472d8cba0b946647abdff04", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20714, "upload_time": "2017-06-24T18:35:01", "url": "https://files.pythonhosted.org/packages/98/7d/ce7e2ff2ea15bdf4c9271da56da600c4629f877e9c88fbdc249337b11488/korg-0.0.8.tar.gz" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "2cbe5a0a59ddc8a7b5fa8d4dce70774c", "sha256": "6f5ebee3294c66b24c7c02a4c31c865bb582b0aa27b0a0a3f99ff6a577bf4872" }, "downloads": -1, "filename": "korg-0.0.9.tar.gz", "has_sig": false, "md5_digest": "2cbe5a0a59ddc8a7b5fa8d4dce70774c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5991, "upload_time": "2017-06-28T14:55:06", "url": "https://files.pythonhosted.org/packages/99/00/0d0fbb0afcec7a7a84938a755223b770c02bb7c69c0a83ba0eec530eff2f/korg-0.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ada91ff65cd76879ebe5c88debe12d9b", "sha256": "7a3ae01c1ab0ef667c2fcdc5d01b12b4ead0e52563f9bb3d71701fc41f643a50" }, "downloads": -1, "filename": "korg-0.0.11.tar.gz", "has_sig": false, "md5_digest": "ada91ff65cd76879ebe5c88debe12d9b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 19223, "upload_time": "2017-07-03T07:17:48", "url": "https://files.pythonhosted.org/packages/1f/79/58d8ddcca6441f96c288c51a5dbd43264ecd426388c96b28e6c3683d4a9c/korg-0.0.11.tar.gz" } ] }