{ "info": { "author": "Julien Tayon", "author_email": "julien@tayon.net", "bugtrack_url": null, "classifiers": [], "description": "--------------------------------------------\n\n1. This LICENSE AGREEMENT is between the Python Software Foundation\n(\"PSF\"), and the Individual or Organization (\"Licensee\") accessing and\notherwise using this software (\"Python\") in source or binary form and\nits associated documentation.\n\n2. Subject to the terms and conditions of this License Agreement, PSF\nhereby grants Licensee a nonexclusive, royalty-free, world-wide\nlicense to reproduce, analyze, test, perform and/or display publicly,\nprepare derivative works, distribute, and otherwise use Python\nalone or in any derivative version, provided, however, that PSF's\nLicense Agreement and PSF's notice of copyright, i.e., \"Copyright (c)\n2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights\nReserved\" are retained in Python alone or in any derivative version\nprepared by Licensee.\n\n3. In the event Licensee prepares a derivative work that is based on\nor incorporates Python or any part thereof, and wants to make\nthe derivative work available to others as provided herein, then\nLicensee hereby agrees to include in any such work a brief summary of\nthe changes made to Python.\n\n4. PSF is making Python available to Licensee on an \"AS IS\"\nbasis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR\nIMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND\nDISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS\nFOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT\nINFRINGE ANY THIRD PARTY RIGHTS.\n\n5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON\nFOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS\nA RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,\nOR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.\n\n6. This License Agreement will automatically terminate upon a material\nbreach of its terms and conditions.\n\n7. Nothing in this License Agreement shall be deemed to create any\nrelationship of agency, partnership, or joint venture between PSF and\nLicensee. This License Agreement does not grant permission to use PSF\ntrademarks or trade name in a trademark sense to endorse or promote\nproducts or services of Licensee, or any third party.\n\n8. By copying, installing or otherwise using Python, Licensee\nagrees to be bound by the terms and conditions of this License\nAgreement.\n\nBEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0\n-------------------------------------------\n\nBEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1\n\n1. This LICENSE AGREEMENT is between BeOpen.com (\"BeOpen\"), having an\noffice at 160 Saratoga Avenue, Santa Clara, CA 95051, and the\nIndividual or Organization (\"Licensee\") accessing and otherwise using\nthis software in source or binary form and its associated\ndocumentation (\"the Software\").\n\n2. Subject to the terms and conditions of this BeOpen Python License\nAgreement, BeOpen hereby grants Licensee a non-exclusive,\nroyalty-free, world-wide license to reproduce, analyze, test, perform\nand/or display publicly, prepare derivative works, distribute, and\notherwise use the Software alone or in any derivative version,\nprovided, however, that the BeOpen Python License is retained in the\nSoftware, alone or in any derivative version prepared by Licensee.\n\n3. BeOpen is making the Software available to Licensee on an \"AS IS\"\nbasis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR\nIMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND\nDISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS\nFOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT\nINFRINGE ANY THIRD PARTY RIGHTS.\n\n4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE\nSOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS\nAS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY\nDERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.\n\n5. This License Agreement will automatically terminate upon a material\nbreach of its terms and conditions.\n\n6. This License Agreement shall be governed by and interpreted in all\nrespects by the law of the State of California, excluding conflict of\nlaw provisions. Nothing in this License Agreement shall be deemed to\ncreate any relationship of agency, partnership, or joint venture\nbetween BeOpen and Licensee. This License Agreement does not grant\npermission to use BeOpen trademarks or trade names in a trademark\nsense to endorse or promote products or services of Licensee, or any\nthird party. As an exception, the \"BeOpen Python\" logos available at\nhttp://www.pythonlabs.com/logos.html may be used according to the\npermissions granted on that web page.\n\n7. By copying, installing or otherwise using the software, Licensee\nagrees to be bound by the terms and conditions of this License\nAgreement.\n\nCNRI OPEN SOURCE LICENSE AGREEMENT (for Python 1.6b1)\n--------------------------------------------------\n\nIMPORTANT: PLEASE READ THE FOLLOWING AGREEMENT CAREFULLY.\n\nBY CLICKING ON \"ACCEPT\" WHERE INDICATED BELOW, OR BY COPYING,\nINSTALLING OR OTHERWISE USING PYTHON 1.6, beta 1 SOFTWARE, YOU ARE\nDEEMED TO HAVE AGREED TO THE TERMS AND CONDITIONS OF THIS LICENSE\nAGREEMENT.\n\n1. This LICENSE AGREEMENT is between the Corporation for National\nResearch Initiatives, having an office at 1895 Preston White Drive,\nReston, VA 20191 (\"CNRI\"), and the Individual or Organization\n(\"Licensee\") accessing and otherwise using Python 1.6, beta 1\nsoftware in source or binary form and its associated documentation,\nas released at the www.python.org Internet site on August 4, 2000\n(\"Python 1.6b1\").\n\n2. Subject to the terms and conditions of this License Agreement, CNRI\nhereby grants Licensee a non-exclusive, royalty-free, world-wide\nlicense to reproduce, analyze, test, perform and/or display\npublicly, prepare derivative works, distribute, and otherwise use\nPython 1.6b1 alone or in any derivative version, provided, however,\nthat CNRIs License Agreement is retained in Python 1.6b1, alone or\nin any derivative version prepared by Licensee.\n\nAlternately, in lieu of CNRIs License Agreement, Licensee may\nsubstitute the following text (omitting the quotes): \"Python 1.6,\nbeta 1, is made available subject to the terms and conditions in\nCNRIs License Agreement. This Agreement may be located on the\nInternet using the following unique, persistent identifier (known\nas a handle): 1895.22/1011. This Agreement may also be obtained\nfrom a proxy server on the Internet using the\nURL:http://hdl.handle.net/1895.22/1011\".\n\n3. In the event Licensee prepares a derivative work that is based on\nor incorporates Python 1.6b1 or any part thereof, and wants to make\nthe derivative work available to the public as provided herein,\nthen Licensee hereby agrees to indicate in any such work the nature\nof the modifications made to Python 1.6b1.\n\n4. CNRI is making Python 1.6b1 available to Licensee on an \"AS IS\"\nbasis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR\nIMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND\nDISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR\nFITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6b1\nWILL NOT INFRINGE ANY THIRD PARTY RIGHTS.\n\n5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE\nSOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR\nLOSS AS A RESULT OF USING, MODIFYING OR DISTRIBUTING PYTHON 1.6b1,\nOR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY\nTHEREOF.\n\n6. This License Agreement will automatically terminate upon a material\nbreach of its terms and conditions.\n\n7. This License Agreement shall be governed by and interpreted in all\nrespects by the law of the State of Virginia, excluding conflict of\nlaw provisions. Nothing in this License Agreement shall be deemed\nto create any relationship of agency, partnership, or joint venture\nbetween CNRI and Licensee. This License Agreement does not grant\npermission to use CNRI trademarks or trade name in a trademark\nsense to endorse or promote products or services of Licensee, or\nany third party.\n\n8. By clicking on the \"ACCEPT\" button where indicated, or by copying,\ninstalling or otherwise using Python 1.6b1, Licensee agrees to be\nbound by the terms and conditions of this License Agreement.\n\nACCEPT\n\nCWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2\n--------------------------------------------------\n\nCopyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,\nThe Netherlands. All rights reserved.\n\nPermission to use, copy, modify, and distribute this software and its\ndocumentation for any purpose and without fee is hereby granted,\nprovided that the above copyright notice appear in all copies and that\nboth that copyright notice and this permission notice appear in\nsupporting documentation, and that the name of Stichting Mathematisch\nCentrum or CWI not be used in advertising or publicity pertaining to\ndistribution of the software without specific, written prior\npermission.\n\nSTICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO\nTHIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND\nFITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE\nFOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\nWHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\nACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT\nOF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n\nDescription: \n - source: https://github.com/jul/yahi\n - doc: http://yahi.readthedocs.org/\n - ticketting: https://github.com/jul/yahi/issues\n \n \n Versatile log parser (providing default extractors for apache/lighttpd/varnish)\n ===============================================================================\n \n Command line usage\n ------------------\n \n Simplest usage is::\n \n speed_shoot -g /usr/local/data/geoIP /var/www/apache/access*log\n \n \n it will return a json in the form::\n \n {\n \"by_date\": {\n \"2012-5-3\": 11\n }, \n \"total_line\": 11, \n \"ip_by_url\": {\n \"/favicon.ico\": {\n \"192.168.0.254\": 2, \n \"192.168.0.35\": 2\n }, \n \"/\": {\n \"74.125.18.162\": 1, \n \"192.168.0.254\": 1, \n \"192.168.0.35\": 5\n }\n }, \n \"by_status\": {\n \"200\": 7, \n \"404\": 4\n }, \n \"by_dist\": {\n \"unknown\": 11\n }, \n \"bytes_by_ip\": {\n \"74.125.18.162\": 151, \n \"192.168.0.254\": 489, \n \"192.168.0.35\": 1093\n }, \n \"by_url\": {\n \"/favicon.ico\": 4, \n \"/\": 7\n }, \n \"by_os\": {\n \"unknown\": 11\n }, \n \"week_browser\": {\n \"3\": {\n \"unknown\": 11\n }\n }, \n \"by_referer\": {\n \"-\": 11\n }, \n \"by_browser\": {\n \"unknown\": 11\n }, \n \"by_ip\": {\n \"74.125.18.162\": 1, \n \"192.168.0.254\": 3, \n \"192.168.0.35\": 7\n }, \n \"by_agent\": {\n \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0,gzip(gfe) (via translate.google.com)\": 1, \n \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0\": 10\n }, \n \"by_hour\": {\n \"9\": 3, \n \"10\": 4, \n \"11\": 1, \n \"12\": 3\n }, \n \"by_country\": {\n \"\": 10, \n \"US\": 1\n }\n }\n \n \n If you use::\n \n speed_shoot -f csv -g /usr/local/data/geoIP /var/www/apache/access*log\n \n \n Your result is::\n \n by_date,2012-5-3,11\n total_line,11\n ip_by_url,/favicon.ico,192.168.0.254,2\n ip_by_url,/favicon.ico,192.168.0.35,2\n ip_by_url,/,74.125.18.162,1\n ip_by_url,/,192.168.0.254,1\n ip_by_url,/,192.168.0.35,5\n by_status,200,7\n by_status,404,4\n by_dist,unknown,11\n bytes_by_ip,74.125.18.162,151\n bytes_by_ip,192.168.0.254,489\n bytes_by_ip,192.168.0.35,1093\n by_url,/favicon.ico,4\n by_url,/,7\n by_os,unknown,11\n week_browser,3,unknown,11\n by_referer,-,11\n by_browser,unknown,11\n by_ip,74.125.18.162,1\n by_ip,192.168.0.254,3\n by_ip,192.168.0.35,7\n by_agent,\"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0,gzip(gfe) (via translate.google.com)\",1\n by_agent,Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0,10\n by_hour,9,3\n by_hour,10,4\n by_hour,11,1\n by_hour,12,3\n by_country,,10\n by_country,US,1\n \n \n Well I guess, it does not work because you first need to fetch geoIP data file::\n \n mkdir data\n wget -O- \"http://www.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz\" | zcat > data/GeoIP.dat\n \n Of course, this is the geoLite database, I don't include the data in the package\n since geoIP must be updated often to stay accurate. \n \n Default path for geoIP is data/GeoIP.dat\n \n Use as a script\n ---------------\n \n speed shoot is in fact a template of how to use yahi as a module::\n \n #!/usr/bin/env python\n from archery.bow import Hankyu as _dict\n from yahi import notch, shoot\n from datetime import datetime\n \n \n context=notch()\n date_formater= lambda dt :\"%s-%s-%s\" % ( dt.year, dt.month, dt.day)\n context.output(\n shoot(\n context,\n lambda data : _dict({\n 'by_country': _dict({data['_country']: 1}),\n 'by_date': _dict({date_formater(data['_datetime']): 1 }),\n 'by_hour': _dict({data['_datetime'].hour: 1 }),\n 'by_os': _dict({data['_os_name']: 1 }),\n 'by_dist': _dict({data['_dist_name']: 1 }),\n 'by_browser': _dict({data['_browser_name']: 1 }),\n 'by_ip': _dict({data['ip']: 1 }),\n 'by_status': _dict({data['status']: 1 }),\n 'by_url': _dict({data['uri']: 1}),\n 'by_agent': _dict({data['agent']: 1}),\n 'by_referer': _dict({data['referer']: 1}),\n 'ip_by_url': _dict({data['uri']: _dict( {data['ip']: 1 })}),\n 'bytes_by_ip': _dict({data['ip']: int(data['bytes'])}),\n 'week_browser' : _dict({data['_datetime'].weekday():\n _dict({data[\"_browser_name\"] :1 })}),\n 'total_line' : 1,\n }),\n ),\n )\n \n \n \n Installation\n ============\n \n easy as::\n \n pip install yahi\n \n or::\n \n easy_install yahi\n \n Recommanded usage\n =================\n \n - for basic log aggregation, I do recommand using command line;\n - for one shot metrics I recommend an interactive console (bpython or ipython);\n - for specific metrics or elaborate filters I recommand using the API. \n \n CHANGELOG\n =========\n \n 0.1.3\n -----\n \n Adding varnish incomplete regexp for log parsing (I miss 2 fields)\n \n \nKeywords: log,parsing\nPlatform: UNKNOWN\nClassifier: License :: OSI Approved :: Python Software Foundation License\nClassifier: Operating System :: MacOS :: MacOS X\nClassifier: Operating System :: Microsoft :: Windows\nClassifier: Operating System :: POSIX\nClassifier: Programming Language :: Python\n", "description_content_type": "", "docs_url": "https://pythonhosted.org/yahi/", "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://yahi.readthedocs.org/", "keywords": "", "license": "PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2", "maintainer": "", "maintainer_email": "", "name": "yahi", "package_url": "https://pypi.org/project/yahi/", "platform": "", "project_url": "https://pypi.org/project/yahi/", "project_urls": { "Homepage": "http://yahi.readthedocs.org/" }, "release_url": "https://pypi.org/project/yahi/0.1.4/", "requires_dist": null, "requires_python": "", "summary": "Versatile parallel log parser", "version": "0.1.4" }, "last_serial": 4419359, "releases": { "0.1.0": [ { "comment_text": "built for Linux-3.0.0-15-generic-x86_64-with-glibc2.7", "digests": { "md5": "3f80594c5ce4b72d0967ab972e608fed", "sha256": "c5f92150b9ea34471c4abd4279697d51fdb7801d5b9b760a4d726eecbe4b6a9e" }, "downloads": -1, "filename": "yahi-0.1.0.linux-x86_64.tar.gz", "has_sig": false, "md5_digest": "3f80594c5ce4b72d0967ab972e608fed", "packagetype": "bdist_dumb", "python_version": "any", "requires_python": null, "size": 26264, "upload_time": "2012-08-31T15:35:25", "url": "https://files.pythonhosted.org/packages/c9/4d/0cd9575e6ee6e951257277118ed72d4ae8858d1403563e0ad47bd8e37a93/yahi-0.1.0.linux-x86_64.tar.gz" }, { "comment_text": "", "digests": { "md5": "dcecc85f44621d04dcd7b623bd2fdd19", "sha256": "9fd6d0be9f15a9998f15329fe24ca20de297ee880cfef9a450cd74431cafa00f" }, "downloads": -1, "filename": "yahi-0.1.0.tar.gz", "has_sig": false, "md5_digest": "dcecc85f44621d04dcd7b623bd2fdd19", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13791, "upload_time": "2012-08-31T15:35:23", "url": "https://files.pythonhosted.org/packages/0d/47/b66337002a899aa779613c8f1daf208914d16b13656611e45e40825bd04c/yahi-0.1.0.tar.gz" } ], "0.1.1": [ { "comment_text": "built for Linux-3.0.0-15-generic-x86_64-with-glibc2.7", "digests": { "md5": "75d4020f62aba8aa6e40173c793daed5", "sha256": "2631e55271c58eff0b8d3c4e55c0b6407b52349bce55ebbc1bf02b2f29b331ef" }, "downloads": -1, "filename": "yahi-0.1.1.linux-x86_64.tar.gz", "has_sig": false, "md5_digest": "75d4020f62aba8aa6e40173c793daed5", "packagetype": "bdist_dumb", "python_version": "any", "requires_python": null, "size": 26274, "upload_time": "2012-09-01T08:50:19", "url": "https://files.pythonhosted.org/packages/7c/0e/0fdd6dad52104bd03e8adf5ea272d3dc6652b6a92308d42417bedb0df472/yahi-0.1.1.linux-x86_64.tar.gz" }, { "comment_text": "", "digests": { "md5": "516159184b5eea75767ee98479b41e40", "sha256": "a7df06bb9083730bd87ab1eec5127d6bb7675b68a9d2ab6e9da368774a9e180a" }, "downloads": -1, "filename": "yahi-0.1.1.tar.gz", "has_sig": false, "md5_digest": "516159184b5eea75767ee98479b41e40", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13796, "upload_time": "2012-09-01T08:50:02", "url": "https://files.pythonhosted.org/packages/11/be/4acefcd70f073f46baea077da1e5ac4d810a0d473eb34f8eeb970eb61eab/yahi-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "5b7dd5014484e0dd0adca8a9cef67d69", "sha256": "ce88c41fbb62d142eba0afdb14326845c9697b76b33211a939b0cb4fb0fe05dc" }, "downloads": -1, "filename": "yahi-0.1.2.tar.gz", "has_sig": false, "md5_digest": "5b7dd5014484e0dd0adca8a9cef67d69", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14363, "upload_time": "2012-09-03T14:16:57", "url": "https://files.pythonhosted.org/packages/12/e3/ae22d22c51dfa0e8d4814a1907d778f8505710a181318bc153c59f78ccdb/yahi-0.1.2.tar.gz" } ], "0.1.3a0": [ { "comment_text": "", "digests": { "md5": "b00409373aca6eded1fa152a4657fae1", "sha256": "0aaf0d43a18fefa2b5575515cbde41c6a2e86d10e71f2ca68e895ae9705a9ada" }, "downloads": -1, "filename": "yahi-0.1.3a0.tar.gz", "has_sig": true, "md5_digest": "b00409373aca6eded1fa152a4657fae1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15139, "upload_time": "2018-10-26T13:24:11", "url": "https://files.pythonhosted.org/packages/ec/1f/3edc3e39cb49f7b4418f7687343767a7e44a07eb8890601a2bdae90e6167/yahi-0.1.3a0.tar.gz" } ], "0.1.4": [ { "comment_text": "", "digests": { "md5": "9ce93d7db3370afeae1473181e48de29", "sha256": "5543c9044a3a875c14f8e2c366eba91d82b555bc06dd33ad44ed2a341dc23ec1" }, "downloads": -1, "filename": "yahi-0.1.4.tar.gz", "has_sig": true, "md5_digest": "9ce93d7db3370afeae1473181e48de29", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15139, "upload_time": "2018-10-26T13:37:50", "url": "https://files.pythonhosted.org/packages/a9/ab/24b036b6560a1d5d48af2a53c4ef8739281f656d5c26088db336dcbc9bde/yahi-0.1.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "9ce93d7db3370afeae1473181e48de29", "sha256": "5543c9044a3a875c14f8e2c366eba91d82b555bc06dd33ad44ed2a341dc23ec1" }, "downloads": -1, "filename": "yahi-0.1.4.tar.gz", "has_sig": true, "md5_digest": "9ce93d7db3370afeae1473181e48de29", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15139, "upload_time": "2018-10-26T13:37:50", "url": "https://files.pythonhosted.org/packages/a9/ab/24b036b6560a1d5d48af2a53c4ef8739281f656d5c26088db336dcbc9bde/yahi-0.1.4.tar.gz" } ] }