{ "info": { "author": "keul", "author_email": "luca@keul.it", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: System Administrators", "License :: OSI Approved :: GNU General Public License (GPL)", "Operating System :: OS Independent", "Programming Language :: Python", "Topic :: Internet :: Log Analysis", "Topic :: Internet :: WWW/HTTP", "Topic :: Utilities" ], "description": ".. contents:: **Table of contents**\n\nIntroduction\n============\n\nThis project adds to your system a new utility command: ``tinylogan``. This utility only works with\nApache-like access HTTP log where the response time data is enabled.\n\nTo know how to do this, see `this blog post`__ or, in brief change the configuration of your log format\nfrom something like::\n\n LogFormat \"%h %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\"\" combined\n\nTo this::\n\n LogFormat \"%h %l %u %t \\\"%r\\\" %>s %b \\\"%{Referer}i\\\" \\\"%{User-Agent}i\\\" %T/%D\" combined\n\nThe log record will change to something like this::\n\n [31/Jan/2008:14:19:07 +0000] \"GET / HTTP/1.1\" 200 7918 \"\"\n ... \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20061201 Firefox/2.0.0.11 (Ubuntu-feisty)\" 0/95491\n\n__ http://www.ducea.com/2008/02/06/apache-logs-how-long-does-it-take-to-serve-a-request/ \n\nSeconds and microsends\n----------------------\n\nThe utility only cares about microsends (`%D`) so you need to have `Apache 2`__.\n\n__ http://httpd.apache.org/docs/\n\nHow to use\n==========\n\nHere the complete help::\n\n Usage: tinylogan [options] logfile\n \n Simple bash utility for analyze HTTP access log with enabled response time\n \n Options:\n --version show program's version number and exit\n -h, --help show this help message and exit\n -v, --verbose verbose output during log analysis\n -s SIZE, --size=SIZE choose the number of record to store in every log\n -q, --keep-query keep query strings in URLs instead of cutting them.\n Using this an URL with different query string is treat\n like different URLs.\n -i INCLUDE_REGEX, --include=INCLUDE_REGEX\n a regexp expression that an URLs must match of will be\n discarded. Can be called multiple times, expanding the\n set\n -e EXCLUDE_REGEX, --exclude=EXCLUDE_REGEX\n a regexp expression that an URLs must not match of\n will be discarded. Can be called multiple times,\n reducing the set\n --skip-day=SKIP_DAY A regexp that repr specific whole day or a set of\n dates that must be ignored. Can be called multiple\n times\n --min-time=MIN_TIME_MILLIS\n ignore all entries that require less than this amount\n of millisecs\n --max-time=MAX_TIME_MILLIS\n ignore all entries that require more than this amount\n of millisecs\n --min-times=MIN_TIMES\n set a minimum number of times that a entry must be\n found to be used in the \"Top average time\" statistic\n \n Date filters:\n For those kind of filters you need to specify a date. You are free to\n use a specific date in the format dd/mmm/aaaa, like \"24/May/2011\", but\n also some keywords for relative date like \"today\", \"yesterday\",\n \"tomorrow\", \"week\" and \"month\". Use of \"week\" and \"month\" mean\n referring to first day of the current week or month. You can also\n provide a numerical modifier using \"+\" or \"-\" followed by a day\n quantity (example: \"week-5\" for going back of 5 days from the start of\n the week).\n \n --start-date=START_DATE\n date where to start analyze and record\n --end-date=END_DATE\n date where to end analyze and record\n \n Time filters:\n When a time is needed, you must enter it in the format hh:mm:ss or\n simply hh:mm, like \"09:21:30\" or \"09:21\". Those filter are used for\n skip record that are registered \"too late at night\" or \"too early in\n the morning\".\n \n --skip-timeperiod-start=SKIP_TIME_START\n do not analyse records before the given time\n --skip-timeperiod-end=SKIP_TIME_END\n do not analyse records later the given time\n \n Default configuration profiles:\n You can read a set of default configuration options from a\n \".tinylogan\" file placed in the user's home directory. If this file is\n found, parameters from the \"DEFAULT\" section are read, but you can\n also add other sections. You can always override those options from\n the command line.\n \n -c PROFILE read a different profile section than DEFAULT\n -U Ignore the user default profile file (if exists)\n --example-profile Print out an example profile file, then exit. You can\n put this output in a \".tinylogan\" file in your home,\n then customize it\n\nYou can also configure your defaults values in a ``.tinylogan`` config file\nplaced in your user's home. Read help above for details.\n\nResults\n=======\n\nLet explain the given results::\n\n Starting from 15/Apr/2011:08:19:06\n enough... stopped by user action\n Ending at 28/Apr/2011:17:00:36\n Elapsed time: 0:00:04.955008\n Timedelta is 13 days, 8:41:30 (but only 7 days, 9:41:30 are counted due to time bounds)\n \n Top total time\n 0001 - /url1 46591.603 (4924 times, average 9.462, 7.28% of the total)\n 0002 - /url2 12660.053 (1212 times, average 10.446, 1.98% of the total)\n ...\n \n Top average time\n 0001 - /url3 32.828 (15 times, 492 total)\n 0002 - /url4 30.549 (7 times, 213 total)\n ...\n\n``Starting from ...``\n First valid entry found in the log\n``enough... stopped by user action``\n Only if you CTRL+C during the log analysis. This will stop the log scan and skip to results immediatly\n``Ending at ...``\n Last entry analyzed\n``Elapsed time: ...``\n Time required for the log analysis\n``Timedelta is ...``\n Number of days from the first and last entry of the log, important for giving to the users a percent\n of the total time taken from an entry.\n \n If you use some of the time filters above the used value for the statistic is the one given in the\n sentence ``but only xxx are counted due to time bounds``.\n\nTop total time\n--------------\n\nThis will show, from the most consuming time to the less ones, a hierarchy of the URLs that take the most\ntime from the analyzed log::\n\n \n Total number of seconds taken\n | Average time per call\n Entry position | |\n | | |\n 0001 - /url1 46591.603 (4924 times, average 9.462, 7.28% of the total)\n | | |\n URL of the entry | |\n | Percentage of the total time\n Times called\n\nTop average time\n----------------\n\nThis will show, from the most slow entry to the less ones, a hierarchy of the URLs that seems slowest,\nconsidering the average time per hit.\n\nNote that you could like to use the ``--min-times`` option for have a better statiscal report for this.\nWithout giving this option, a on-time call to a very slow procedure will probably be reported in this\nhierarchy, even if it will not give you a good average data.\n\nLet's details::\n\n Average number of seconds taken\n |\n Entry position | Total time in seconds\n | | |\n 0001 - /url3 32.828 (15 times, 492 total)\n | |\n URL of the entry |\n |\n Times called\n\nTODO\n====\n\n* a way to ignore min and max values from multiple occurrences of a match\n* right now all records are stored in memory... obviously this is not the way to\n parse a potentially multiple-gigabyte-long-file\n* a way to recognize default views (like: that ``foo/other_foo`` is the same as\n ``foo/other_foo/index.html``)\n* right now the log is read from the first line. In this way reaching a far-from-first\n entry, when using the ``--start-date`` is used, can be *really* slow\n\nChangelog\n=========\n\n0.4.0 (2011-10-04)\n------------------\n\n* when error happens during log analysis, report the error\n line number\n* fixed *severe* bug in the month array, that simply make this\n script useless for everything after August\n* use the python logging module for handle error and verbosity;\n this clean the output a little\n* log a warning if encounter a line that doesn't match the log\n format\n* added ``week`` and ``month`` date filters\n* added quantity modifiers to date filters\n* fixed documentation help\n* added the new ``skip-day`` filter\n* implemented users default profiles and added related options\n* do not continue log analysis when the ``--end-date`` is found\n* the ``--include`` option was buggy. Now using it more than once will\n include additional URLs to the set\n\n0.3.0 (2011-06-23)\n------------------\n\n* added \"Elapsed time\" report\n* added ``--verbose`` option\n* added a way to skip records if they require too little,\n or too much time\n* handled keyboard interrupt (*CTRL+C*); the first one will\n simply stop the log analysis while the second terminate\n the process\n* added the ``--min-times`` option, to control when an entry\n must be in the \"Top average time\" statistic\n* more complete documentation\n\n0.2.2 (2011-05-02)\n------------------\n\n* incredibly, I forgot a month: ``May`` was missing!\n\n0.2.1 (2011-04-29)\n------------------\n\n* descriptions of ``--skip-timeperiod-start`` and\n ``-skip-timeperiod-end`` were inverted\n* URLs to the root page (\"``/``\") were transformed to\n empty strings\n* reveted changes that automatically change space in a ``%20``\n for filters (not a good idea for something that must be\n a valid regexp)\n\n0.2.0 (2011-04-28)\n------------------\n\n* pep8 cleanup\n* URLs with trailing ``/`` now are collected has the same as URLs\n without ``/``\n* filters with spaces chars are now handled as ``%20``\n* added ``-q`` option, for not merging anymore URLs different\n only in query string\n\n0.1.0 (2011-04-27)\n------------------\n\n* initial release", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://kpython-utils.svn.sourceforge.net/svnroot/kpython-utils/TinyLogAnalyzer/", "keywords": "file apache log response-time analyser http access", "license": "GPL", "maintainer": null, "maintainer_email": null, "name": "TinyLogAnalyzer", "package_url": "https://pypi.org/project/TinyLogAnalyzer/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/TinyLogAnalyzer/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://kpython-utils.svn.sourceforge.net/svnroot/kpython-utils/TinyLogAnalyzer/" }, "release_url": "https://pypi.org/project/TinyLogAnalyzer/0.4.0/", "requires_dist": null, "requires_python": null, "summary": "Command line utility for perform response time analysis onto HTTP access logs", "version": "0.4.0" }, "last_serial": 785805, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "7b741657a68ecfb6b5bd7fc356b08562", "sha256": "6491901e394cba32004e09ae6b1d84bef1a239a2c4145173dd013e134230c842" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.1.0-py2.6.egg", "has_sig": false, "md5_digest": "7b741657a68ecfb6b5bd7fc356b08562", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 5879, "upload_time": "2011-04-27T20:59:15", "url": "https://files.pythonhosted.org/packages/ce/8d/6667df164879ddc598799d04fbe49e4d156cd21dbaafe2ea32cc0ce5fa40/TinyLogAnalyzer-0.1.0-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "2b5c1923ebc3b3d9d92b6a62e4c9fc57", "sha256": "2c32ab125b58d19a9af8263fb483af917a3531adbbf5d73c5b1168f2d795be7c" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.1.0.tar.gz", "has_sig": false, "md5_digest": "2b5c1923ebc3b3d9d92b6a62e4c9fc57", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10624, "upload_time": "2011-04-27T20:59:15", "url": "https://files.pythonhosted.org/packages/23/16/60f9603ba3a4067bcbb14adfbbb565e958616857f61c7190782cd99a4981/TinyLogAnalyzer-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "11921551ff096812c98849ccd81d077f", "sha256": "ef4d0b440dc7437afb0a8f36a577f4627156769f50f047215aa203400d11d19f" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.2.0-py2.6.egg", "has_sig": false, "md5_digest": "11921551ff096812c98849ccd81d077f", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 6239, "upload_time": "2011-04-28T17:42:28", "url": "https://files.pythonhosted.org/packages/76/9c/d8bc3ba47a3f68baad09ceec6803fa95e849ee019a7bc8c03413f5948c03/TinyLogAnalyzer-0.2.0-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "cc973616fc02c9306eb712108efd9aba", "sha256": "2762b2ee00110ed8d442bb7cabd2bb72378ccaf2538095aa95a932cf027876ee" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.2.0.tar.gz", "has_sig": false, "md5_digest": "cc973616fc02c9306eb712108efd9aba", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10984, "upload_time": "2011-04-28T17:42:27", "url": "https://files.pythonhosted.org/packages/84/26/f83359ba8918c7257cf48ceb3262c987c3a9d785446857c6e9b4e628940c/TinyLogAnalyzer-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "775a659f0d3300614c66746322caedc9", "sha256": "d2569f369561371d495b2ecb7f79757681e21ab2b6b58c2d7b9c99c6892f21b7" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.2.1-py2.6.egg", "has_sig": false, "md5_digest": "775a659f0d3300614c66746322caedc9", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 6374, "upload_time": "2011-04-29T15:52:44", "url": "https://files.pythonhosted.org/packages/ba/74/5105759c0c63bd463b41f7aaf98ba81f99e07168f10dd80f58e966f858e3/TinyLogAnalyzer-0.2.1-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "467759e85edf0298556fa07f4655da65", "sha256": "b8a222efa6514f34e760bc466fbb2b37c2a2a8f535eaf6de1bb8c51c72e9d78d" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.2.1.tar.gz", "has_sig": false, "md5_digest": "467759e85edf0298556fa07f4655da65", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11155, "upload_time": "2011-04-29T15:52:42", "url": "https://files.pythonhosted.org/packages/43/c5/92d5e8795f41e8062424e6bc6641fc9101be360dfa99aa960561c25a97a8/TinyLogAnalyzer-0.2.1.tar.gz" } ], "0.2.2": [ { "comment_text": "", "digests": { "md5": "0f59aa9beb2389781ee81e0f6ec2027e", "sha256": "f2e23f1e3fda58b4110458587dd98adc1f19a8c99676179d75a512e4e13e9e98" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.2.2-py2.6.egg", "has_sig": false, "md5_digest": "0f59aa9beb2389781ee81e0f6ec2027e", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 6409, "upload_time": "2011-05-02T15:34:56", "url": "https://files.pythonhosted.org/packages/fe/6f/dfb303d8533afa4a6a1a24dc19d9c4b32a6bd60d230202d90dcff15b48c4/TinyLogAnalyzer-0.2.2-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "32a580eb523100d8ad97f8d860d1ab4c", "sha256": "8b78be41889e392282ac63335ad021dab3b8683dcb32e79d5690c200f1078c42" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.2.2.tar.gz", "has_sig": false, "md5_digest": "32a580eb523100d8ad97f8d860d1ab4c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11222, "upload_time": "2011-05-02T15:34:55", "url": "https://files.pythonhosted.org/packages/93/02/9ec3c9d258cca363ca1a726c70f585c1dd40e754466aff7264c1ba091397/TinyLogAnalyzer-0.2.2.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "74f88a7a6800b5fd0b242b93fbf60292", "sha256": "06098d9f5acfb25387d808565c71fa5c20b695bd992677d00740dbec1da80c12" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.3.0-py2.6.egg", "has_sig": false, "md5_digest": "74f88a7a6800b5fd0b242b93fbf60292", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 8450, "upload_time": "2011-06-23T11:54:56", "url": "https://files.pythonhosted.org/packages/0e/db/c4cd40477193aa26a482bbd0848138b2a395e56c2fd2d82b65fb76f74476/TinyLogAnalyzer-0.3.0-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "03792c0cfdaaac75d4fe743a938672ef", "sha256": "2288f9edcc718690a8c7b648886001732b37f49ac4dbcae65e5ff261b4def81b" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.3.0.tar.gz", "has_sig": false, "md5_digest": "03792c0cfdaaac75d4fe743a938672ef", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 13416, "upload_time": "2011-06-23T11:54:55", "url": "https://files.pythonhosted.org/packages/ca/1e/957ce373997111e636378512f7c52df1cd1ebcf96f5caafe8f0861907d20/TinyLogAnalyzer-0.3.0.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "265ebf7f9579112e7501e19bbc3bd090", "sha256": "b4718e2c0e926db93d8df81f6b5d770388f50934b6ad7b0670d2384ae7293a46" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.4.0-py2.6.egg", "has_sig": false, "md5_digest": "265ebf7f9579112e7501e19bbc3bd090", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 19015, "upload_time": "2011-10-04T23:28:14", "url": "https://files.pythonhosted.org/packages/ce/37/f3542adb8812e8b210e2ce292768e7c5caf9843151f7d54b6673cf42273a/TinyLogAnalyzer-0.4.0-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "8d89ac476b20d7fbaca713bedce0ef8b", "sha256": "aecf671033c5e79c55d36e3c7c318aff47b1bd44426a9cc2080f5df38a20811c" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.4.0.tar.gz", "has_sig": false, "md5_digest": "8d89ac476b20d7fbaca713bedce0ef8b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20150, "upload_time": "2011-10-04T23:28:13", "url": "https://files.pythonhosted.org/packages/80/f5/a6efe7389b9951913320418b3f5f0afecc75bbb6ed71b5b9d1d8d0174912/TinyLogAnalyzer-0.4.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "265ebf7f9579112e7501e19bbc3bd090", "sha256": "b4718e2c0e926db93d8df81f6b5d770388f50934b6ad7b0670d2384ae7293a46" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.4.0-py2.6.egg", "has_sig": false, "md5_digest": "265ebf7f9579112e7501e19bbc3bd090", "packagetype": "bdist_egg", "python_version": "2.6", "requires_python": null, "size": 19015, "upload_time": "2011-10-04T23:28:14", "url": "https://files.pythonhosted.org/packages/ce/37/f3542adb8812e8b210e2ce292768e7c5caf9843151f7d54b6673cf42273a/TinyLogAnalyzer-0.4.0-py2.6.egg" }, { "comment_text": "", "digests": { "md5": "8d89ac476b20d7fbaca713bedce0ef8b", "sha256": "aecf671033c5e79c55d36e3c7c318aff47b1bd44426a9cc2080f5df38a20811c" }, "downloads": -1, "filename": "TinyLogAnalyzer-0.4.0.tar.gz", "has_sig": false, "md5_digest": "8d89ac476b20d7fbaca713bedce0ef8b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 20150, "upload_time": "2011-10-04T23:28:13", "url": "https://files.pythonhosted.org/packages/80/f5/a6efe7389b9951913320418b3f5f0afecc75bbb6ed71b5b9d1d8d0174912/TinyLogAnalyzer-0.4.0.tar.gz" } ] }