{ "info": { "author": "Vincent Pelletier", "author_email": "vincent@nexedi.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: System :: Logging", "Topic :: Text Processing :: Filters", "Topic :: Text Processing :: Markup :: HTML" ], "description": ".. contents::\n\nCompute APDEX from Apache-style logs.\n\nOverview\n========\n\nParses Apache-style logs and generates several statistics intended for a\nwebsite developer audience:\n\n- APDEX (Application Performance inDEX, see http://www.apdex.org) ratio\n (plotted)\n\n Because you want to know how satisfied your users are.\n\n- hit count (plotted)\n\n Because achieving 100% APDEX is easy when there is nobody around.\n\n- HTTP status codes, with optional detailed output of the most frequent URLs\n per error status code, along with their most frequent referers\n\n Because your forgot to update a link to that conditionally-used browser\n compatibility javascript you renamed.\n\n- Hottest pages (pages which use rendering time the most)\n\n Because you want to know where to invest time to get highest user experience\n improvement.\n\n- ERP5 sites: per-module statistics, with module and document views separated\n\n Because module and document types are not born equal in usage patterns.\n\nSome parsing performance figures:\n\nOn a 2.3Ghz Corei5, apachedex achieves 97000 lines/s (\npypy-c-jit-62994-bd32583a3f11-linux64) and 43000 lines/s (CPython 2.7).\nThose were measures on a 3000000-hits logfile, with 3 --skip-base, 1\n--erp5-base, 3 --base and --default set. --\\*base values were similar in\nsimplicity to the ones provided in examples below.\n\nWhat APacheDEX is not\n=====================\n\nAPacheDEX does not produce website audience statistics like AWStats, Google\nAnalytics (etc) could do.\n\nAPacheDEX does not monitor website availability & resource usage like Zabbix,\nCacti, Ganglia, Nagios (etc) could do.\n\nRequirements\n============\n\nDependencies\n------------\n\nAs such, apachedex has no strict dependencies outside of standard python 2.7\ninstallation.\nBut generated output needs a few javascript files which come from other\nprojects:\n\n- jquery.js\n\n- jquery.flot.js\n\n- jquery.flot.time.js (official flot plugin)\n\n- jquery.flot.axislabels.js (third-party flot plugin)\n\nIf you installed apachedex (using an egg or with a distribution's package) you\nshould have them already.\nIf you are running from repository, you need to fetch them first::\n\n python setup.py deps\n\nAlso, apachedex can make use of backports.lzma\n(http://pypi.python.org/pypi/backports.lzma/) if it's installed to support xz\nfile compression.\n\nInput\n-----\n\nAll default \"combined\" log format fields are supported (more can easily be\nadded), plus %D.\n\nMandatory fields are (in any order) `%t`, `%r` (for request's URL), `%>s`,\n`%{Referer}i`, `%D`. Just tell apachedex the value from your apache log\nconfiguration (see `--logformat` argument documentation).\n\nInput files may be provided uncompressed or compressed in:\n\n- bzip\n\n- gzip2\n\n- xz (if module backports.lzma is installed)\n\nInput filename \"-\" is understood as stdin.\n\nOutput\n------\n\nThe output is HTML + CSS + JS, so you need a web browser to read it.\n\nOutput filename \"-\" is understood as stdout.\n\nUsage\n=====\n\nA few usage examples. See embedded help (`-h`/`--help`) for further options.\n\nMost basic usage::\n\n apachedex --default website access.log\n\nGenerate stand-alone output (suitable for inclusion in a mail, for example)::\n\n apachedex --default website --js-embed access.log --out attachment.html\n\nA log file with requests for 2 websites for which individual stats are\ndesired, and hits outside those base urls are ignored::\n\n apachedex --base \"/site1(/|$|\\?)\" \"/site2(/|$|\\?)\"\n\nA log file with a site section to ignore. Order does not matter::\n\n apachedex --skip-base \"/ignored(/|$|\\?)\" --default website\n\nA mix of both above examples. Order matters !::\n\n apachedex --skip-base \"/site1/ignored(/|$|\\?)\" \\\n --base \"/site1(/|$|\\?)\" \"/site2(/|$|\\?)\"\n\nMatching non-ASCII urls works by using urlencoded strings::\n\n apachedex --base \"/%E6%96%87%E5%AD%97%E5%8C%96%E3%81%91(/|$|\\\\?)\" access.log\n\nNaming websites so that report looks less intimidating, by interleaving\n\"+\"-prefixed titles with regexes (title must be just before regex)::\n\n apachedex --default \"Public website\" --base \"+Back office\" \\\n \"/backoffice(/|$|\\\\?)\" \"+User access\" \"/secure(/|$|\\\\?)\" access.log\n\nSaving the result of an analysis for faster reuse::\n\n apachedex --default foo --format json --out save_state.json --period day \\\n access.log\n\nAlthough not required, it is strongly advised to provide `--period` argument,\nas mixing states saved with different periods (fixed or auto-detected from\ndata) give hard-to-read results and can cause problems if loaded data gets\nconverted to a larger period.\n\nContinuing a saved analysis, updating collected data::\n\n apachedex --default foo --format json --state-file save_state.json \\\n --out save_state.json --period day access.2.log\n\nGenerating HTML output from two state files, aggregating their content\nwithout parsing more logs::\n\n apachedex --default foo --state-file save_state.json save_state.2.json \\\n --period day --out index.html\n\n\nConfiguration files\n===================\n\nProviding a filename prefixed by \"@\" puts the content of that file in place of\nthat argument, recursively. Each file is loaded relative to the containing\ndirectory of referencing file, or current working directory for command line.\n\n- foo/dev.cfg::\n\n --error-detail\n @site.cfg\n --stats\n\n- foo/site.cfg::\n\n --default Front-office\n # This is a comment\n --prefix \"+Back office\" \"/back(/|$|\\?)\" # This is another comment\n --skip-prefix \"/baz/ignored(/|$|\\?)\" --prefix +Something \"/baz(/|$|\\?)\"\n\n- command line::\n\n apachedex --skip-base \"/ignored(/|$|\\?)\" @foo/dev.cfg --out index.html \\\n access.log\n\nThis is equivalent to::\n\n apachedex --skip-base \"/ignored(/|$|\\?)\" --error-detail \\\n --default Front-office --prefix \"+Back office\" \"/back(/|$|\\?)\" \\\n --skip-prefix \"/baz/ignored(/|$|\\?)\" --prefix +Something \"/baz(/|$|\\?)\" \\\n --stats --out index.html access.log\n\nPortability note: the use of paths containing directory elements inside\nconfiguration files is discouraged, as it's not portable. This may change\nlater (ex: deciding that import paths are URLs and applying their rules).\n\nPeriods\n=======\n\nWhen providing the `--period` argument, two related settings are affected:\n\n- the period represented by each point in a graph (most important for the\n hit graph, as it represents the number of hits per such period)\n\n- the period represented by each column in per-period tables (status codes\n per date, hits per day...)\n\nAlso, when `--period` is not provided, apachedex uses a threshold to tell\nwhen to switch to the larger period. That period was chosen to correspond\nto 200 graph points, which represents a varying number of table columns.\n\n.. table :: Details of `--period` argument\n\n =========== ========== ========== ============== =========================\n --period graph table to next period columns until next period\n =========== ========== ========== ============== =========================\n quarterhour minute 15 minutes 200 minutes 8 (3.3 hours)\n halfday 30 minutes 12 hours 100 hours 9 (4.1 days)\n day hour day 200 hours 9 (8.3 days)\n week 6 hours week 1200 hours 8 (7.1 weeks)\n month day month 5000 hours 7 (~6.7 months)\n quarter 7 days quarter 1400 days 16 (15.3 weeks)\n year month year (n/a) (infinity)\n =========== ========== ========== ============== =========================\n\n\"7 days\" period used in `--period quarter` are not weeks strictly\nspeaking: a week starts a monday/sunday, pendending on the locale.\n\"7 days\" start on the first day of the year, for simplicity - and\nperformance. \"week\" used for `--period week` are really weeks, although\nstarting on monday independently from locale.\n\nWhen there are no hits for more than a graph period, placeholders are\ngenerated at 0 hit value (which is the reality) and 100% apdex (this is\narbitrary). Those placeholders only affect graphs, and do not affect\naverages nor table content.\n\nBecause not all graph periods are actually equal in length (because of\nleap seconds, DST, leap years, year containing a non-integer number of\nweeks), some hit graph points are artificially corrected against these\neffects. Here also, the correction only affects graphs, neither averages\nnor table content. For example, on non-leap years, the last year's\n\"7 days\" period lasts a single day. Ploted hit count is then multiplied\nby 7 (and 3.5 on leap years).\n\nPerformance\n===========\n\nFor better performance...\n\n- pipe decompressed files to apachedex instead of having apachedex decompress\n files itself::\n\n bzcat access.log.bz2 | apachedex [...] -\n\n- when letting apachedex decide statistic granularity with multiple log files,\n provide earliest and latest log files first (whatever order) so apachedex can\n adapt its data structure to analysed time range before there is too much\n data::\n\n apachedex [...] access.log.1.gz access.log.99.gz access.log.2.gz \\\n access.log.3.gz [...] access.98.gz\n\n- parse log files in parallel processes, saving analysis output and aggregating\n them in the end::\n\n for LOG in access*.log; do\n apachedex \"$@\" --format json --out \"$LOG.json\" \"$LOG\" &\n done\n wait\n apachedex \"$@\" --out access.html --state-file access.*.json\n\n If you have bash and have an xargs implementation supporting `-P`, you may\n want to use `parallel_parse.sh` available in source distribution or from\n repository.\n\nNotes\n=====\n\nLoading saved states generated with different sets of parameters is not\nprevented, but can produce nonsense/unreadable results. Or it can save the day\nif you do want to mix different parameters (ex: you have some logs generated\nwith %T, others with %D).\n\nIt is unclear how saved state format will evolve. Be prepared to have\nto regenerate saved states when you upgrade APacheDEX.\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://git.erp5.org/gitweb/apachedex.git", "keywords": "", "license": "GPL 2+", "maintainer": "", "maintainer_email": "", "name": "APacheDEX", "package_url": "https://pypi.org/project/APacheDEX/", "platform": "any", "project_url": "https://pypi.org/project/APacheDEX/", "project_urls": { "Homepage": "http://git.erp5.org/gitweb/apachedex.git" }, "release_url": "https://pypi.org/project/APacheDEX/1.7.0/", "requires_dist": null, "requires_python": "", "summary": "Compute APDEX from Apache-style logs.", "version": "1.7.0" }, "last_serial": 5325513, "releases": { "1.0": [ { "comment_text": "", "digests": { "md5": "aaee38da9938f61e4a2ac2140003a64c", "sha256": "ec6c4dfe8a21bd6ebf6502dfdbf3ecf810a28cc9f97ddaeb58b6fce623dfb10f" }, "downloads": -1, "filename": "APacheDEX-1.0.tar.gz", "has_sig": false, "md5_digest": "aaee38da9938f61e4a2ac2140003a64c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 84576, "upload_time": "2013-04-05T08:37:05", "url": "https://files.pythonhosted.org/packages/29/0f/99384e6e3a39987e1d02daa1dc99dd7a9cb6af161886f6fa7b69ab937b51/APacheDEX-1.0.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "563df4460acac7a9033b69ffc6cebe70", "sha256": "a2bd13c9e9f893b7ceb2f65c11f49abe31f347cae5288e47c09bcde55dc57b86" }, "downloads": -1, "filename": "APacheDEX-1.1.tar.gz", "has_sig": false, "md5_digest": "563df4460acac7a9033b69ffc6cebe70", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 85744, "upload_time": "2013-04-05T17:27:38", "url": "https://files.pythonhosted.org/packages/76/b8/0ed373f12273139fe3ff7b1cea6c827efc0dc05c1b5171a47f340a3cb4c5/APacheDEX-1.1.tar.gz" } ], "1.2": [ { "comment_text": "", "digests": { "md5": "8790458c369014517eb79c95e7da5194", "sha256": "0a2998680429f57ecc30662c86d1a8e2669c3e297d6e7381fe9812fcefc15f7c" }, "downloads": -1, "filename": "APacheDEX-1.2.tar.gz", "has_sig": false, "md5_digest": "8790458c369014517eb79c95e7da5194", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 152759, "upload_time": "2013-04-08T08:15:52", "url": "https://files.pythonhosted.org/packages/bb/9e/c561d466b8cabd29ec9de0465edd01b1c8f0772ae60c11e8c3fdc736b5d1/APacheDEX-1.2.tar.gz" } ], "1.3": [ { "comment_text": "", "digests": { "md5": "98a1c8f47e79b0950046820751c55043", "sha256": "6d47c3d9763e4bec26a2ca0ffce9b9d9640b10e8142bb296adfebafdb8316b35" }, "downloads": -1, "filename": "APacheDEX-1.3.tar.gz", "has_sig": false, "md5_digest": "98a1c8f47e79b0950046820751c55043", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 157324, "upload_time": "2013-04-10T22:05:31", "url": "https://files.pythonhosted.org/packages/1c/cb/22362fa4e93223adeceefb4de7dd332fbdc438e8a03aeed6a7bb273a09b1/APacheDEX-1.3.tar.gz" } ], "1.3.1": [ { "comment_text": "", "digests": { "md5": "f72485e5450bcb1f53a9a58aeba8bee9", "sha256": "dbbfbc91213f18e0c88b6ef1f4a18b18954912e50334e7047f6cadef7292bf88" }, "downloads": -1, "filename": "APacheDEX-1.3.1.tar.gz", "has_sig": false, "md5_digest": "f72485e5450bcb1f53a9a58aeba8bee9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 160209, "upload_time": "2013-04-11T13:09:49", "url": "https://files.pythonhosted.org/packages/64/aa/b74b64e9153bf10a8fea3e97f85b6a30c0c23b75282ccf2429a257d07c4a/APacheDEX-1.3.1.tar.gz" } ], "1.3.2": [ { "comment_text": "", "digests": { "md5": "b780029d1f1f65510a7468b852bcc393", "sha256": "c7b2e72e4962cb1fa5b29afc862a4e0b934aac5ef0b1b391f950b387d10af44b" }, "downloads": -1, "filename": "APacheDEX-1.3.2.tar.gz", "has_sig": false, "md5_digest": "b780029d1f1f65510a7468b852bcc393", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 160229, "upload_time": "2013-04-11T13:24:20", "url": "https://files.pythonhosted.org/packages/d0/2d/fa943e9bd539e1feacd3e5e417bb8b2c2e57bea58553e1ee3fc021b54819/APacheDEX-1.3.2.tar.gz" } ], "1.4": [ { "comment_text": "", "digests": { "md5": "41c4b9fa55eed7a1364ae2c8227846ac", "sha256": "84169fdad9ddf211b494f315dee706e85dbb1eebf30350938d361f69691ce449" }, "downloads": -1, "filename": "APacheDEX-1.4.tar.gz", "has_sig": false, "md5_digest": "41c4b9fa55eed7a1364ae2c8227846ac", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 163699, "upload_time": "2013-04-23T20:49:01", "url": "https://files.pythonhosted.org/packages/fd/34/5b9a6ecb62053f11a6265995bb5f96c0ad291c95f172dbb0f377ddcf0799/APacheDEX-1.4.tar.gz" } ], "1.4.1": [ { "comment_text": "", "digests": { "md5": "648e6dbb4e425e0810f1aa9edb77206f", "sha256": "1568d4485b3c3c22022334dc6b261c7a30e7dd29f8b2e0bec563799fff103371" }, "downloads": -1, "filename": "APacheDEX-1.4.1.tar.gz", "has_sig": false, "md5_digest": "648e6dbb4e425e0810f1aa9edb77206f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 163708, "upload_time": "2013-04-24T19:24:38", "url": "https://files.pythonhosted.org/packages/ac/b8/061dbf26f7bff43d8ab2574381310afac72c886b2c70287a8a828db7c14e/APacheDEX-1.4.1.tar.gz" } ], "1.5": [ { "comment_text": "", "digests": { "md5": "e3848231b8230bf0af119471cefe05e2", "sha256": "3135add5990d888f0784592fc4d7011acb6b78e184fc6e55125ded636284465f" }, "downloads": -1, "filename": "APacheDEX-1.5.tar.gz", "has_sig": false, "md5_digest": "e3848231b8230bf0af119471cefe05e2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 164499, "upload_time": "2013-08-10T10:28:52", "url": "https://files.pythonhosted.org/packages/d1/4c/37a751cfb4d67870f7178883fe60bee6a37184fc964086524764684d3530/APacheDEX-1.5.tar.gz" } ], "1.6": [ { "comment_text": "", "digests": { "md5": "9cdb3610a51c75ddf63a9dbc32b4fb68", "sha256": "aa4e6bc09ac8e809058d53d3df0982bf3f0cea3743ee7b0c6b29f7538aa883cb" }, "downloads": -1, "filename": "APacheDEX-1.6.tar.gz", "has_sig": false, "md5_digest": "9cdb3610a51c75ddf63a9dbc32b4fb68", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 164461, "upload_time": "2014-03-20T15:04:42", "url": "https://files.pythonhosted.org/packages/79/0b/3ff0d495188a63e7eb0245edb4c2681085bccdf85cd0b24ac7314cfdb598/APacheDEX-1.6.tar.gz" } ], "1.6.1": [ { "comment_text": "", "digests": { "md5": "26da13555a1dc20f273e7e0cfc8a1c0b", "sha256": "26797b232c6128d5556b80e67639856d828e93cea866ebca1282288a81a89836" }, "downloads": -1, "filename": "APacheDEX-1.6.1.tar.gz", "has_sig": false, "md5_digest": "26da13555a1dc20f273e7e0cfc8a1c0b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 164559, "upload_time": "2014-03-27T10:00:37", "url": "https://files.pythonhosted.org/packages/7a/17/bf42ea0bde0a1ed7aa39149fc2a9514fc7aca07b5ef44ef5c9e50d707a9b/APacheDEX-1.6.1.tar.gz" } ], "1.6.2": [ { "comment_text": "", "digests": { "md5": "4483dd4fbd53d60134928fc671961866", "sha256": "3f2c5694d97888a8599d6a4a4b151d248cf882eda9fef66003671e772648b977" }, "downloads": -1, "filename": "APacheDEX-1.6.2.tar.gz", "has_sig": false, "md5_digest": "4483dd4fbd53d60134928fc671961866", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 164559, "upload_time": "2014-04-17T13:42:57", "url": "https://files.pythonhosted.org/packages/88/e3/0e56fef1c844c5569a101473f27593f21d1a932effe4228e686b1834a4e0/APacheDEX-1.6.2.tar.gz" } ], "1.6.3": [ { "comment_text": "", "digests": { "md5": "f7f2278f1849561e849e59b122430a2e", "sha256": "d685ca17cca6e08f4b0d86e5475208e185d747ba942d3059eccb6553980e68dc" }, "downloads": -1, "filename": "APacheDEX-1.6.3.tar.gz", "has_sig": false, "md5_digest": "f7f2278f1849561e849e59b122430a2e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 166090, "upload_time": "2018-01-23T08:47:02", "url": "https://files.pythonhosted.org/packages/f2/40/c973cef4350121285cbc0a921fe8d10bd744e0165df66e20e213cdde15dd/APacheDEX-1.6.3.tar.gz" } ], "1.7.0": [ { "comment_text": "", "digests": { "md5": "ac12c841243ba46767718ec0343f27fe", "sha256": "aa7fdbd132be29954278d4215b1b589aa52adec290a2c139360343e2d069c03f" }, "downloads": -1, "filename": "APacheDEX-1.7.0.tar.gz", "has_sig": false, "md5_digest": "ac12c841243ba46767718ec0343f27fe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 164380, "upload_time": "2019-05-28T07:48:36", "url": "https://files.pythonhosted.org/packages/6a/34/753e3e6e48ff5351c2154c1c38a0abeaede4923c0c758c60e9940c18b61f/APacheDEX-1.7.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "ac12c841243ba46767718ec0343f27fe", "sha256": "aa7fdbd132be29954278d4215b1b589aa52adec290a2c139360343e2d069c03f" }, "downloads": -1, "filename": "APacheDEX-1.7.0.tar.gz", "has_sig": false, "md5_digest": "ac12c841243ba46767718ec0343f27fe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 164380, "upload_time": "2019-05-28T07:48:36", "url": "https://files.pythonhosted.org/packages/6a/34/753e3e6e48ff5351c2154c1c38a0abeaede4923c0c758c60e9940c18b61f/APacheDEX-1.7.0.tar.gz" } ] }