{ "info": { "author": "Antonio Valente", "author_email": "y3sman@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Intended Audience :: Developers", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Topic :: Software Development :: Quality Assurance", "Topic :: System :: Monitoring" ], "description": "AppMetrics\n++++++++++\n\n.. image:: https://travis-ci.org/avalente/appmetrics.png?branch=master\n :target: https://travis-ci.org/avalente/appmetrics\n :alt: Build status\n\n\n.. image:: https://coveralls.io/repos/avalente/appmetrics/badge.png\n :target: https://coveralls.io/r/avalente/appmetrics\n :alt: Code coverage\n\n\n``AppMetrics`` is a python library used to collect useful run-time application's metrics, based on\n`Folsom from Boundary `_, which is in turn inspired by\n`Metrics from Coda Hale `_.\n\nThe library's purpose is to help you collect real-time metrics from your Python applications,\nbeing them web apps, long-running batches or whatever. ``AppMetrics`` is not a persistent store,\nyou must provide your own persistence layer, maybe by using well established monitoring tools.\n\n``AppMetrics`` works on python 2.7 and 3.3.\n\nGetting started\n---------------\n\nInstall ``AppMetrics`` into your python environment::\n\n pip install appmetrics\n\nor, if you don't use ``pip``, download and unpack the package an then::\n\n python setup.py install\n\nOnce you have installed ``AppMetrics`` you can access it by the ``metrics`` module::\n\n >>> from appmetrics import metrics\n >>> histogram = metrics.new_histogram(\"test\")\n >>> histogram.notify(1.0)\n True\n >>> histogram.notify(2.0)\n True\n >>> histogram.notify(3.0)\n True\n >>> histogram.get()\n {'arithmetic_mean': 2.0, 'kind': 'histogram', 'skewness': 0.0, 'harmonic_mean': 1.6363636363636365, 'min': 1.0, 'standard_deviation': 1.0, 'median': 2.0, 'histogram': [(3.0, 3), (5.0, 0)], 'percentile': [(50, 2.0), (75, 2.0), (90, 3.0), (95, 3.0), (99, 3.0), (99.9, 3.0)], 'n': 3, 'max': 3.0, 'variance': 1.0, 'geometric_mean': 1.8171205928321397, 'kurtosis': -2.3333333333333335}\n\nBasically you create a new metric by using one of the ``metrics.new_*`` functions. The metric will be stored into\nan internal registry, so you can access it in different places in your application::\n\n >>> test_histogram = metrics.metric(\"test\")\n >>> test_histogram.notify(4.0)\n True\n\nThe ``metrics`` registry is thread-safe, you can safely use it in multi-threaded web servers.\n\nUsing the ``with_histogram`` decorator we can time a function::\n\n >>> import time, random\n >>> @metrics.with_histogram(\"test\")\n ... def my_worker():\n ... time.sleep(random.random())\n ...\n >>> my_worker()\n >>> my_worker()\n >>> my_worker()\n\nand let's see the results::\n\n >>> metrics.get(\"test\")\n {'arithmetic_mean': 0.41326093673706055, 'kind': 'histogram', 'skewness': 0.2739718270714368, 'harmonic_mean': 0.14326954591313346, 'min': 0.0613858699798584, 'standard_deviation': 0.4319169569113129, 'median': 0.2831099033355713, 'histogram': [(1.0613858699798584, 3), (2.0613858699798584, 0)], 'percentile': [(50, 0.2831099033355713), (75, 0.2831099033355713), (90, 0.895287036895752), (95, 0.895287036895752), (99, 0.895287036895752), (99.9, 0.895287036895752)], 'n': 3, 'max': 0.895287036895752, 'variance': 0.18655225766752892, 'geometric_mean': 0.24964828731906127, 'kurtosis': -2.3333333333333335}\n\nIt is also possible to time specific sections of the code by using the ``timer`` context manager::\n\n >>> import time, random\n ... def my_worker():\n ... with metrics.timer(\"test\"):\n ... time.sleep(random.random())\n ...\n\nLet's print the metrics data on the screen every 5 seconds::\n\n >>> from appmetrics import reporter\n >>> def stdout_report(metrics):\n ... print metrics\n ...\n >>> reporter.register(stdout_report, reporter.fixed_interval_scheduler(5))\n '5680173c-0279-46ec-bd88-b318f8058ef4'\n >>> {'test': {'arithmetic_mean': 0.0, 'kind': 'histogram', 'skewness': 0.0, 'harmonic_mean': 0.0, 'min': 0, 'standard_deviation': 0.0, 'median': 0.0, 'histogram': [(0, 0)], 'percentile': [(50, 0.0), (75, 0.0), (90, 0.0), (95, 0.0), (99, 0.0), (99.9, 0.0)], 'n': 0, 'max': 0, 'variance': 0.0, 'geometric_mean': 0.0, 'kurtosis': 0.0}}\n >>> my_worker()\n >>> my_worker()\n >>> {'test': {'arithmetic_mean': 0.5028266906738281, 'kind': 'histogram', 'skewness': 0.0, 'harmonic_mean': 0.2534044030939462, 'min': 0.14868521690368652, 'standard_deviation': 0.50083167520453, 'median': 0.5028266906738281, 'histogram': [(1.1486852169036865, 2), (2.1486852169036865, 0)], 'percentile': [(50, 0.14868521690368652), (75, 0.8569681644439697), (90, 0.8569681644439697), (95, 0.8569681644439697), (99, 0.8569681644439697), (99.9, 0.8569681644439697)], 'n': 2, 'max': 0.8569681644439697, 'variance': 0.2508323668881758, 'geometric_mean': 0.35695727672917066, 'kurtosis': -2.75}}\n >>> reporter.remove('5680173c-0279-46ec-bd88-b318f8058ef4')\n \n\n\n\nDecorators\n**********\n\nThe ``metrics`` module also provides a couple of decorators: ``with_histogram`` and ``with_meter`` which are\nan easy and fast way to use ``AppMetrics``: just decorate your functions/methods and you will have metrics\ncollected for them. You can decorate multiple functions with the same metric's name, as long as the decorator's\ntype and parameters are the same, or a ``DuplicateMetricError`` will be raised.\nSee the documentation for `Histograms`_ and `Meters`_ for more details.\n\n\nAPI\n---\n\n``AppMetrics`` exposes a simple and consistent API; all the metric objects have three methods:\n\n * ``notify(value)`` - add a new value to the metric\n * ``get()`` - get the computed metric's value (if any)\n * ``raw_data()`` - get the raw data stored in the metrics\n\nHowever, the ``notify`` input type and the ``get()`` and ``raw_data()`` data format depend on the kind\nof metric chosen. Please notice that ``get()`` returns a dictionary with the mandatory\nfield ``kind`` which depends on the metric's type.\n\nMetrics\n-------\n\nSeveral metric types are available:\n\nCounters\n********\n\nCounter metrics provide increment and decrement capabilities for a single integer value.\nThe ``notify`` method accepts an integer: the counter will be incremented or decremented according\nto the value's sign. Notice that the function tries to cast the input value to integer, so\na ``TypeError`` or a ``ValueError`` may be raised::\n\n >>> counter = metrics.new_counter(\"test\")\n >>> counter.notify(10)\n >>> counter.notify(-5)\n >>> counter.get()\n {'kind': 'counter', 'value': 5}\n >>> counter.notify(\"wrong\")\n Traceback (most recent call last):\n File \"\", line 1, in \n File \"appmetrics/simple_metrics.py\", line 40, in notify\n value = int(value)\n ValueError: invalid literal for int() with base 10: 'wrong'\n\nGauges\n******\n\nGauges are point-in-time single value metrics. The ``notify`` method accepts any data type::\n\n >>> gauge = metrics.new_gauge(\"gauge_test\")\n >>> gauge.notify(\"version 1.0\")\n >>> gauge.get()\n {'kind': 'gauge', 'value': 'version 1.0'}\n\nThe ``gauge`` metric is useful to expose almost-static values such as configuration parameters, constants and so on.\nAlthough you can use any python data type as the value, you won't be able to use the ``wsgi`` middleware unless\nyou use a valid ``json`` type.\n\nHistograms\n**********\n\nHistograms are collections of values on which statistical analysis are performed automatically. They are useful\nto know how the application is performing. The ``notify`` method accepts a single floating-point value, while\nthe ``get`` method computes and returns the following values:\n\n * arithmetic mean\n * geometric mean\n * harmonic mean\n * data distribution histogram with automatic bins\n * kurtosis\n * maximum value\n * median\n * minimum value\n * number of values\n * 50, 75, 90, 95, 99 and 99.9th percentiles of the data distribution\n * skewness\n * standard deviation\n * variance\n\nNotice that the ``notify`` method tries to cast the input value to a float, so a ``TypeError`` or a ``ValueError`` may\nbe raised.\n\nYou can use the histogram metric also by the ``with_histogram`` decorator: the time spent in the decorated\nfunction will be collected by an ``histogram`` with the given name::\n\n >>> @metrics.with_histogram(\"histogram_test\")\n ... def fun(v):\n ... return v*2\n ...\n >>> fun(10)\n 20\n >>> metrics.metric(\"histogram_test\").raw_data()\n [5.9604644775390625e-06]\n\nThe full signature is::\n\n with_histogram(name, reservoir_type, *reservoir_args, **reservoir_kwargs)\n\nwhere:\n\n * name is the metric's name\n * reservoir_type is a string which identifies a ``reservoir`` class, see reservoirs documentation\n * reservoir_args and reservoir_kwargs are passed to the chosen reservoir's \\_\\_init\\_\\_\n\n\nSample types\n^^^^^^^^^^^^\n\nTo avoid unbound memory usage, the histogram metrics are generated from a *reservoir* of values.\n\nUniform reservoir\n.................\n\nThe default *reservoir* type is the *uniform* one, in which a fixed number of values (default 1028)\nis kept, and when the reservoir is full new values replace older ones randomly with an uniform\nprobability distribution, ensuring that the sample is always statistically representative.\nThis kind of reservoir must be used when you are interested in statistics over the whole stream of\nobservations. Use ``\"uniform\"`` as ``reservoir_type`` in ``with_histogram``.\n\n\nSliding window reservoir\n........................\n\nThis *reservoir* keeps a fixed number of observations (default 1028) and when a new value comes in the first\none is discarded. The statistics are representative of the last N observations. Its ``reservoir_type``\nis ``sliding_window``.\n\nSliding time window reservoir\n.............................\n\nThis *reservoir* keeps observation for a fixed amount of time (default 60 seconds), older values get discarded.\nThe statistics are representative of the last N seconds, but if you have a lot of readings in N seconds this could\neat a lot amount of memory. Its ``reservoir_type`` is ``sliding_time_window``.\n\nExponentially-decaying reservoir\n................................\n\nThis *reservoir* keeps a fixed number of values (default 1028), with\n`exponential decaying `_ of older values\nin order to give greater significance to recent data. The bias towards newer values can be adjusted by\nspecifying a proper `alpha` value to the reservoir's init (defaults to 0.015).\nIts ``reservoir_type`` is ``exp_decaying``.\n\n\nMeters\n******\n\nMeters are increment-only counters that measure the rate of events (such as ``\"http requests\"``) over time. This kind of\nmetric is useful to collect throughput values (such as ``\"requests per second\"``), both on average and on different time\nintervals::\n\n >>> meter = metrics.new_meter(\"meter_test\")\n >>> meter.notify(1)\n >>> meter.notify(1)\n >>> meter.notify(3)\n >>> meter.get()\n {'count': 5, 'kind': 'meter', 'five': 0.0066114184713530035, 'mean': 0.27743058841197027, 'fifteen': 0.0022160607980413085, 'day': 2.3147478365093123e-05, 'one': 0.031982234148270686}\n\nThe return values of the ``get`` method are the following:\n\n * ``count``: number of operations collected so far\n * ``mean``: the average throughput since the metric creation\n * ``one``: one-minute\n `exponentially-weighted moving average `_\n (*EWMA*)\n * ``five``: five-minutes *EWMA*\n * ``fifteen``: fifteen-minutes *EWMA*\n * ``day``: last day *EWMA*\n * ``kind``: \"meter\"\n\nNotice that the ``notify`` method tries to cast the input value to an integer, so a ``TypeError`` or a ``ValueError``\nmay be raised.\n\nYou can use the meter metric also by the ``with_meter`` decorator: the number of calls to the decorated\nfunction will be collected by a ``meter`` with the given name.\n\nTagging\n-------\n\nYou can group several metrics together by \"tagging\" them::\n\n >>> metrics.new_histogram(\"test1\")\n \n >>> metrics.new_gauge(\"test2\")\n \n >>> metrics.new_meter(\"test3\")\n \n >>> metrics.tag(\"test1\", \"group1\")\n >>> metrics.tag(\"test3\", \"group1\")\n >>> metrics.tags()\n {'group1': set(['test1', 'test3'])}\n >>> metrics.metrics_by_tag(\"group1\")\n {'test1': {'arithmetic_mean': 0.0, 'skewness': 0.0, 'harmonic_mean': 0.0, 'min': 0, 'standard_deviation': 0.0, 'median': 0.0, 'histogram': [(0, 0)], 'percentile': [(50, 0.0), (75, 0.0), (90, 0.0), (95, 0.0), (99, 0.0), (99.9, 0.0)], 'n': 0, 'max': 0, 'variance': 0.0, 'geometric_mean': 0.0, 'kurtosis': 0.0}, 'test3': {'count': 0, 'five': 0.0, 'mean': 0.0, 'fifteen': 0.0, 'day': 0.0, 'one': 0.0}}\n >>> metrics.untag('test1', 'group1')\n True\n >>> metrics.untag('test1', 'group1')\n False\n\n\nAs you can see above, four functions are available:\n\n * ``metrics.tag(metric_name, tag_name)``: tag the metric named ```` with ````.\n Raise ``InvalidMetricError`` if ```` does not exist.\n * ``metrics.tags()``: return the currently defined tags.\n * ``metrics.metrics_by_tag(tag_name)``: return a dictionary with metric names as keys\n and metric values as returned by ``.get()``. Return an empty dictionary if ``tag_name`` does\n not exist.\n * ``metrics.untag(metric_name, tag_name)``: remove the tag named ```` from the metric named\n ````. Return True if the tag was removed, False if either the metric or the tag did not exist. When a\n tag is no longer used, it gets implicitly removed.\n\n\nExternal access\n---------------\n\nYou can access the metrics provided by ``AppMetrics`` externally by the ``WSGI``\nmiddleware found in ``appmetrics.wsgi.AppMetricsMiddleware``. It is a standard ``WSGI``\nmiddleware with only ``werkzeug`` as external dependency and it can be plugged in any framework supporting\nthe ``WSGI`` standard, for example in a ``Flask`` application::\n\n from flask import Flask\n from appmetrics import metrics\n\n metrics.new_histogram(\"test-histogram\")\n metrics.new_gauge(\"test-counter\")\n metrics.metric(\"test-counter\").notify(10)\n\n app = Flask(__name__)\n\n @app.route('/hello')\n def hello_world():\n return 'Hello World!'\n\n if __name__ == '__main__':\n from appmetrics.wsgi import AppMetricsMiddleware\n app.wsgi_app = AppMetricsMiddleware(app.wsgi_app)\n app.run()\n\nIf you launch the above application you can ask for metrics::\n\n $ curl http://localhost:5000/hello\n Hello World!\n\n $ curl http://localhost:5000/_app-metrics\n [\"test-counter\", \"test-histogram\"]\n\n $ curl http://localhost:5000/_app-metrics/test-counter\n 10\n\nIn this way you can easily expose your application's metrics to an external monitoring service.\nMoreover, since the ``AppMetricsMiddleware`` exposes a full *RESTful API*, you can create metrics\nfrom anywhere and also populate them with foreign application's data.\n\nUsage\n*****\n\nAs usual, instantiate the middleware with the wrapped ``WSGI`` application; it looks for\nrequest paths starting with ``\"/_app-metrics\"``: if not found, the wrapped application\nis called. The following resources are defined:\n\n``/_app-metrics/metrics``\n - **GET**: return the list of the registered metrics\n``/_app-metrics/metrics/``\n - **GET**: return the value of the given metric or ``404``.\n - **PUT**: create a new metric with the given name. The body must be a ``JSON`` object with a\n mandatory attribute named ``\"type\"`` which must be one of the metrics types allowed,\n by the ``\"metrics.METRIC_TYPES\"`` dictionary, while the other attributes are\n passed to the ``new_`` function as keyword arguments.\n Request's ``content-type`` must be ``\"application/json\"``.\n - **POST**: add a new value to the metric. The body must be a ``JSON`` object with a mandatory\n attribute named ``\"value\"``: the notify method will be called with the given value.\n Other attributes are ignored.\n Request's ``content-type`` must be ``\"application/json\"``.\n - **DELETE**: remove the metric with the given name. Return \"deleted\" or \"not deleted\".\n``/_app-metrics/tags``\n - **GET**: return the list of registered tags\n``/_app-metrics/tags/``\n - **GET**: return the metrics tagged with the given tag. If the value of the ``GET`` parameter ``\"expand\"``\n is ``\"true\"``, a JSON object is returned, with the name of each tagged metric as keys and corresponding values.\n If it is ``\"false\"`` or not provided, the list of metric names is returned.\n Return a ``404`` if the tag does not exist\n``/_app-metrics/tags//``\n - **PUT**: tag the metric named ```` with ````. Return a ``400`` if the given metric\n does not exist.\n - **DELETE**: remove the tag ```` from ````. Return \"deleted\" or \"not deleted\". If\n ```` is no longer used, it gets implicitly removed.\n\n\nThe response body is always encoded in JSON, and the ``Content-Type`` is ``application/json``.\nThe root doesn't have to be ``\"/_app-metrics\"``, you can customize it by providing your own to\nthe middleware constructor.\n\nA standalone ``AppMetrics`` webapp can be started by using ``werkzeug``'s development server::\n\n $ python -m werkzeug.serving appmetrics.wsgi.standalone_app\n * Running on http://127.0.0.1:5000/\n\nThe standalone app mounts on the root (no ``_app-metrics`` prefix). DON'T use it for production purposes!!!\n\nReporting\n---------\n\n``AppMetrics`` provides another easy way to get your application's metrics: the ``reporter`` module. It allows\nto register any number of callbacks that will be called at scheduled times with the metrics, allowing you\nto \"export\" your application's metrics into your favourite storage system.\nThe main entry point for the ``reporter`` feature is ``reporter.register``::\n\n reporter.register(callback, schedule, tag=None)\n\nwhere:\n\n* *callback* must be a callback function that will be called with a dictionary of ``{metric name: metric values}``\n* *schedule* must be an iterable object yielding a future timestamp (in ``time.time()`` format) at each iteration\n* *tag* must be a tag to narrow the involved metrics to the ones with that tag, if ``None`` all the\n available metrics will be used.\n\nWhen a callback is registered, a new thread will be started, waiting for the next scheduled call. Please notice\nthat the callback will be executed in a thread. ``register`` returns an opaque id identifying the registration.\n\nA callback registration can be removed by calling ``reporter.remove`` with the id returned by ``register``.\n\n``reporter`` provides a simple scheduler object, ``fixed_interval_scheduler``::\n\n >>> sched = reporter.fixed_interval_scheduler(10)\n >>> next(sched)\n 1397297405.672592\n >>> next(sched)\n 1397297415.672592\n >>> next(sched)\n 1397297425.672592\n\nCSV reporter\n************\n\nA simple reporter callback is exposed by ``reporter.CSVReporter``. As the name suggests, it will create\ncsv reports with metric values, a file for each metric, a row for each call. See ``examples/csv_reporter.py``\n\n\nTesting\n-------\n\n``AppMetrics`` has an exhaustive, fully covering test suite, made up by both doctests and unit tests. To run the\nwhole test suite (including the coverage test), just issue::\n\n $ nosetests --with-coverage --cover-package=appmetrics --cover-erase\n\nYou will need to install a couple of packages in your python environment, the list is in the\n``\"requirements.txt\"`` file.\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/avalente/appmetrics", "keywords": "metrics,folsom,histogram,metric,monitor", "license": "Apache 2.0", "maintainer": null, "maintainer_email": null, "name": "AppMetrics", "package_url": "https://pypi.org/project/AppMetrics/", "platform": "Platform Independent", "project_url": "https://pypi.org/project/AppMetrics/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/avalente/appmetrics" }, "release_url": "https://pypi.org/project/AppMetrics/0.5.0/", "requires_dist": null, "requires_python": null, "summary": "Application metrics collector", "version": "0.5.0" }, "last_serial": 1604426, "releases": { "0.2": [ { "comment_text": "", "digests": { "md5": "cd3679d410ca1aa5ba218fa51e8c2f6a", "sha256": "0082cac28e3da9dc4354d8b807f4adb514b33021c4658eb5582cf97296a578e7" }, "downloads": -1, "filename": "AppMetrics-0.2.tar.gz", "has_sig": false, "md5_digest": "cd3679d410ca1aa5ba218fa51e8c2f6a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 22476, "upload_time": "2014-03-16T12:29:26", "url": "https://files.pythonhosted.org/packages/8b/66/b159ecb7d83647eb23839a3b635ecae191cba1abc3ac2271475ebb62c53b/AppMetrics-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "7c8b6c4b8236f6a3b35adff503592df2", "sha256": "b3073d825b20f1a77955790873f7f9eaa17faf453bbae3c8ce784675530a25b0" }, "downloads": -1, "filename": "AppMetrics-0.3.tar.gz", "has_sig": false, "md5_digest": "7c8b6c4b8236f6a3b35adff503592df2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 31935, "upload_time": "2014-04-13T10:14:09", "url": "https://files.pythonhosted.org/packages/b1/00/dc529c5b63e55ce4e015c8056d66fe35250059b6bc6d0c1d9d00105c0b6b/AppMetrics-0.3.tar.gz" } ], "0.4.2": [ { "comment_text": "", "digests": { "md5": "02b65cb81e662c9aaa3c1327527e1ec9", "sha256": "d0c981ed3ce2e629a83a8d2e243e8fa5a32016194d11dd16c54ce90872a4aeb7" }, "downloads": -1, "filename": "AppMetrics-0.4.2.tar.gz", "has_sig": false, "md5_digest": "02b65cb81e662c9aaa3c1327527e1ec9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33136, "upload_time": "2014-11-01T19:01:35", "url": "https://files.pythonhosted.org/packages/4e/57/192e5427e37ac7d95ec52a1066c17fa320cb85e2ed40e0acf527ca66c997/AppMetrics-0.4.2.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "6fea728a7ea6068b18f613a3a152e3cd", "sha256": "6b4a767a3219c7b430dd89898705a2b31f186cb22612c43cf48835f88da6f5b7" }, "downloads": -1, "filename": "AppMetrics-0.5.0.tar.gz", "has_sig": false, "md5_digest": "6fea728a7ea6068b18f613a3a152e3cd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33343, "upload_time": "2015-06-24T09:04:11", "url": "https://files.pythonhosted.org/packages/ec/67/36b09c8845d2a83f4746626a0ad90907d96e8e215bd0276cd1b87d5c53ad/AppMetrics-0.5.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "6fea728a7ea6068b18f613a3a152e3cd", "sha256": "6b4a767a3219c7b430dd89898705a2b31f186cb22612c43cf48835f88da6f5b7" }, "downloads": -1, "filename": "AppMetrics-0.5.0.tar.gz", "has_sig": false, "md5_digest": "6fea728a7ea6068b18f613a3a152e3cd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 33343, "upload_time": "2015-06-24T09:04:11", "url": "https://files.pythonhosted.org/packages/ec/67/36b09c8845d2a83f4746626a0ad90907d96e8e215bd0276cd1b87d5c53ad/AppMetrics-0.5.0.tar.gz" } ] }