{ "info": { "author": "sashgorokhov", "author_email": "sashgorokhov@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 3 - Alpha", "Environment :: Plugins", "Framework :: Scrapy", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: System :: Monitoring" ], "description": "scrapy_prometheus\n*****************\n\n.. image:: https://img.shields.io/pypi/status/scrapy_prometheus.svg\n :target: https://github.com/sashgorokhov/scrapy_prometheus\n\n.. image:: https://travis-ci.org/sashgorokhov/scrapy_prometheus.svg?branch=master\n :target: https://travis-ci.org/sashgorokhov/scrapy_prometheus\n\n.. image:: https://img.shields.io/github/license/sashgorokhov/scrapy_prometheus.svg\n :target: https://raw.githubusercontent.com/sashgorokhov/scrapy_prometheus/master/LICENSE\n\n.. image:: https://img.shields.io/pypi/pyversions/scrapy_prometheus.svg\n :target: https://pypi.python.org/pypi/scrapy-prometheus\n\n.. image:: https://badge.fury.io/py/scrapy_prometheus.svg \n :target: https://badge.fury.io/py/scrapy-prometheus\n\nScrapy stats collector that exports scrapy stats as prometheus metrics through pushgateway service.\n\nInstallation\n============\n\nVia pip:\n\n.. code-block:: console\n\n pip install scrapy_prometheus\n\n\nUsage\n=====\n\nTo start using, add ``scrapy_prometheus.PrometheusStatsCollector`` to STATS_CLASS setting:\n\n.. code-block:: python\n\n STATS_CLASS = 'scrapy_prometheus.PrometheusStatsCollector'\n\n\nThis stats collector works exactly like the vanilla one (because it subclasses it), but also\ncreates prometheus metrics and pushes them to pushgateway service on spider close signal.\n\nIt supports two metric types: ``Counter`` and ``Gauge``. Stat metric type is determined by operation used on\nthe stat: ``stats.inc_value`` will create a ``Counter`` metric, while other methods,\n``stats.set_value``, ``stats.max_value``, ``stats.min_value``, will create ``Gauge``.\n\nAll metrics will have a `spider` label attached with spider name.\n\nStat value must be either ``int`` or ``float``.\n\nNote, that trying to perform action on a metric, that is not supposed to be used with this\naction (set_value on Counter or inc_value on Gauge) will produce\n``scrapy_prometheus.InvalidMetricType`` error. To suppress it, set ``PROMETHEUS_SUPPRESS_TYPE_CHECK`` to True.\n\nIf you want to create custom metrics, you can access your spider's CollectorRegistry by using ``stats.get_registry(spider)``.\n\nAvailable settings\n==================\n\n.. code-block:: python\n\n # Prometheus pushgateway host\n PROMETHEUS_PUSHGATEWAY = 'localhost:9091' # default\n\n # Metric name prefix\n PROMETHEUS_METRIC_PREFIX = 'scrapy_prometheus' # default\n \n # Timeout for pushing metrics to pushgateway\n PROMETHEUS_PUSH_TIMEOUT = 5 # default\n \n # Method to use when pushing metrics\n # Read https://github.com/prometheus/pushgateway#put-method\n PROMETHEUS_PUSH_METHOD = 'POST' # default\n\n # Do not raise scrapy_prometheus.InvalidMetricType when stat is accessed as different type metric.\n # For example, doing stats.inc_value('foo', 1) and then stats.set_value('foo', 2) will raise an error,\n # Because metric of type Counter was already created for stat foo.\n PROMETHEUS_SUPPRESS_TYPE_CHECK = False\n\n # job label value, applied to all metrics.\n PROMETHEUS_JOB = 'scrapy' # default\n\n # grouping label dict, applied to all metrics.\n # by default it is an instance key with hostname value.\n PROMETHEUS_GROUPING_KEY = {'instance': }\n\n\nHow metrics are created\n=======================\n\nMetric name is build from ``PROMETHEUS_METRIC_PREFIX`` and stat name, where all ``/`` are replaced with ``_``.\n\nFor example:\n\n* stat ``foo: 67`` whill produce metric ``scrapy_prometheus_foo{instance=\"...\",job=\"scrapy\",spider=\"...\"} 67``\n* stat ``foo/bar: 67`` whill produce metric ``scrapy_prometheus_foo_bar{instance=\"...\",job=\"scrapy\",spider=\"...\"} 67``\n* stat ``foo/bar/baz: 67`` whill produce metric ``scrapy_prometheus_foo_bar_baz{instance=\"...\",job=\"scrapy\",spider=\"...\"} 67``", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/sashgorokhov/scrapy_prometheus/archive/master.zip", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/sashgorokhov/scrapy_prometheus", "keywords": "scrapy,prometheus,pushgateway,monitoring", "license": "MIT License", "maintainer": "", "maintainer_email": "", "name": "scrapy-prometheus", "package_url": "https://pypi.org/project/scrapy-prometheus/", "platform": "", "project_url": "https://pypi.org/project/scrapy-prometheus/", "project_urls": { "Download": "https://github.com/sashgorokhov/scrapy_prometheus/archive/master.zip", "Homepage": "https://github.com/sashgorokhov/scrapy_prometheus" }, "release_url": "https://pypi.org/project/scrapy-prometheus/0.4.4/", "requires_dist": null, "requires_python": "", "summary": "Exporting scrapy stats as prometheus metrics through pushgateway service", "version": "0.4.4" }, "last_serial": 3178923, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "dedaa6575bd079dc06f86d1f5727f049", "sha256": "3858e1112446d140dca2be4e8f6a34cc8d33570560771aa1ae615b84d8dc3e71" }, "downloads": -1, "filename": "scrapy_prometheus-0.1.tar.gz", "has_sig": false, "md5_digest": "dedaa6575bd079dc06f86d1f5727f049", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 2470, "upload_time": "2017-09-12T00:48:22", "url": "https://files.pythonhosted.org/packages/cb/e2/adef9394dcc0a374d84483a2aaec9b52e9c871deda64fa0cc603ca152da0/scrapy_prometheus-0.1.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "554361ab2317f34554c5c80650a5c104", "sha256": "d11e2058390e91810486d85ae047252d677563c7bcf4ec5329d2fd0d1d4479b0" }, "downloads": -1, "filename": "scrapy_prometheus-0.3.tar.gz", "has_sig": false, "md5_digest": "554361ab2317f34554c5c80650a5c104", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3476, "upload_time": "2017-09-12T23:32:03", "url": "https://files.pythonhosted.org/packages/aa/90/2a52351bdb1c14e079983b560ca68c583a145786dca177492534c1df0b5c/scrapy_prometheus-0.3.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "9ec791888348c240affe95ed1af4feec", "sha256": "2c36f672c772cd4333343321c03a9d7999b32634fbff2a9f73aa2fc9171a54e5" }, "downloads": -1, "filename": "scrapy_prometheus-0.3.1.tar.gz", "has_sig": false, "md5_digest": "9ec791888348c240affe95ed1af4feec", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3479, "upload_time": "2017-09-13T10:42:13", "url": "https://files.pythonhosted.org/packages/82/58/0d24808e9200294b08b4dafaa5cd408b17b780a477617298b4af17ec9d45/scrapy_prometheus-0.3.1.tar.gz" } ], "0.3.2": [ { "comment_text": "", "digests": { "md5": "df78fadaaab1246d3c57bf71e5685b3a", "sha256": "f38170ea53a59bd2f9399e5f47974a9bc36dae4e7931efebb8a5ca0a12330559" }, "downloads": -1, "filename": "scrapy_prometheus-0.3.2.tar.gz", "has_sig": false, "md5_digest": "df78fadaaab1246d3c57bf71e5685b3a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3549, "upload_time": "2017-09-15T16:00:45", "url": "https://files.pythonhosted.org/packages/8d/2e/7be980ce6d18e2ae657257124d4ea9cdf8e31004b80593067be1a57fc3e6/scrapy_prometheus-0.3.2.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "a9332982cf7710324a8dd8462abb46b7", "sha256": "664f9791c8a745a0524f42e620e4a1f66c46f3233ff66142f7b8112c5c0f316c" }, "downloads": -1, "filename": "scrapy_prometheus-0.4.tar.gz", "has_sig": false, "md5_digest": "a9332982cf7710324a8dd8462abb46b7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3661, "upload_time": "2017-09-15T16:53:45", "url": "https://files.pythonhosted.org/packages/e7/79/b815c7040b418369dd5adb8ab93dc12620fc8c4653a00bee6404d0d7e769/scrapy_prometheus-0.4.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "a420a7dc2ee1de9bf2396e9fcedc6212", "sha256": "23d1f364f2173c3b23f596410b9db56e81589cb8f107217a4a500260b83ff04e" }, "downloads": -1, "filename": "scrapy_prometheus-0.4.1.tar.gz", "has_sig": false, "md5_digest": "a420a7dc2ee1de9bf2396e9fcedc6212", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3672, "upload_time": "2017-09-15T17:44:29", "url": "https://files.pythonhosted.org/packages/6f/c5/127c6ab888c7fa46beb65a2db2d4fd9ac954bfca88ba2d90081e4a439281/scrapy_prometheus-0.4.1.tar.gz" } ], "0.4.2": [ { "comment_text": "", "digests": { "md5": "b7a3d208a27599e844a068695a06ae57", "sha256": "ed0adb08be798729c4c23063d99d92e680162c9cfc2db6e3bb80d54718485077" }, "downloads": -1, "filename": "scrapy_prometheus-0.4.2.tar.gz", "has_sig": false, "md5_digest": "b7a3d208a27599e844a068695a06ae57", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3687, "upload_time": "2017-09-15T19:06:06", "url": "https://files.pythonhosted.org/packages/45/9c/64e931345cab6f2659ae1672f9ce737b154a9b957c715224961ebb11fd8a/scrapy_prometheus-0.4.2.tar.gz" } ], "0.4.3": [ { "comment_text": "", "digests": { "md5": "29460b4e1ef484ec70a0a354cd7da34f", "sha256": "41a5de1e881f308ae35a75cc874c357d9432e5e6135153704f46dd5bd99f5a6f" }, "downloads": -1, "filename": "scrapy_prometheus-0.4.3.tar.gz", "has_sig": false, "md5_digest": "29460b4e1ef484ec70a0a354cd7da34f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3984, "upload_time": "2017-09-15T20:15:38", "url": "https://files.pythonhosted.org/packages/1d/23/24e8b9f3a0bb3d943965b32e1023b2e15732b36722b18bb2ca1546c8a149/scrapy_prometheus-0.4.3.tar.gz" } ], "0.4.4": [ { "comment_text": "", "digests": { "md5": "5fd8a51e1f138f482a3ded5710c92430", "sha256": "0c501c33ccfa4483533f18e101b3892ecdb1a2fd851d1af8a6e8e8503d9ba20f" }, "downloads": -1, "filename": "scrapy_prometheus-0.4.4.tar.gz", "has_sig": false, "md5_digest": "5fd8a51e1f138f482a3ded5710c92430", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3996, "upload_time": "2017-09-16T16:19:02", "url": "https://files.pythonhosted.org/packages/ab/32/f5effba16931631289b76bbcaaaea29a5ac07ced80409cb95e45139d585a/scrapy_prometheus-0.4.4.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5fd8a51e1f138f482a3ded5710c92430", "sha256": "0c501c33ccfa4483533f18e101b3892ecdb1a2fd851d1af8a6e8e8503d9ba20f" }, "downloads": -1, "filename": "scrapy_prometheus-0.4.4.tar.gz", "has_sig": false, "md5_digest": "5fd8a51e1f138f482a3ded5710c92430", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3996, "upload_time": "2017-09-16T16:19:02", "url": "https://files.pythonhosted.org/packages/ab/32/f5effba16931631289b76bbcaaaea29a5ac07ced80409cb95e45139d585a/scrapy_prometheus-0.4.4.tar.gz" } ] }