{ "info": { "author": "Anton Bobrov", "author_email": "baverman@gmail.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6" ], "description": "Hisser\n======\n\n|travis| |coverage| |pyver|\n\n.. |travis| image:: https://travis-ci.org/baverman/hisser.svg?branch=master\n :target: https://travis-ci.org/baverman/hisser\n\n.. |coverage| image:: https://img.shields.io/badge/coverage-100%25-brightgreen.svg\n\n.. |pyver| image:: https://img.shields.io/badge/python-3.5%2C_3.6-blue.svg\n\n\nTime series database, backend for graphite, fast alternative to carbon + whisper.\n\nFeatures:\n\n* Low disk usage (IOPS) for metric store, it depends from actual data\n volumes instead of a number of metrics (in case of whisper). Hisser\n was designed to process million of metrics.\n\n* Fast queries. Optimized query parsing and response rendering (~3x\n boost comparing with vanilla graphite-web).\n\n* Tag support.\n\n* Drop-in replacement for whisper + carbon.\n\n* 100% test coverage.\n\n\n.. contents:: **Table of Contents**\n\n\nConfiguration\n-------------\n\nDefault options and documentation for them can be read in\n`default config`_.\n\n.. _default config: hisser/defaults.py\n\nYou can create custom configuration file and use ``--config`` cli option or\nuse ``HISSER_*`` environment variables to override default values.\nFor example ``HISSER_DATA_DIR`` will set ``DATA_DIR`` configuration\nparameter.\n\n\nRun\n---\n\nSimplest way is to use official `docker image`::\n\n docker run --rm -u $(id -u):$(id -g) -p 2003:2003 -p 8080:8080 -v /path/to/data:/data baverman/graphite-hisser\n\nPort `2003` is a graphite protocol. `8080` is graphite API, you can point\ngrafana to it. In production you don't need 8080 port accessible from\nexternal network. In this case you should use separate docker network\nand map 2003 port only or use ``--network host`` and specify ``GRAPHITE_BIND=127.0.0.1:8080``\nenvvar.\n\nIMPORTANT! To use tag support with grafana you need grafana 5.x and set graphite\nversion ``1.1.x`` in storage settings.\n\nNote: for grafana you can use tiny `grafana image`_.\n\n.. _docker image: https://hub.docker.com/r/baverman/graphite-hisser/\n.. _grafana image: https://hub.docker.com/r/baverman/grafana/\n\n\nInternals\n---------\n\nHisser is a very simple metric storage. All heavy work is done by `lmdb`_.\nMetrics are organized into blocks (lmdb databases). Each block\ncontains all metrics and their data for particular amount of time. Blocks\nwith same resolution are grouped under corresponding directory:\n\nExample data layout:\n\n::\n\n data_dir/\n \u251c\u2500\u2500 300 # resolution (1 data point every 5-minute)\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1533990300.519.hdb # timestamp-of-block-start.number-of-points.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534621800.191.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534679100.48.hdb\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 blocks.state # lock file\n \u251c\u2500\u2500 60 # resolution (1 data point every minute)\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534621920.700.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534663920.320.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534683120.160.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534692720.40.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534695120.11.hdb\n \u2502\u00a0\u00a0 \u251c\u2500\u2500 1534695900.6.hdb\n \u2502\u00a0\u00a0 \u2514\u2500\u2500 blocks.state\n \u2514\u2500\u2500 metric.index # metric name and tag index\n\n\nThis layout allows to dump data from memory buffer very efficiently (whisper\nneeds one io-operation per metric and can kneel a host with several hundreds of\nmetrics).\n\nIf points in memory exceed ``BUFFER_FLUSH_SIZE`` or ``BUFFER_MAX_POINTS`` it will be\nflushed into separate block::\n\n +----------+----------+----------+\n | block1 | block2 | block3 | resolution 60\n +----------+----------+----------+\n\nFrom time to time small blocks are merged into greater one::\n\n +---------------------+----------+\n | block12 | block3 | resolution 60\n +---------------------+----------+\n\nAnd from time to time big blocks are downsampled into blocks with lower\nresolution::\n\n +---------------------+----------+\n | block12 | block3 | resolution 60\n +---------------------+----------+\n |\n v\n +----------+\n | block12' | resolution 300\n +----------+\n\nYes, it is very simple.\n\n.. _lmdb: http://www.lmdb.tech/doc/\n\n\nFAQ\n---\n\n1. But there is a better alternative to whisper. InfluxDB!\n\n Yes, InfluxDB is a way better than whisper. But is has some drawbacks\n comparing to hisser.\n\n * Requires more data space.\n * Consumes more IOPS, memory and CPU.\n * Needs manual retention configuration.\n * Slower to query.\n * Implicit metric grouping can lead to confusing graphs in grafana.\n You have to limit groups to explicit tag values or do ``group by\n $tag``.\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/baverman/hisser", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "hisser", "package_url": "https://pypi.org/project/hisser/", "platform": "", "project_url": "https://pypi.org/project/hisser/", "project_urls": { "Homepage": "https://github.com/baverman/hisser" }, "release_url": "https://pypi.org/project/hisser/0.10.1/", "requires_dist": null, "requires_python": "", "summary": "Fast TSDB backend for graphite", "version": "0.10.1" }, "last_serial": 4187088, "releases": { "0.10": [ { "comment_text": "", "digests": { "md5": "db0070897a5c4fdb0846a07ef02b00b0", "sha256": "c5a484cb74e1c0f0fc51023ab5fbb7afb47ca5083485c8c4ccc0e526b5bee0f8" }, "downloads": -1, "filename": "hisser-0.10.tar.gz", "has_sig": false, "md5_digest": "db0070897a5c4fdb0846a07ef02b00b0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18557, "upload_time": "2018-08-19T19:07:12", "url": "https://files.pythonhosted.org/packages/d3/fc/5fefdb2c3195fe5f19934456cb796c4eb01ac5de16567b0f3abd7e4bb8ce/hisser-0.10.tar.gz" } ], "0.10.1": [ { "comment_text": "", "digests": { "md5": "e72c8a62f61deee2fdc10c3998d9e2d8", "sha256": "a461fcb415a432746ec113a3277bfc5ecb33cc8450b93e88770d4ab5f11b2204" }, "downloads": -1, "filename": "hisser-0.10.1.tar.gz", "has_sig": false, "md5_digest": "e72c8a62f61deee2fdc10c3998d9e2d8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18592, "upload_time": "2018-08-20T05:05:45", "url": "https://files.pythonhosted.org/packages/74/2a/2c98798853788e432ce5f7ad7e1af7c0459b6f85286d10817a50727cf17f/hisser-0.10.1.tar.gz" } ], "0.9": [ { "comment_text": "", "digests": { "md5": "fcd14573f37bcf864c000314160bf618", "sha256": "cd3876dbb5d9fb5e138802abdfd10d706a0789535f4611077fdd236734029b76" }, "downloads": -1, "filename": "hisser-0.9.tar.gz", "has_sig": false, "md5_digest": "fcd14573f37bcf864c000314160bf618", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16397, "upload_time": "2018-08-12T20:36:39", "url": "https://files.pythonhosted.org/packages/1f/ed/d6b32e73b87a13c6410f39a6d18ce54d573840d6b995786ef9e6366848ed/hisser-0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e72c8a62f61deee2fdc10c3998d9e2d8", "sha256": "a461fcb415a432746ec113a3277bfc5ecb33cc8450b93e88770d4ab5f11b2204" }, "downloads": -1, "filename": "hisser-0.10.1.tar.gz", "has_sig": false, "md5_digest": "e72c8a62f61deee2fdc10c3998d9e2d8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 18592, "upload_time": "2018-08-20T05:05:45", "url": "https://files.pythonhosted.org/packages/74/2a/2c98798853788e432ce5f7ad7e1af7c0459b6f85286d10817a50727cf17f/hisser-0.10.1.tar.gz" } ] }