{ "info": { "author": "Matthias Endler", "author_email": "matthias.endler@trivago.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "License :: OSI Approved :: BSD License", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: Utilities" ], "description": "![logo](contrib/assets/protector_rabbit.png)\n\n# Protector\n\nA circuit breaker for Time Series Databases like InfluxDB\n\n## Introduction\n\nThis project aims to create a protection mechanism for time series databases.\nIt prevents malicious queries that could overload the browser or the database itself.\n**Currently only InfluxDB 0.8 is supported**. But other databases could be added as well.\n\nGoals:\n\n* Prevent malicious queries to Time Series Databases (e.g. InfluxDB) that could bring it down by running out of memory.\n* Prevent the Browser from freezing because of too many incoming datapoints.\n* Provide helpful error messages for rejected queries.\n\nProtector is supposed to be **running between your dashboard solution and the database**. \nFor example, you should point Grafana to Protector instead of InfluxDB to filter the queries: \n\n![logo](contrib/assets/diagram_protector.jpeg)\n\nFor more information on why we built this tool, please read our announcement post on the trivgo techblog.\n\n\n## How it works\n\nProtector checks each query against a list of rules before it even reaches the database. \nFiltered queries will not be executed at all and an error message will be returned instead. \nRules can be specified in your `config.yaml`.\n\nYou can show a description of all available rules with `protector --show_rules`.\nHere's the current list of rules:\n\n#### Prevent negative group by statements (`negative_groupby_statement`) ####\nNegative group by statements lead to undefined behavior.\nThey can even bring down the server. That's why they are forbidden.\n\n#### Prevent drop queries (`prevent_drop`) ####\nDrop queries mean data loss. This is a risky operation that should be restricted to admin users\n\n#### Prevent delete queries (`prevent_delete`) ####\nDeleting data can be a very expensive operation. This should not be done in InfluxDB 0.8 and before.\nSee: https://influxdb.com/docs/v0.8/api/query_language.html#deleting-data-or-dropping-series\n\n#### Prevent series names that end with a dot (`series_endswith_dot`) ####\nSuch series usually indicate that the query is unfinished and \nwas executed by accident. To avoid the error, just remove the dot \nor add another word (e.g. 'my.graphite.series.' -> 'my.graphite.series')\n\n#### Prevent too many datapoints per query (`too_many_datapoints`) ####\nSuch queries can bring down the time series database\nor overload the client with too much data transferred over the wire.\n\n#### Prevent queries for short series names (`short_series_name`) ####\nThe shorter the regex for the series name, the more series names get potentially matched.\nThis is a huge performance hit for InfluxDB.\n\n#### Prevent querying for very old data (`query_old_data`) ####\nSuch queries can bring down the time series database\nbecause it needs to open and parse very old shards from disk\n\n\n## Whitelisting\n\nYou can whitelist series names in the config. Queries for series names matching one of the patterns are always executed without any previous checking. This is useful if you want to allow all queries for [Grafana annotations](http://docs.grafana.org/reference/annotations/) for example. Look at `config.yaml` for some sample patterns. \n\n## Usage\n\nYou can either run Protector as a Docker container or as a stand-alone Python application.\n\nBut first, please create a `config.yaml` which includes all your settings.\nWe've added a sample config file that should get you started.\nPlease make sure to adjust the `backend_host` and `backend_port` to point to your database\n\n### Option 1: Running as a stand-alone application\n\nYou need to have Python 2.7 installed on your server\n\n```Python\npip install protector\nprotector -c config.yaml\n```\n\n### Option 2: Running as a Docker container\n\nStart the container in the same directory where your `config.yaml` is located:\n\n```\ndocker run -it -v `pwd`:config trivago/protector\n```\n\n### Wiring up\n\nAfter you've started Protector, point all your user-facing endpoints (e.g. Grafana) to Protector instead of InfluxDB. \nThat should do the trick.\n\n\n## Commandline options\n\nYou can overwrite the following settings from the command-line:\n\n```\nusage: protector [-h] [--host HOST] [--port PORT]\n [--backend_host BACKEND_HOST] [--backend_port BACKEND_PORT]\n [-c CONFIGFILE] [-v] [--show_rules] [-f] [--version]\n [{start,stop,status,restart}]\n\nProtector - A circruit breaker for Time Series Databases\n\npositional arguments:\n {start,stop,status,restart}\n One of the following options:\n start: Start the daemon (default)\n stop: Stop the daemon\n status: Show current status\n restart: Restart the daemon\n\noptional arguments:\n -h, --help show this help message and exit\n --host HOST Hostname to bind to (default: localhost)\n --port PORT Port to bind to (default: 8888)\n --backend_host BACKEND_HOST\n Hostname of Time Series Database (default: localhost)\n --backend_port BACKEND_PORT\n Port of Time Series Database (default: 8086)\n -c CONFIGFILE, --configfile CONFIGFILE\n Configfile path (default: None)\n -v, --verbose Set verbosity level. Increase verbosity by adding a v:\n -v -vv -vvv (default: 0)\n --show_rules Show a list of available rules and quit\n -f, --foreground Run in foreground. Don't daemonize on start.\n --version Show version\n```\n\n\n## Performance\n\nProtector can easily check hundreds of queries per second.\nWe included a load test that you can run yourself. Look into `docker-compose-integration.yml` for more info.\n\n\n## Credits\n\nLogo created by [Bradley Ashburn](https://thenounproject.com/bashburn/). Thanks for this!\n\n\n## License\n\nBSD-3", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/trivago/protector", "keywords": "influxdb proxy graphite circuit-breaker", "license": "BSD", "maintainer": null, "maintainer_email": null, "name": "protector", "package_url": "https://pypi.org/project/protector/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/protector/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/trivago/protector" }, "release_url": "https://pypi.org/project/protector/0.6.0/", "requires_dist": null, "requires_python": null, "summary": "A circuit breaker for Time series databases like InfluxDB that prevents expensive queries", "version": "0.6.0" }, "last_serial": 1972242, "releases": { "0.6.0": [ { "comment_text": "", "digests": { "md5": "5a7e9623d1e09d4176f575fbc50e22a4", "sha256": "9d7a5bf3fa0f292fcd6bafbffe41853ac47e065767fe8dc02b8b87b624c6887b" }, "downloads": -1, "filename": "protector-0.6.0.tar.gz", "has_sig": false, "md5_digest": "5a7e9623d1e09d4176f575fbc50e22a4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32389, "upload_time": "2016-02-23T18:52:32", "url": "https://files.pythonhosted.org/packages/ab/30/ed4fd343d1251e3ebfe3af41278289b6e4102b5534307f9cd8facba9f204/protector-0.6.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5a7e9623d1e09d4176f575fbc50e22a4", "sha256": "9d7a5bf3fa0f292fcd6bafbffe41853ac47e065767fe8dc02b8b87b624c6887b" }, "downloads": -1, "filename": "protector-0.6.0.tar.gz", "has_sig": false, "md5_digest": "5a7e9623d1e09d4176f575fbc50e22a4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 32389, "upload_time": "2016-02-23T18:52:32", "url": "https://files.pythonhosted.org/packages/ab/30/ed4fd343d1251e3ebfe3af41278289b6e4102b5534307f9cd8facba9f204/protector-0.6.0.tar.gz" } ] }