{
    "info": {
        "author": "Dave Carroll",
        "author_email": "davecarrollno@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 2 - Pre-Alpha",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: MIT License",
            "Operating System :: POSIX :: Linux",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Topic :: System :: Monitoring"
        ],
        "description": "Ganglia-DynGraph\n================\n\nOverview\n--------\n\n\nDynamic graph sweeper for Ganglia. \n\nGanglia is a great trending platform but like any project it is fun to make modifications \nto suit the needs of any team or org. I had one such need a couple of years ago and that was \nto have sets of dynamic graphs that were more \"top like\" and would show me short-term metric\nresults rather than long term trending. For example, I had farms of API servers that handled\nhundreds of millions of requests per day and I wanted to know from each API server which \nof our clients were the top 5 users of those API endpoints. In this example, I needed to tally\nrequests, extract the top 5 requests by client id and graph them. Easy enough, but what if I \nwanted to know the top 5 requests every 60 seconds and those top 5 clients changed by volume? \nI needed a way to record the top 5 metrics per API node and 60 seconds later if any of those \nchanged position, tear down the metric graph, re-publish it, and expire out the metric files \nto keep the system from filling up.  Another example where I used this was to show short-term\nswapping metrics. I wanted to see what was swapped out on any given system at any given time.\nThis can be highly dynamic in nature. \n\nGanglia-DynGraph is a daemon started from rc script. It reads a configurable file where you\nset options and acts upon those accordingly to produce dynamic metrics. Behind the scenes it \nsweeps your RRDtree for configured metrics and expires (deletes) them off your system at a \nconfigurable expiration period. It also searches for configured metrics where you want the top\nn results, gathers last metric value information, ranks that and creates Ganglia report graphs\nin .json format under your Ganglia Gweb tree. \n\nI have removed some custom functions and classes that are unique to my work environment but\nhave reworked the core modules into this package so others can use them or improve upon them.\n\n\nCore Requirements:\n------------------\n- Python 2.6 or Python 2.7 (as tested)\n- Ganglia Gweb - where you store your rrds and expose them via Ganglia web\n\nInstallation\n------------\nThis package can be easily installed by using 'pip install ganglia-dyngraph' and upgraded\nusing pip install --upgrade ganglia-dyngraph accordingly.  I use CentOS 6.x as a standard\nso the included init file is specific to that and will be installed into /etc/init.d. Files\ninstalled:\n\n- /etc/init.d/DynamicGraph - the init script to start and background DynamicGraph. \n- /etc/DynamicGraph.cfg - the configuration file you must configure for your needs\n- /var/run/DynamicGraph.pid - pid file \n- /var/log/DynamicGraph.log - operations logging with helpful info\n\nUsage\n-----\nAfter installation, edit /etc/DynamicGraph.cfg.  That file has some instructions included. You\nwill want to verify your rrd tree and Ganglia graphs locations are set correctly.  You will want\nto set various global options and finally configure any metrics you wish to have dynamically managed.\n\nOnce done, as root, chkconfig --level 3 DynamicGraph on and run service DynamicGraph start to fire \nup DynamicGraph.  You will find that in your ps listing and can check logging output in \n/var/log/DynamicGraph.log for any issues. You may also want to include DynamicGraph.log in any\nlogrotate configuration you manage. \n\nDynamicGraph.cfg\n----------------\nCurrently the following configuration options exist:\n\n- GraphDir - This is where you have Ganglia configured to hold your metric .json graph files. This \n  is also where DynamicGraph will produce dynamically created reports. You will want to edit your \n  hosts or cluster configuration preferences typically under /var/lib/ganglia-web/conf to include \n  these as needed. \n\n- RRDIDR - This is the location of your rrd files. This is typically /var/lib/ganglia/rrds\n\n- logfile - You may specifiy a custom location for your DynamicGraph log file. \n\n- runTime - This numeric option controls how often the dynamic metric sweeper will run, purging any\n  metrics that have expired and creating new dynamic graphs. The default is 60 seconds. \n\n- expireTime - This is the time in seconds we consider our metrics stale. For example, if you are \n  trending swap metrics per process, set this to 60 and when then sweeper runs it will remove these\n  metrics that are older than 60 seconds. \n\n- purged_metric_count - We trend how many metrics are expired out and purged and write those to state\n  files. You can then create a Ganglia module that reads these and reports them accordingly to watch\n  for abnormal purge numbers.  \n\n- purged_error_count - Like the purged metric count, this trends any errors incurred in purging metrics.\n  Metrics may error out for a number of reasons. \n\n- numMetrics - This option sets the number of metrics to sort and rank. I am typically interested in the \n  top 5 of any metric. For example, only graph the top 5 processes swapped out by a specific system or\n  the top 5 requests to a web server or api server etc.. \n\n- Cluster Section - This section is where you configure clusters and metrics to work with. Each section\n  is titled with a [Cluster_<name of your Ganglia cluster>] section. So if you had a cluster named\n  \"WebServers\" you would configure this section header as [Cluster_WebServers].  DynamicGraph will \n  pick this up and search for the WebServer cluster in your rrd tree. \n\n  The next line is prefixed \"metrics=\" and that prefix is required. You then enter a comma-separated\n  list of metrics you wish to include as dynamic.  You do not include the .rrd suffix. For example, \n  say I created a Ganglia metric module that trended processes currently swapped out and that metric\n  was named swapping_ such that if I had processes mysql-server and data-daemon currently swapping, it\n  would produce swapping_mysql-server.rrd and swapping_data-daemon.rrd in my rrdtree. I would configure \n  this section then as metric=swapping_ since \"swapping_\" is unique. DynamicGraph will find the metrics\n  swapping_* under your node and cluster as configured.  \n\n\nLogging\n-------\nLogging is sent to /var/log/DynamicGraph.log by default. There you will find interesting status \nmessages such as:\n\n2016-12-22 17:21:45,679 - root - DEBUG Thread-140212695631616 - INFO: Metric /var/lib/ganglia/rrds/db_cluster/db1/swapping_mysql-server.rrd successfully purged\n\n2016-12-22 17:21:45,679 - root - DEBUG Thread-140212695631616 - INFO: Metric /var/lib/ganglia/rrds/db_cluster/__SummaryInfo__/swapping_mysql-server.rrd successfully purged\n\nwhich show the metric swapping_mysql-server was purged successfully from your system for the node db1. \n\n\nAdditional\n----------\nAs I mentioned earlier, I use dynamic metrics for trending the current top 5 users of a very busy API\nfarm, every 60 seconds across tens and tens of nodes and farms, globally. With this I am able to quickly spot\nwho the top consumers of our resources are at any given time. How I differentiate these clients is by a id\nthat a Ganglia module I wrote obtains on each API node. The id is unique and is how I rank the top clients. \n\nTo find the customer name that I include in the graphs vs. graphing by an id, I do a lookup in either sqlite\nor in Redis where I maintain a mapping of id to customer name. This allows me to show customer names right in \nmy dynamic graphs.  To accomplish this, I utilize several other modules and classes to handle the lookups by\ncluster type. You can really come up with many possibilities and customize this as needed.",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "https://github.com/dcarrollno/Ganglia-Modules/tree/master/Ganglia-DynGraph/ganglia-dyngraph",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/dcarrollno/Ganglia-Modules/wiki/Overview:-Ganglia-Dynamic-Metrics%3F",
        "keywords": "Ganglia dynamic graphs metrics",
        "license": "UNKNOWN",
        "maintainer": null,
        "maintainer_email": null,
        "name": "ganglia-dyngraph",
        "package_url": "https://pypi.org/project/ganglia-dyngraph/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/ganglia-dyngraph/",
        "project_urls": {
            "Download": "https://github.com/dcarrollno/Ganglia-Modules/tree/master/Ganglia-DynGraph/ganglia-dyngraph",
            "Homepage": "https://github.com/dcarrollno/Ganglia-Modules/wiki/Overview:-Ganglia-Dynamic-Metrics%3F"
        },
        "release_url": "https://pypi.org/project/ganglia-dyngraph/0.1.2/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "A daemon to dynamically create and purge Ganglia graphs.",
        "version": "0.1.2"
    },
    "last_serial": 2543382,
    "releases": {
        "0.1.1": [],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "5c4feff4cb550f59add4292f91b0d196",
                    "sha256": "39048ea08be801793cebd90f6a9f7fbb2e1313fd371c52e6c8bfcb91d7253b2a"
                },
                "downloads": -1,
                "filename": "ganglia-dyngraph-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "5c4feff4cb550f59add4292f91b0d196",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 12545,
                "upload_time": "2016-12-29T00:09:25",
                "url": "https://files.pythonhosted.org/packages/69/cb/9bfcc1da6d0a59c66903ecefc73ffac0137c4fa817fa2b55a1966fcabc1a/ganglia-dyngraph-0.1.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "5c4feff4cb550f59add4292f91b0d196",
                "sha256": "39048ea08be801793cebd90f6a9f7fbb2e1313fd371c52e6c8bfcb91d7253b2a"
            },
            "downloads": -1,
            "filename": "ganglia-dyngraph-0.1.2.tar.gz",
            "has_sig": false,
            "md5_digest": "5c4feff4cb550f59add4292f91b0d196",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 12545,
            "upload_time": "2016-12-29T00:09:25",
            "url": "https://files.pythonhosted.org/packages/69/cb/9bfcc1da6d0a59c66903ecefc73ffac0137c4fa817fa2b55a1966fcabc1a/ganglia-dyngraph-0.1.2.tar.gz"
        }
    ]
}