{
    "info": {
        "author": "Andy Altepeter",
        "author_email": "aaltepet@bethel.edu",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 5 - Production/Stable",
            "Environment :: Web Environment",
            "Framework :: Zope2",
            "Intended Audience :: Developers",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: GNU General Public License (GPL)",
            "Programming Language :: Python",
            "Topic :: Internet :: Proxy Servers",
            "Topic :: Internet :: WWW/HTTP :: Site Management",
            "Topic :: System :: Monitoring"
        ],
        "description": "bethel.clustermgmt\n==================\n\n.. contents:: Table of Contents\n\n\nIntroduction\n------------\n\nThis package contains support for managing and monitoring nodes \nin a cluster. When deploying changes to a zope cluster, it is necessary to\nproceed linearly across all nodes.  Each node should be taken\nout of service prior to any service disruption.  Load balancers\ntypically use a configurable http health-check, and if that health\ncheck fails enough times in a certain window, the node is taken\nout of service.  [ this is how varnish works ].  Before deploying\nchanges, we simulate a service disruption on the node, causing the\nload balancer to take it out of service.\n\nThis package contains a \"health status\" object which the load\nbalancers call for health checks.  We can inform the health status \nobject that a node is to be taken out of service.  It will then\nreport the node as down (returning an error for the health check),\nand the load balancer will take it out of service.\n\nBecause a load balancer may not send enough information to the backend zope\nnode to enable it to effectively determine which node it is (sounds odd, right?),\nnodes need to be manually entered via the ZMI manage screen.  On the same \nscreen, these nodes can be marked as offline.\n\n\nInstallation\n------------\n\nAdd bethel.clustermgmt to the eggs and zcml lists in the [instance] part of\nbuildout.cfg, then rerun buildout.\n\nThis package uses silva.core functionality to register itself with the zope \ninfrastructure.  As such it is listed as an extension in the Silva extension \nservice.  It does not need activation in order to be used.\n\nA 'Cluster Health Reporter' can now be found in the 'add' list in the ZMI.\n\n\nConfiguration\n-------------\n\nThe management screen for a cluster health reporter has two sections.  The\nfirst is the list of nodes, and the second provides an interface for taking\nnodes offline.\n\n\nList of Nodes\n~~~~~~~~~~~~~\n\nEnter the list of nodes in the cluster, one per line.  This does not need to\nbe the fqdn of the node, but each node does need a unique entry.\n\n\nOffline Nodes\n~~~~~~~~~~~~~\n\nThe list of nodes is represented here with checkboxes.  A node is out of\noffline (out of service) if it's box is checked.  To manually change\nthe service status of an node (putting it online, taking it offline), check or\nuncheck the box for that node and click \"Save Offline Nodes\".\n\n\nUse for Monitoring\n------------------\n\nThe load balancer should be configured to query the health status object.\nIf the zope node fails, the health status check will return a system error, or\nreturn no response at all (hang).  The load balancer will then automatically\ntake the node out of service.\n\nUpon recovery the health status checks will succeed, and the load balancer\nwill automatically bring the node back into service.\n\nLoad Balancer configuration (varnish)\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nConfiguring Varnish as a load balancer, and leveraging this health reporter is\neasy.  Let's assume the following:\n\n1. there are two nodes in your cluster, node1.example.com:8080 and\n   node2.example.com:8080\n\n2. the cluster health reporter is located at /health\n\nAdd a director for these two nodes in the varnish VCL file::\n\n  director zope random {\n    {\n      .backend = {\n        .host = \"node1.example.com\";\n        .port = \"8080\";\n        .first_byte_timeout = 30s;\n      }\n      .weight = 1;\n    }\n    {\n      .backend = {\n        .host = \"node2.example.com\";\n        .port = \"8080\";\n        .first_byte_timeout = 30s;\n      }\n      .weight = 1;\n    }\n  }\n\nA health check is called a \"probe\" in VCL.  Adding a probe to each backend,\nthe VCL now looks like::\n\n  director silva23 random {\n  {\n    .backend = {\n      .host = \"node1.example.com\";\n      .port = \"8080\";\n      .first_byte_timeout = 30s;\n      .probe = {\n\t.url = \"/health?node=node1\";\n\t.timeout = 0.3 s;\n\t.window = 8;\n\t.threshold = 3;\n\t.initial = 3;\n      }\n    }\n    .weight = 1;\n  }\n  {\n    .backend = {\n      .host = \"node2.example.com\";\n      .port = \"8080\";\n      .first_byte_timeout = 30s;\n      .probe = {\n\t.url = \"/health?node=node2\";\n\t.timeout = 0.3 s;\n\t.window = 8;\n\t.threshold = 3;\n\t.initial = 3;\n      }\n    }\n    .weight = 1;\n  }\n  }\n\nSee the `varnish configuration <https://www.varnish-cache.org/docs/3.0/>`_ for more information.\n\n\nUse for Deployments\n-------------------\n\nUsing a health status object, rather than an arbitrary web page, for the \nload balancers health check makes it useful for automatic service removal\nduring system deployments.\n\nThe node can me marked as 'out of service' via the ZMI, or using REST.\nThe REST approach is useful for automated deployment scripts.\n\n\nAutomated deployments\n---------------------\n\n\nREST API\n~~~~~~~~\n\nThis object also responds to REST requests to adjust the service status.\nUsing this method, automated deployment scripts (e.g. using fabric)\ncan take nodes out of service before deploying updates.\n\nAccess to the REST API calls are protected using the 'bethel.clustermgmt.rest'\npermission.  To access the api calls, the request needs to be authenticated as\na manager, or as a user in a role granting this permission.\n\nThe REST api has two methods.\n\n1) Get the status of all nodes (HTTP GET)::\n\n   /path/to/health/++rest++nodestatus\n\n   Returns a json-formatted dictionary of all nodes, and their status (either\n   online or offline), like this::\n\n   {nodeA: {status: offline}, nodeB: {status: online}}\n\n2) Alter the status of one or more nodes (HTTP POST)::\n\n   /path/to/health/++rest++setstatus\n\n   POST data instructs the reporter on the new status for the given nodes.  Due to\n   infrae.rest's lack of support for accepting json payloads, the json input is\n   passed in via a POST parameter named \"change\".  See the unittests for more info.\n\n   The input format is the same the the output from ++rest++nodestatus.\n\n\nUse in Fabric\n~~~~~~~~~~~~~\n\nA simple python function can trigger a status change for a node.  This in turn\ncan be converted into a fabric task.  The following is the fabric task we use\nat Bethel for changing the service status of a node::\n\n  env.roledefs = {\n    'prod': ['node1.example.com', 'node2.example.com'],\n    'dev': ['test-node.example.com']\n  }\n  env.buildout_root = \"/home/zope/silva23/buildout\"\n\n  def alter_service_status(newstatus):\n    #alter the service status of a zope node,\n    #either putting online or offline\n    host = env['host_string']\n    node = host.split('.')[0]\n    url = 'http://%s:8080/silva/varnish_node_is_up/++rest++setstatus'%host\n    query = {'change': json.dumps({node: {'status': newstatus}}),\n             'skip-bethel-auth': 1}\n    req = urllib2.Request(url, query)\n    authh = \"Basic \" + base64.encodestring('%s:%s'%rest_creds)[:-1]\n    req.add_header(\"Authorization\", authh)\n    response = urllib2.urlopen(req,\n                               urllib.urlencode(query))\n    back = ''.join(response.readlines())\n    return 'OK'\n\nThe username and password are read from a protected file when the fabfile is\nloaded.\n\nThis task in turn can be used as a component of a larger automated deployment\ntask (this is the rest of of Bethel's fabfile)::\n\n  def buildout():\n    with prefix(\"export HOME=/home/zope/\"):\n        with cd(env.buildout_root):\n            sudo(\"hg --debug pull -u\"%env, user=\"zope\")\n            sudo(\"./bin/buildout\"%env, user=\"zope\")\n\n  def restart_apache():\n    #using the sudo command does not work; it issues the following:\n    # sudo -S -p 'sudo password:'  /bin/bash -l -c \"/etc/init.d/httpd restart\"\n    # which runs a shell executing the command in quotes.  Ross was not\n    # able to configure sudo to allow multiple httpd options with\n    # one line, but suggested the run command instead.\n    #sudo(\"/etc/init.d/httpd restart\")\n    run(\"sudo /etc/init.d/httpd restart\")\n\n  def push_buildout(apache_restart=True):\n    if type(apache_restart) in StringTypes:\n        apache_restart = (apache_restart == 'True')\n\n    change_status = False\n    if env.host_string in env.roledefs['prod']:\n        change_status = True\n\n    #take out of service, it takes less time to take out of service than \n    #  it does to put back into service\n    if change_status:\n        puts(\"taking offline; sleeping 20 seconds\")\n        alter_service_status('offline')\n        sleep(20)\n\n    buildout()\n    if apache_restart:\n        restart_apache()\n\n    #TODO: test some urls, loading up the local ZODB cache before bringing\n    #  back in to service\n \n\n    #put back into service\n    if change_status:\n        puts(\"taking online; sleeping 30 seconds\")\n        alter_service_status('online')\n        sleep(30)\n\nAdding fabric to your buildout is detailed here: `<http://www.vlent.nl/weblog/2010/09/27/fabric-easy-deployment/>`_\n\nThis fabfile is located in the buildout root.  Running an automated deployment\nof our production environment is simple::\n\n  ./bin/fab -R prod push_buildout\n\nWhen using mod_wsgi to serve Zope, a restart of apache is required for change\nto take effect.  If for any reason you'd want to push buildout but not restart\napache, pass in False to the restart_apache per-task argument::\n\n  ./bin/fab -R prod push_buildout:restart_apache=False\n\nThe combination of fabric and bethel.clustermgmt has decreased deployment time\nconsiderably.  It is now one command run in the background, whereas before it\nwas a 5-10 minute long repetitive rinse/repeat cycle for each node in the cluster.\n\n\n\nbethel.clustermgmt changlog\n===========================\n\nbethel.clustermgmt 1.0 (2012-04-30)\n-----------------------------------\n\n* first release of cluster mgmt utilities",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://pypi.python.org/pypi/bethel.clustermgmt",
        "keywords": "python zope infrae varnish fabric",
        "license": "GPL",
        "maintainer": null,
        "maintainer_email": null,
        "name": "bethel.clustermgmt",
        "package_url": "https://pypi.org/project/bethel.clustermgmt/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/bethel.clustermgmt/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "http://pypi.python.org/pypi/bethel.clustermgmt"
        },
        "release_url": "https://pypi.org/project/bethel.clustermgmt/1.0/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Zope Cluster Management facilities",
        "version": "1.0"
    },
    "last_serial": 786818,
    "releases": {
        "1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c9cb4666ab2207e52eb49639d6ac7fd3",
                    "sha256": "eab8999dcb46ac395468be9b03dac3b18e2bfa16f03441b54b51afd18dd25e88"
                },
                "downloads": -1,
                "filename": "bethel.clustermgmt-1.0-py2.6.egg",
                "has_sig": false,
                "md5_digest": "c9cb4666ab2207e52eb49639d6ac7fd3",
                "packagetype": "bdist_egg",
                "python_version": "2.6",
                "requires_python": null,
                "size": 23972,
                "upload_time": "2012-05-29T18:53:05",
                "url": "https://files.pythonhosted.org/packages/01/91/441aa1b7458c4691791cd3d2ad5466ec85d26cc8a1a18438e0954d8554cd/bethel.clustermgmt-1.0-py2.6.egg"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ff6f809bc5bed8112d94cb9733190e7b",
                    "sha256": "c9d3c22aacdb48424b534fa24a8b87490da36eea3c0e003eb4f3614ca9ac525d"
                },
                "downloads": -1,
                "filename": "bethel.clustermgmt-1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ff6f809bc5bed8112d94cb9733190e7b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 19291,
                "upload_time": "2012-05-29T18:53:04",
                "url": "https://files.pythonhosted.org/packages/63/8d/1fd6c82dfeacf48abb3203b75a4165dc77874dfb38b7c3af97360524ccfb/bethel.clustermgmt-1.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "c9cb4666ab2207e52eb49639d6ac7fd3",
                "sha256": "eab8999dcb46ac395468be9b03dac3b18e2bfa16f03441b54b51afd18dd25e88"
            },
            "downloads": -1,
            "filename": "bethel.clustermgmt-1.0-py2.6.egg",
            "has_sig": false,
            "md5_digest": "c9cb4666ab2207e52eb49639d6ac7fd3",
            "packagetype": "bdist_egg",
            "python_version": "2.6",
            "requires_python": null,
            "size": 23972,
            "upload_time": "2012-05-29T18:53:05",
            "url": "https://files.pythonhosted.org/packages/01/91/441aa1b7458c4691791cd3d2ad5466ec85d26cc8a1a18438e0954d8554cd/bethel.clustermgmt-1.0-py2.6.egg"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "ff6f809bc5bed8112d94cb9733190e7b",
                "sha256": "c9d3c22aacdb48424b534fa24a8b87490da36eea3c0e003eb4f3614ca9ac525d"
            },
            "downloads": -1,
            "filename": "bethel.clustermgmt-1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "ff6f809bc5bed8112d94cb9733190e7b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 19291,
            "upload_time": "2012-05-29T18:53:04",
            "url": "https://files.pythonhosted.org/packages/63/8d/1fd6c82dfeacf48abb3203b75a4165dc77874dfb38b7c3af97360524ccfb/bethel.clustermgmt-1.0.tar.gz"
        }
    ]
}