{
    "info": {
        "author": "Wouter van Bommel",
        "author_email": "wouter@vanbommelonline.nl",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "Status\n======\n[![Build Status](https://travis-ci.org/woutervb/supervisor_elastic.png)](https://travis-ci.org/woutervb/supervisor_elastic)\n\nIntroduction\n============\n\nConnecting the logging from python to elasticsearch is not always trivial, as there are multiple routes that can be taken.\nThis project uses, redis as a middleman. The main reason is, that redis is relative simple to setup in an high-availability\nsetup, so that it will not (or hardly) block the main processing loop in python.\nThe time to process the data is completely dependend on the performance of the logstash agent, that will read log records\nfrom redis and put them in elasticsearch for future processing.\nThis setup has proven to be quite stable for already a few year. But one of the things missing, was the ability to add\nlogging from supervisor to this log system. Obviously it would be possible to use filebeat to read log file(s) and then\nsend them to logstash, but as supervisor is written in python, it makes more sense to use the same route via redis.\nThis is wat this project has created.\n\nConfiguration\n=============\n\nThe configuration should take place at 2 locations, that is in supervisor, and in logstash. The supervisor configuration\nis needed to send the messages to redis, and the logstash configuration is neede to pull the data from redis.\n\nConfiguration of supervisor\n---------------------------\n\nIn supervisor we use 2 environment variables to configure the software that send the data to elasticsearch. These\nvariables are listed below.\n\n  * REDIS_LOG_URI, contains an url to the redis, following the documentation from iana in the form of redis://[:password]@host:post/database\n  * REDIS_LOG_CHANNEL, the channel used for logging, this should be the same as in the logstash configuration\n\nso an example section of the supervisor.conf would be:\n\n    [eventlistener:logging]\n    command = supervisor_elastic\n    events = PROCESS_LOG\n    environment = REDIS_LOG_URI=\"redis://dockerhost\",REDIS_LOG_CHANNEL=\"app:python\"\n\nAnother importand thing to notice, is that in each program definition inside the supervisor configuration the following\nkey's should be activated:\n\n    stdout_events_enabled=true\n    stderr_events_enabled=true\n\nThese key's will ensure that any data that is send to log file(s) is also send to the eventlistner(s) that are defined.\n\nConfiguration of logstash\n-------------------------\n\nAnd the equivalent section in logstash would be:\n\n    # Collect python logging from redis\n    input {\n      redis {\n        host => \"dockerhost\"\n        codec => json\n        data_type => \"channel\"\n        type => \"python\"\n        key  => \"app:python\"\n        tags => [\"json\"]\n      }\n    }\n\nThings that should be equal, is at least the key (in logstash) that should be equal to REDIS_LOG_CHANNEL in the supervisor\nconfiguration.\n\nNotice\n======\n\nParts of this project are inspired from: https://github.com/infoxchange/supervisor-logging, so they should be mentioned.",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/woutervb/supervisor_elastic",
        "keywords": "",
        "license": "GPLv2",
        "maintainer": "",
        "maintainer_email": "",
        "name": "supervisor-elastic",
        "package_url": "https://pypi.org/project/supervisor-elastic/",
        "platform": "",
        "project_url": "https://pypi.org/project/supervisor-elastic/",
        "project_urls": {
            "Homepage": "https://github.com/woutervb/supervisor_elastic"
        },
        "release_url": "https://pypi.org/project/supervisor-elastic/0.0.2/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Stream supervisord logs to elastic via redis",
        "version": "0.0.2"
    },
    "last_serial": 4108426,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "946b6006071a4309c6ef7a56c63c89ac",
                    "sha256": "51c45b8a9f89d52c8f91e5c3350c17b4871046d1288e034889bf28ab009dbbcf"
                },
                "downloads": -1,
                "filename": "supervisor-elastic-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "946b6006071a4309c6ef7a56c63c89ac",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3984,
                "upload_time": "2018-07-27T10:10:22",
                "url": "https://files.pythonhosted.org/packages/35/fe/bb92a3306808ea42ceec59137eafb5e3844865cc2cdc8345dc2cb538e2ef/supervisor-elastic-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "417d50199f566131ca3b60c148ebb992",
                    "sha256": "68abba5fe61bea9c1a0a34e171a5cdef5e01a0d2716e9647f69d65f6e733c159"
                },
                "downloads": -1,
                "filename": "supervisor-elastic-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "417d50199f566131ca3b60c148ebb992",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4117,
                "upload_time": "2018-07-27T11:59:54",
                "url": "https://files.pythonhosted.org/packages/17/dd/a09b51cf9860b71d1ab9d0891f30d24dfceb483b31b2e80aaee81d8558ae/supervisor-elastic-0.0.2.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "417d50199f566131ca3b60c148ebb992",
                "sha256": "68abba5fe61bea9c1a0a34e171a5cdef5e01a0d2716e9647f69d65f6e733c159"
            },
            "downloads": -1,
            "filename": "supervisor-elastic-0.0.2.tar.gz",
            "has_sig": false,
            "md5_digest": "417d50199f566131ca3b60c148ebb992",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 4117,
            "upload_time": "2018-07-27T11:59:54",
            "url": "https://files.pythonhosted.org/packages/17/dd/a09b51cf9860b71d1ab9d0891f30d24dfceb483b31b2e80aaee81d8558ae/supervisor-elastic-0.0.2.tar.gz"
        }
    ]
}