{ "info": { "author": "tumbler", "author_email": "zimbler@gmail.com", "bugtrack_url": null, "classifiers": [], "description": "logcollect\n==========\n\nPython library for centralized log collecting\n\nProvides simple configuration for collecting python logs to ELK stack\nvia RabbitMQ.\n\nSupported message flow is following:\n\n::\n\n\n python.logging\n ||\n \\/\n logcollect\n ||\n \\/\n RabbitMQ \n ||\n \\/\n Logstash\n ||\n \\/\n ElasticSearch\n ||\n \\/\n Kibana\n\nMechanics\n---------\n\nNative logging\n~~~~~~~~~~~~~~\n\n``logcollect.boot.default_config`` ensures that root logger has\ncorrectly configured amqp handler.\n\nDjango\n~~~~~~\n\n``logcollect.boot.django_dict_config`` modifies\n``django.conf.settings.LOGGING`` to ensure correct amqp handler for root\nlogger. It should be called in settings module after LOGGING definition.\n\nCelery\n~~~~~~\n\n``logcollect.boot.celery_config`` adds signal handler for\n``worker_process_init`` signal, and after that adds amqp handler to\n``task_logger`` base handler. If necessary, root logger can be also\nattached to amqp handler.\n\nTips for configuration\n----------------------\n\nLogstash\n~~~~~~~~\n\n.. code:: ruby\n\n\n input {\n rabbitmq {\n exchange => \"logstash\"\n queue => \"logstash\"\n host => \"rabbitmq-host\"\n type => \"amqp\"\n durable => true\n codec => \"json\"\n }\n }\n output {\n elasticsearch { host => localhost }\n stdout { codec => rubydebug }\n }\n\nlogcollect\n~~~~~~~~~~\n\nAll boot helpers have same parameters:\n\n- broker\\_uri - celery-style RabbitMQ connection string, i.e.\n ``amqp://guest@localhost//vhost``\n- exchange, routing\\_key - message routing info for RabbitMQ\n- durable - message delivery mode\n- level - handler loglevel\n- activity\\_identity - dict with \"process type info\"\n\nActivity Identity\n~~~~~~~~~~~~~~~~~\n\nAssuming we deployed two projects on same host: \"github\" and \"jenkins\".\nBoth have web backends and background workers. Activity identity helps\nto identify messages from these workers:\n\n+-----------+--------------+-----------------------------------------------------------+\n| Project | Worker | Activity identity |\n+===========+==============+===========================================================+\n| github | backend | ``{\"project\": \"github\", \"application\": \"backend\"}`` |\n+-----------+--------------+-----------------------------------------------------------+\n| jenkins | background | ``{\"project\": \"jenkins\", \"application\": \"background\"}`` |\n+-----------+--------------+-----------------------------------------------------------+\n\n``loggername`` could be used for separating different parts of code\nwithin a worker. Hostnames and process PIDs are added automatically.\n\nCorrelation ID\n~~~~~~~~~~~~~~\n\nNot supported yet, but idea is marking log messages about same object\nwith ID information about this object.\n\nExamples\n--------\n\nNative python logging\n~~~~~~~~~~~~~~~~~~~~~\n\n.. code:: sh\n\n python test_native/native_logging.py\n\nDjango\n~~~~~~\n\n.. code:: sh\n\n python test_django/manage.py test_log\n\nCelery\n~~~~~~\n\nFirst, start worker:\n\n.. code:: sh\n\n celery worker -A test_celery.app.celery\n\nThen send a task to that worker:\n\n.. code:: sh\n\n python test_celery/send_task.py\n\nRelated works\n-------------\n\nAMQPHandler and AMQPLogstashFormatter are copied from\n`python-logstash `__.\n\nSee also: \\* `RabbitMQ `__\n\\* `Logstash `__ \\*\n`Kibana `__ \\*\n`Django `__ \\*\n`Celery `__", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://github.com/rutube/logcollect/", "keywords": null, "license": "Beerware", "maintainer": null, "maintainer_email": null, "name": "logcollect", "package_url": "https://pypi.org/project/logcollect/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/logcollect/", "project_urls": { "Download": "UNKNOWN", "Homepage": "http://github.com/rutube/logcollect/" }, "release_url": "https://pypi.org/project/logcollect/0.14.0/", "requires_dist": null, "requires_python": null, "summary": "Helper for collecting logs to ELK stack via RabbitMQ", "version": "0.14.0" }, "last_serial": 2152769, "releases": { "0.10.0": [ { "comment_text": "", "digests": { "md5": "fbcb58eb3648f54f7d0d12740d031420", "sha256": "f507b710a46c35172390d7a3914eb9dd145812f42b2dc5d09b7b1ea6162b68b4" }, "downloads": -1, "filename": "logcollect-0.10.0.tar.gz", "has_sig": false, "md5_digest": "fbcb58eb3648f54f7d0d12740d031420", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5778, "upload_time": "2015-08-19T12:01:09", "url": "https://files.pythonhosted.org/packages/f7/20/814dc578fd46cf03ab89d07201c0d861b4fc4c52e4d73e9afb12921c23bb/logcollect-0.10.0.tar.gz" } ], "0.11.0": [ { "comment_text": "", "digests": { "md5": "0d4c74408d943bca9ee6b04b8a0f0b18", "sha256": "ecf7c7320d267717833cd6d574d71289d3bf860ff8029ae2060f4651eab7c763" }, "downloads": -1, "filename": "logcollect-0.11.0.tar.gz", "has_sig": false, "md5_digest": "0d4c74408d943bca9ee6b04b8a0f0b18", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6934, "upload_time": "2015-08-21T11:41:55", "url": "https://files.pythonhosted.org/packages/0e/dc/293fd22d4e9be6fccb331257f328692f21ba2bd289c1bd5dac3aa3813cc2/logcollect-0.11.0.tar.gz" } ], "0.12.0": [ { "comment_text": "", "digests": { "md5": "b22457929aae61a165e8691c618cf797", "sha256": "e971fde05ad530933c49b57b6995be2e27755602df343634bb99fab719fe09d6" }, "downloads": -1, "filename": "logcollect-0.12.0.tar.gz", "has_sig": false, "md5_digest": "b22457929aae61a165e8691c618cf797", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7012, "upload_time": "2015-10-05T14:10:41", "url": "https://files.pythonhosted.org/packages/80/68/7f595b302bddab89dc02828193eb5773ca68c1db95dbcaca500e2bcfec5e/logcollect-0.12.0.tar.gz" } ], "0.13.0": [ { "comment_text": "", "digests": { "md5": "7dba30982899680914541be2c8faa46f", "sha256": "b70ce0f12460e3c32617831bb4ebb6123fcb0de25ed744e46b74d514288650bd" }, "downloads": -1, "filename": "logcollect-0.13.0.tar.gz", "has_sig": false, "md5_digest": "7dba30982899680914541be2c8faa46f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7115, "upload_time": "2016-03-15T12:55:37", "url": "https://files.pythonhosted.org/packages/fe/ba/928ab6fadeffe5dd32cf85767811c20b76d2ad6ef48ffcfaa35be4c14859/logcollect-0.13.0.tar.gz" } ], "0.13.1": [ { "comment_text": "", "digests": { "md5": "c5c04fc2512fab3b5e99cda07d8b502f", "sha256": "aff1b58ceb6b3235869c9da73180efcc0df66758f7ced5b65754b3556be2c336" }, "downloads": -1, "filename": "logcollect-0.13.1.tar.gz", "has_sig": false, "md5_digest": "c5c04fc2512fab3b5e99cda07d8b502f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7099, "upload_time": "2016-04-08T12:04:10", "url": "https://files.pythonhosted.org/packages/ce/33/13b770036334b853197602121c10727c78ab7f3b1f034817ea5f4b0ad029/logcollect-0.13.1.tar.gz" } ], "0.14.0": [ { "comment_text": "", "digests": { "md5": "efbd52b98144445155939d666e7a50c4", "sha256": "cf61aa098cc10ffadd65cc49879cd5de6bfd308197e094b4fa9001048dc259f0" }, "downloads": -1, "filename": "logcollect-0.14.0.tar.gz", "has_sig": false, "md5_digest": "efbd52b98144445155939d666e7a50c4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7125, "upload_time": "2016-06-06T10:30:16", "url": "https://files.pythonhosted.org/packages/b6/f8/8d50c5114611fc2d60492f533a7c95ff5ca86e326904d7683632492aed49/logcollect-0.14.0.tar.gz" } ], "0.9.0": [ { "comment_text": "", "digests": { "md5": "3bd4c506476d20b6227d4f8cbcd282fa", "sha256": "e5165593106ec1b52edf7cb434fe89a8b3b71286d8abf95d4e839f4a9d7d1db2" }, "downloads": -1, "filename": "logcollect-0.9.0.tar.gz", "has_sig": false, "md5_digest": "3bd4c506476d20b6227d4f8cbcd282fa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5710, "upload_time": "2015-07-30T10:58:12", "url": "https://files.pythonhosted.org/packages/8d/d2/50383b7cbbc21a1736f62ca3ed560c8ea88bab14526147cd001b9af334f1/logcollect-0.9.0.tar.gz" } ], "0.9.1": [ { "comment_text": "", "digests": { "md5": "bb61eff29bb77febe66b1d070131d8eb", "sha256": "f2a801a6c44a9b877f4a58c984692b7e2453d4fd70ab4684e2eb896924c9740c" }, "downloads": -1, "filename": "logcollect-0.9.1.tar.gz", "has_sig": false, "md5_digest": "bb61eff29bb77febe66b1d070131d8eb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5480, "upload_time": "2015-08-04T10:08:53", "url": "https://files.pythonhosted.org/packages/3e/15/d451c5d55569494f3e1b08314278af10fd19ba1fda868c5bda7732c79a26/logcollect-0.9.1.tar.gz" } ], "0.9.4": [] }, "urls": [ { "comment_text": "", "digests": { "md5": "efbd52b98144445155939d666e7a50c4", "sha256": "cf61aa098cc10ffadd65cc49879cd5de6bfd308197e094b4fa9001048dc259f0" }, "downloads": -1, "filename": "logcollect-0.14.0.tar.gz", "has_sig": false, "md5_digest": "efbd52b98144445155939d666e7a50c4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7125, "upload_time": "2016-06-06T10:30:16", "url": "https://files.pythonhosted.org/packages/b6/f8/8d50c5114611fc2d60492f533a7c95ff5ca86e326904d7683632492aed49/logcollect-0.14.0.tar.gz" } ] }