{ "info": { "author": "David Neary", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Topic :: Internet :: WWW/HTTP", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: System :: Logging" ], "description": "=====================\npython-logstash-async\n=====================\n\nPython logging handler for asynchronous event processing and transport to Logstash.\nhttp://logstash.net/\n\n\nAbout\n-----\n\nThis Python logging handler is a fork of\nhttps://github.com/eht16/python-logstash-async\n\nIt has the following features:\n\n * Asynchronous transport of log events\n * Transport of events via TCP and UDP\n * TCP transport optionally SSL-encrypted\n * Special formatter ready to be used in Django projects\n\nCompared to https://github.com/eht16/python-logstash-async, this fork does not try to preserve log statements if\nno connection can be made to logstash.\n\n\nAsynchronous processing\n^^^^^^^^^^^^^^^^^^^^^^^\n\nUnlike the original ``python-logstash``, this handler will try to\nhandle log events as fast as possible so that the sending program\ncode can continue with its primary job.\nIn other words, for web applications or web services it is important\nto not slow down request times due to logging delays, e.g. waiting\nfor network timeouts to the Logstash server or similar.\n\nSo this handler will accept log events and pass them for further\nprocessing to a separate worker thread which will try to send\nthe events to the configured Logstash server asynchronously.\n\nThe sending intervals and timeouts can be configured in the\n``logstash_async.constants`` module by the corresponding\nmodule-level constants, see below for details.\n\n\nInstallation\n------------\n\nUsing pip::\n\n pip install git+https://github.com/koendeschacht/python-logstash-async.git\n\nUsage\n-----\n\n`AsynchronousLogstashHandler` is a custom logging handler which\nsends Logstash messages using UDP and TCP. For example:\n\n.. code:: python\n\n import logging\n import sys\n from logstash_async.handler import AsynchronousLogstashHandler\n\n host = 'localhost'\n port = 5959\n\n test_logger = logging.getLogger('python-logstash-logger')\n test_logger.setLevel(logging.INFO)\n test_logger.addHandler(AsynchronousLogstashHandler(\n host, port)))\n # test_logger.addHandler(AsynchronousLogstashHandler(host, port))\n\n test_logger.error('python-logstash-async: test logstash error message.')\n test_logger.info('python-logstash-async: test logstash info message.')\n test_logger.warning('python-logstash-async: test logstash warning message.')\n\n # add extra field to logstash message\n extra = {\n 'test_string': 'python version: ' + repr(sys.version_info),\n 'test_boolean': True,\n 'test_dict': {'a': 1, 'b': 'c'},\n 'test_float': 1.23,\n 'test_integer': 123,\n 'test_list': [1, 2, '3'],\n }\n test_logger.info('python-logstash: test extra fields', extra=extra)\n\nWhen using the ``extra`` field make sure you don't use reserved names.\nFrom `Python documentation `_::\n\n \"The keys in the dictionary passed in extra should not clash\n with the keys used by the logging system.\n (See the `Formatter `_ documentation\n for more information on which keys are used by the logging system.)\"\n\nYou can also specify an additional extra dictionary in the logging configuration with static\nvalues like the application name, environment, etc. These values will be merged with any\nextra dictionary items passed in the logging call into the configured extra prefix.\n\n\nUsage with Django\n-----------------\n\nModify your ``settings.py`` to integrate ``python-logstash-async`` with Django's logging:\n\n.. code:: python\n\n LOGGING = {\n ...\n 'formatters': {\n ...\n 'logstash': {\n '()': 'logstash_async.formatter.DjangoLogstashFormatter',\n 'message_type': 'python-logstash',\n 'fqdn': False, # Fully qualified domain name. Default value: false.\n 'extra_prefix': 'dev', #\n 'extra': {\n 'application': PROJECT_APP,\n 'project_path': PROJECT_APP_PATH,\n 'environment': 'production'\n }\n },\n },\n 'handlers': {\n ...\n 'logstash': {\n 'level': 'DEBUG',\n 'class': 'logstash_async.handler.AsynchronousLogstashHandler',\n 'transport': 'logstash_async.transport.TcpTransport',\n 'host': 'logstash.host.tld',\n 'port': 5959,\n 'ssl_enable': True,\n 'ssl_verify': True,\n 'ca_certs': 'etc/ssl/certs/logstash_ca.crt',\n 'certfile': '/etc/ssl/certs/logstash.crt',\n 'keyfile': '/etc/ssl/private/logstash.key',\n },\n },\n 'loggers': {\n 'django.request': {\n 'handlers': ['logstash'],\n 'level': 'DEBUG',\n 'propagate': True,\n },\n },\n ...\n }\n\nThis would result in a Logstash event like the following\n(note: to some extend dependent of your Logstash configuration):\n\n.. code:: json\n\n {\n \"@timestamp\": \"2016-10-23T15:11:16.853Z\",\n \"@version\": \"1\",\n \"extra\": {\n \"application\": \"django_example\",\n \"django_version\": \"1.10.2\",\n \"environment\": \"production\",\n \"func_name\": \"get_response\",\n \"interpreter\": \"/home/enrico/example/venv/bin/python\",\n \"interpreter_version\": \"2.7.12\",\n \"line\": 152,\n \"logger_name\": \"django.request\",\n \"path\": \"/home/enrico/example/venv/lib/python2.7/site-packages/django/core/handlers/base.py\",\n \"process_name\": \"MainProcess\",\n \"project_path\": \"/home/enrico/example/app\",\n \"req_host\": \"localhost\",\n \"req_method\": \"GET\",\n \"req_referer\": \"\",\n \"req_remote_address\": \"127.0.0.1\",\n \"req_uri\": \"http://localhost/hosts/nonexistent/\",\n \"req_user\": \"enrico\",\n \"req_useragent\": \"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1\",\n \"request\": \"\",\n \"status_code\": 404,\n \"thread_name\": \"Thread-1\"\n },\n \"host\": \"my.host.tld\",\n \"level\": \"WARNING\",\n \"logsource\": \"endor.l8failed.net\",\n \"message\": \"Not Found: /hosts/nonexistent/\",\n \"pid\": 23605,\n \"port\": 56170,\n \"program\": \"manage.py\",\n \"type\": \"python-logstash\"\n }\n\n\nConfiguration\n-------------\n\nOptions for configuring the log handler\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nhost\n The host of the Logstash server (no default)\n\nport\n The port of the Logstash server (default 5959)\n\ntransport\n Callable or path to a compatible transport class\n (default: 'logstash_async.transport.TcpTransport').\n\n You can specify your own transport class, e.g. to implement\n a transport via Redis or the Beats protocol.\n If you support pass a string, it should be a path to a\n class which can be imported.\n If you pass anything else, it should be an object of a class\n with a similar interface as `logstash_async.transport.TcpTransport`.\n Especially it should provide a `close()` and a `send()` method.\n\nssl_enable\n Should SSL be enabled for the connection? (default: False)\n Only used for `logstash_async.transport.TcpTransport`.\n\nssl_verify\n Should the server's SSL certificate be verified? (default: True)\n Only used for `logstash_async.transport.TcpTransport`.\n\nkeyfile\n The path to client side SSL key file (default: None)\n Only used for `logstash_async.transport.TcpTransport`.\n\ncertfile\n The path to client side SSL certificate file (default: None)\n Only used for `logstash_async.transport.TcpTransport`.\n\nca_certs\n The path to the file containing recognized CA certificates\n (default: None)\n Only used for `logstash_async.transport.TcpTransport`.\n\nenable\n Flag to enable log processing (default is True, disabling\n might be handy for local testing, etc.)\n\n\nOptions for configuring the log formatter\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThe following settings are only valid for the provided formatters\n`logstash_async.handler.LogstashFormatter` and\n`logstash_async.handler.DjangoLogstashFormatter`.\n\nYou can use any other formatter by configuring Python's logging\nsystem accordingly. Any other formatter's `format()` method just\nshould return valid JSON suitable to be sent to Logstash\n(see `Example Logstash Configuration`_ below).\n\nOptions:\n\nmessage_type\n The `type` field in the message sent to Logstash\n (default: 'python-logstash')\n\ntags\n Additional tags to include in the Logstash message (default: None)\n\nfqdn\n Use the system's FQDN (fully qualified domain name) in the `host`\n field of the message sent to Logstash.\n `socket.getfqdn()` is used to retrieve the FQDN, otherwise\n `socket.gethostname()` is used for the default hostname.\n (default: False)\n\nextra_prefix\n Name of the field in the resulting message sent to Logstash where\n all additional fields are grouped into. Consider it as some sort\n of namespace for all non-standard fields in the log event.\n This field will take any items passed in as extra fields via\n the `extra` configuration option (see below) as well as any extra\n items passed in the logging call.\n\n To disable grouping of the extra items and have them on the top\n level of the log event message, simply set this option to `None`\n or the empty string.\n (default: 'extra')\n\nextra\n Dictionary with static items to be included in the message sent\n to Logstash. This dictionary will be merged with any other extra\n items passed in the logging call.\n (default: None)\n\n\nOptions for the asynchronous processing (in module logstash_async.constants)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nSOCKET_TIMEOUT\n Timeout in seconds for TCP connections (default: 5.0)\n\nQUEUE_CHECK_INTERVAL\n Interval in seconds to check the internal queue for new messages (default: 0.2)\n\nTIMEOUT_SENDING_MESSAGES_AT_TERMINATION\n When the program terminates, any remaining log messages are still send to logstash. This constant defines\n for how many seconds the handler can still send messages before terminating (default: 2.0)\n\n\nExample Logstash Configuration\n------------------------------\n\nExample ``logstash.conf`` for unencrypted TCP transport::\n\n input {\n tcp {\n host => \"127.0.0.1\"\n port => 5959\n mode => server\n codec => json\n }\n }\n\n\nExample ``logstash.conf`` for SSL-encrypted TCP transport::\n\n input {\n tcp {\n host => \"127.0.0.1\"\n port => 5958\n mode => server\n codec => json\n\n ssl_enable => true\n ssl_verify => true\n ssl_extra_chain_certs => [\"/etc/ssl/certs/logstash_ca.crt\"]\n ssl_cert => \"/etc/ssl/certs/logstash.crt\"\n ssl_key => \"/etc/ssl/private/logstash.key\"\n }\n }\n\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/davidwneary/python-logstash-async", "keywords": "logging logstash asynchronous", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "python-logstash-async-dwn", "package_url": "https://pypi.org/project/python-logstash-async-dwn/", "platform": "", "project_url": "https://pypi.org/project/python-logstash-async-dwn/", "project_urls": { "Homepage": "https://github.com/davidwneary/python-logstash-async" }, "release_url": "https://pypi.org/project/python-logstash-async-dwn/1.0.0/", "requires_dist": [ "six" ], "requires_python": "", "summary": "Asynchronous Python logging handler for Logstash.", "version": "1.0.0" }, "last_serial": 3707731, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "1f9e5b295c28d983b9bdd5b240be92cf", "sha256": "135eb9b05192de9befe009f4c9e19680e5ef52c438ec66559267cad06a6e7402" }, "downloads": -1, "filename": "python_logstash_async_dwn-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "1f9e5b295c28d983b9bdd5b240be92cf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18643, "upload_time": "2018-03-26T19:16:29", "url": "https://files.pythonhosted.org/packages/01/a0/320f237f7a83439c3c460e447e890a3f8368ce05a4f83df94d53ff49cef4/python_logstash_async_dwn-1.0.0-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1f9e5b295c28d983b9bdd5b240be92cf", "sha256": "135eb9b05192de9befe009f4c9e19680e5ef52c438ec66559267cad06a6e7402" }, "downloads": -1, "filename": "python_logstash_async_dwn-1.0.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "1f9e5b295c28d983b9bdd5b240be92cf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 18643, "upload_time": "2018-03-26T19:16:29", "url": "https://files.pythonhosted.org/packages/01/a0/320f237f7a83439c3c460e447e890a3f8368ce05a4f83df94d53ff49cef4/python_logstash_async_dwn-1.0.0-py2.py3-none-any.whl" } ] }