{ "info": { "author": "Stephen D. Spencer", "author_email": "stephen@revsys.com", "bugtrack_url": null, "classifiers": [], "description": "\n# jslog4kube (python/JSON logging for kubernetes pod/containers)\n\n\n## Why?\n\n * provide a JSON-to-stdout setup for python\n * provide the same JSON-to-stdout setup for [gunicorn](http://gunicorn.org)\n * because creating complex log collector configs to handle whatever\n that other person thought was a good-idea-at-the-time is for the birds.\n\nWe want to make it easier for our clients and ourselves to start new projects\nthat emit good logging.\n\n### making it go\n\nTwo environment variables configure this module:\n\n * `KUBE_META`: specifies the mount-point for the Kubernetes downward-API\n [volumes](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/) bits (default: /etc/meta)\n * `KUBE_META_ENV_PREFIX`: the textual prefix for any\n [environment variables](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)\n targetted for inclusion in this pod\u2019s log records (default: X)\n\n----\n\n\n\n---------------------------------------------------------------------------------------\n These bits have been verified to be functional under python 2.7.13, 3.5.3 and 3.6.1\n---------------------------------------------------------------------------------------\n\n----\n\n\n\n```python\n\nfrom logging.config import dictConfig\nfrom jslog4kube import LOGGING\n\ndictConfig(LOGGING)\n\n```\n\n\n#### From Django\n\nproject `settings.py`\n```python\n\nfrom jslog4kube import LOGGING\n\n```\n\n#### [gunicorn](http://gunicorn.org)\n\ngunicorn.conf\n```python\n\naccess_log_format = 'remote!%({X-Forwarded-For}i)s|method!%(m)s|url-path!%(U)s|query!%(q)s|username!%(u)s|protocol!%(H)s|status!%(s)s|response-length!%(b)s|referrer!%(f)s|user-agent!%(a)s|request-time!%(L)s'\naccesslog = '-'\nlogger_class = 'jslog4kube.GunicornLogger'\n\n```\n\ngunicorn CLI\n```bash\n\ngunicorn -c /path/to/gunicorn.conf [rest of your options here]\n```\n\n\nThis will produce the following kind of output:\n\n```json\n{\n \"asctime\": \"2017-07-12 16:07:34,624\",\n \"message\": \"Booting worker with pid: 6801\",\n \"name\": \"gunicorn.error\",\n \"created\": 1499893654.6243172,\n \"filename\": \"glogging.py\",\n \"module\": \"glogging\",\n \"funcName\": \"info\",\n \"lineno\": 247,\n \"msecs\": 624.3171691894531,\n \"pathname\": \"/home/gladiatr/.virtualenvs/json-logs-for-kube/lib/python3.6/site-packages/gunicorn/glogging.py\",\n \"process\": 6801,\n \"processName\": \"MainProcess\",\n \"relativeCreated\": 70.62673568725586,\n \"thread\": 140275859264576,\n \"threadName\": \"MainThread\",\n \"levelname\": \"INFO\",\n \"x_node_name\": \"ip-10-70-59-190.eu-central-1.compute.internal\",\n \"x_sa_name\": \"default\",\n \"x_pod_ip\": \"100.96.1.11\",\n \"build\": \"5000\",\n \"builder\": \"Stephen Spencer\",\n \"image\": \"gladiatr72/kube-demo\",\n \"version\": \"1.0.2\",\n \"app\": \"kube-demo\",\n \"env\": \"dev\",\n \"pod-template-hash\": \"2802633501\",\n \"something\": \"else\"\n}\n{\n \"asctime\": \"2017-07-12 21:08:16,354\",\n \"message\": \"in view: Chameleon\",\n \"name\": \"efk.views\",\n \"created\": 1499893696.3544216,\n \"filename\": \"views.py\",\n \"module\": \"views\",\n \"funcName\": \"Chameleon\",\n \"lineno\": 12,\n \"msecs\": 354.42161560058594,\n \"pathname\": \"/home/gladiatr/git/json-logs-for-kube/demo/efk/views.py\",\n \"process\": 6800,\n \"processName\": \"MainProcess\",\n \"relativeCreated\": 41800.73118209839,\n \"thread\": 140275726399232,\n \"threadName\": \"_0\",\n \"levelname\": \"INFO\",\n \"x_node_name\": \"ip-10-70-59-190.eu-central-1.compute.internal\",\n \"x_sa_name\": \"default\",\n \"x_pod_ip\": \"100.96.1.11\",\n \"build\": \"5000\",\n \"builder\": \"Stephen Spencer\",\n \"image\": \"gladiatr72/kube-demo\",\n \"version\": \"1.0.2\",\n \"app\": \"kube-demo\",\n \"env\": \"dev\",\n \"pod-template-hash\": \"2802633501\",\n \"something\": \"else\",\n \"additional data\": \"whee\"\n}\n{\n \"asctime\": \"2017-07-12 21:08:16,369\",\n \"message\": \"(access record)\",\n \"name\": \"gunicorn.access\",\n \"created\": 1499893696.3695881,\n \"filename\": \"glogging.py\",\n \"module\": \"glogging\",\n \"funcName\": \"access\",\n \"lineno\": 327,\n \"msecs\": 369.58813667297363,\n \"pathname\": \"/home/gladiatr/.virtualenvs/json-logs-for-kube/lib/python3.6/site-packages/gunicorn/glogging.py\",\n \"process\": 6800,\n \"processName\": \"MainProcess\",\n \"relativeCreated\": 41815.89770317078,\n \"thread\": 140275726399232,\n \"threadName\": \"_0\",\n \"levelname\": \"INFO\",\n \"x_node_name\": \"ip-10-70-59-190.eu-central-1.compute.internal\",\n \"x_sa_name\": \"default\",\n \"x_pod_ip\": \"100.96.1.11\",\n \"build\": \"5000\",\n \"builder\": \"Stephen Spencer\",\n \"image\": \"gladiatr72/kube-demo\",\n \"version\": \"1.0.2\",\n \"app\": \"kube-demo\",\n \"env\": \"dev\",\n \"pod-template-hash\": \"2802633501\",\n \"something\": \"else\",\n \"access\": {\n \"remote\": \"10.0.1.195\",\n \"method\": \"GET\",\n \"url-path\": \"/\",\n \"query\": \"\",\n \"username\": \"-\",\n \"protocol\": \"HTTP/1.0\",\n \"status\": \"200\",\n \"response-length\": \"140\",\n \"referrer\": \"-\",\n \"user-agent\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36\",\n \"request-time\": \"0.019269\"\n }\n}\n```\n\n\n## Example Kubernetes deployment:\n\n```yaml\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata: <<-- This is not the metadata you are looking for\n name: kube-demo\n labels:\n project: kube-demo\n environment: dev\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: kube-demo\n env: dev\n template:\n metadata:\n labels:\n app: kube-demo\n env: dev\n something: else\n annotations:\n build: \"5000\"\n builder: \"Stephen Spencer\"\n image: \"gladiatr72/kube-demo\"\n version: \"1.0.2\"\n\n```\n\n### Currently only `metadata.(labels|annotations)` are supported for exposure via volume.\n(_Reasons_, right?)\n\n\n```yaml\n spec:\n volumes:\n name: podinfo\n downwardAPI:\n items:\n - path: labels\n fieldRef:\n fieldPath: metadata.labels\n - path: annotations\n fieldRef:\n fieldPath: metadata.annotations\n containers:\n - name: kube-demo\n image: gladiatr72/kube-demo:1.0.2\n volumeMounts:\n -\n name: run\n mountPath: /run\n -\n name: podinfo\n readOnly: true\n mountPath: /etc/meta <<-- KUBE_META must equal this\n```\n\n### environment variable prefixes\n(or: how to avoid looking like a complete idiot when you spam your data-store\npassword into your logging system)\n\nYou can use whatever letter or sequence for the prefix as long as it gets matched\nwith the value of `KUBE_META_ENV_PREFIX`.\n\n```yaml\n env:\n - name: MEMCACHE_HOST\n value: unix:/run/memcache.sock\n - name: DJANGO_SETTINGS_MODULE\n value: \"revsys.settings.dev\"\n - name: DJANGO_FQDN\n value: kube-demo.dev.revsys.com\n - name: *X_NODE_NAME\n valueFrom:\n fieldRef:\n fieldPath: spec.nodeName\n - name: *X_POD_IP\n valueFrom:\n fieldRef:\n fieldPath: status.podIP\n - name: *X_SA_NAME\n valueFrom:\n fieldRef:\n fieldPath: spec.serviceAccountName\n - name: REDIS_PASSWORD (oh, nos!)\n valueFrom:\n secretKeyRef:\n name: redis\n key: pass\n ports:\n - containerPort: 8000\n\n * unmagical prefix set in KUBE_META_ENV_PREFIX\n\n```\n\n## The logging configuration\n\nIt\u2019s just a standard python dictionary. The most obvious thing to change\nis the handler definition.\n\n\n```python\n\nfrom jslog4kube import LOGGING\n\nLOGGING_HANDLERS = {\n 'mypackage': {\n 'handlers': ['json-stdout'],\n 'formatters': ['json'],\n 'propagate\u2019: False,\n 'level\u2019: 'ERROR',\n }\n}\n\nLOGGING['handlers'].update(LOGGING_HANDLERS)\n\n```\n\n## Setup your python/django apps to log correctly\n\nThe `LOGGING` dict this provides sets up the \"general\" things to log, but if\nyou want to include your own Python libraries or Django apps you need to specify\nthem. To specify a Django app named 'foo', you would simply adjust the `LOGGING` dict like this:\n\n```\nLOGGERS = {\n 'foo': {\n 'handlers': ['json-stdout'],\n 'formatters': ['json'],\n 'propagate': True,\n 'level': 'INFO',\n }\n}\n\nLOGGING['loggers'].update(LOGGERS)\n\ndictConfig(LOGGING)\n```\n\nOr if you want to log EVERYTHING at an `DEBUG` level you can set a blank (aka default) logger:\n\n```\nLOGGERS = {\n '': {\n 'handlers': ['json-stdout'],\n 'formatters': ['json'],\n 'propagate': True,\n 'level': 'DEBUG',\n }\n}\n\nLOGGING['loggers'].update(LOGGERS)\n\ndictConfig(LOGGING)\n```\n\n## Usage\n\nThis is normal Python logging so you can do something simple like our info call\nbelow or more complex like the debug call and add additional data to the log information:\n\n```\nimport logging\n\nlogger = logging.getLogger(__name__)\n\nlogger.info(\"Simple log message\")\n\nfoo = 12\nbar = 'something else'\nlogger.debug(\"More complicated message\", extra={\n \"foo\": foo,\n \"bar\": bar,\n})\n\n```\n\n## Need help?\n\n[REVSYS](http://www.revsys.com?utm_medium=github&utm_source=jslog4kube) can help with your Python, Django, and infrastructure projects. If you have a question about this project, please open a GitHub issue. If you love us and want to keep track of our goings-on, here's where you can find us online:\n\n\n\n\n\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://gitlab.com/stephen6/jslog4kube", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "jslog4kube", "package_url": "https://pypi.org/project/jslog4kube/", "platform": "", "project_url": "https://pypi.org/project/jslog4kube/", "project_urls": { "Homepage": "http://gitlab.com/stephen6/jslog4kube" }, "release_url": "https://pypi.org/project/jslog4kube/1.0.6/", "requires_dist": [ "python-json-logger" ], "requires_python": "", "summary": "relatively hassle-free JSON logging for Kubernetes pods", "version": "1.0.6" }, "last_serial": 5446960, "releases": { "1.0.4": [ { "comment_text": "", "digests": { "md5": "7bdb6aded0a97ac5a15f426760ade6ca", "sha256": "7d054e9c68fd85625122191f298cdf362f5d8c8f3e39faae48e7d4c3bcb5d4e4" }, "downloads": -1, "filename": "jslog4kube-1.0.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7bdb6aded0a97ac5a15f426760ade6ca", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 7170, "upload_time": "2019-06-25T15:28:25", "url": "https://files.pythonhosted.org/packages/dd/23/726e840905e08cd831a72cb1a5eb1704223dd248303b2775d1441156fd51/jslog4kube-1.0.4-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "11c5fa39a0b5f232dbebd0fa57d5bb40", "sha256": "8582c61e5516cb410c370843cc45da8a6542a88c4b8d68d2629cd2cbab7059aa" }, "downloads": -1, "filename": "jslog4kube-1.0.4.tar.gz", "has_sig": false, "md5_digest": "11c5fa39a0b5f232dbebd0fa57d5bb40", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8625, "upload_time": "2019-06-25T15:28:27", "url": "https://files.pythonhosted.org/packages/f9/7b/3dc610cf299e723a491d44fa4d71524d38cf16b2579c61a4a7cde5b437c2/jslog4kube-1.0.4.tar.gz" } ], "1.0.5": [ { "comment_text": "", "digests": { "md5": "025e273038fc665d9cef2b128909ea26", "sha256": "bccfa00739d8981317ce9dbde2f15eb402b51f2c93ddebbbc3abd074d7600df7" }, "downloads": -1, "filename": "jslog4kube-1.0.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "025e273038fc665d9cef2b128909ea26", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10866, "upload_time": "2019-06-25T15:32:46", "url": "https://files.pythonhosted.org/packages/4c/6b/0211b2a60a19b45340b37c42e87ed427f8e93515955e675eb57429cc9baf/jslog4kube-1.0.5-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "14753ad2ae0a39cca1d37a37f7872096", "sha256": "8e316a99c5557ec53f3d5d726a72aead6230f1b446890eaaea14e0fbdfb9b56f" }, "downloads": -1, "filename": "jslog4kube-1.0.5.tar.gz", "has_sig": false, "md5_digest": "14753ad2ae0a39cca1d37a37f7872096", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10968, "upload_time": "2019-06-25T15:32:47", "url": "https://files.pythonhosted.org/packages/25/fe/321c6caeba5e8a86f9842b8e480b95b3585d369ac83a0111be9a452536a9/jslog4kube-1.0.5.tar.gz" } ], "1.0.6": [ { "comment_text": "", "digests": { "md5": "26919d63b812a5afa6c7e1825357b9c6", "sha256": "07cbf494e5265198726b8203460401105e099a2816ba4ec9dae3ed5fe21322fb" }, "downloads": -1, "filename": "jslog4kube-1.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "26919d63b812a5afa6c7e1825357b9c6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10881, "upload_time": "2019-06-25T16:04:06", "url": "https://files.pythonhosted.org/packages/cc/7d/484b541fdd060ac9a145e061dcd3c45ad29292db21b828add95667728550/jslog4kube-1.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "90694e3343b67fd8c7b017a09bfb9b01", "sha256": "4b2fa3a9f9b920b74dae9d6707359b2bf62730ba502b8e92a918e8a5def036ff" }, "downloads": -1, "filename": "jslog4kube-1.0.6.tar.gz", "has_sig": false, "md5_digest": "90694e3343b67fd8c7b017a09bfb9b01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10971, "upload_time": "2019-06-25T16:04:07", "url": "https://files.pythonhosted.org/packages/58/19/a1d395d5de8998e889304b76ba2239659611bc43a2bae23cc0c9c792cbb2/jslog4kube-1.0.6.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "26919d63b812a5afa6c7e1825357b9c6", "sha256": "07cbf494e5265198726b8203460401105e099a2816ba4ec9dae3ed5fe21322fb" }, "downloads": -1, "filename": "jslog4kube-1.0.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "26919d63b812a5afa6c7e1825357b9c6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 10881, "upload_time": "2019-06-25T16:04:06", "url": "https://files.pythonhosted.org/packages/cc/7d/484b541fdd060ac9a145e061dcd3c45ad29292db21b828add95667728550/jslog4kube-1.0.6-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "90694e3343b67fd8c7b017a09bfb9b01", "sha256": "4b2fa3a9f9b920b74dae9d6707359b2bf62730ba502b8e92a918e8a5def036ff" }, "downloads": -1, "filename": "jslog4kube-1.0.6.tar.gz", "has_sig": false, "md5_digest": "90694e3343b67fd8c7b017a09bfb9b01", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10971, "upload_time": "2019-06-25T16:04:07", "url": "https://files.pythonhosted.org/packages/58/19/a1d395d5de8998e889304b76ba2239659611bc43a2bae23cc0c9c792cbb2/jslog4kube-1.0.6.tar.gz" } ] }