{ "info": { "author": "Ismael Martinez Ramos", "author_email": "ismaelmartinez@gmail.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.7" ], "description": "# logger-to-kibana\n\n[![Build Status](https://dev.azure.com/ismaelmartinez0550/logger-to-kibana/_apis/build/status/IsmaelMartinez.logger-to-kibana?branchName=master)](https://dev.azure.com/ismaelmartinez0550/logger-to-kibana/_build/latest?definitionId=5&branchName=master)\n\n---\n\nThis project is inteded to generate view from the log messages encountered.\n\nThe python executable can be found in [https://pypi.org/project/logger-to-kibana/](https://pypi.org/project/logger-to-kibana/)\n\nYou will need to install the dependences by running\n\n```bash\npip install -r requirements.txt\n```\n\nTo get the programs help just type:\n\n```bash\npython main.py\n```\n\nThis returns:\n\n```bash\n Usage: main.py [OPTIONS] COMMAND [ARGS]...\n\n Process the file provided according to conditions\n\nOptions:\n --help Show this message and exit.\n\nCommands:\n pommands:\n process Process the folder\n process_and_generate Process the folder and generate visualization\n process_generate_and_send Process the folder, generate visualization and\n send\n```\n\n## Default settings\n\nThe default settings can be found in the [settings.ini](settings.ini) file. You can provide a different settings\nfile by specifying it as an environment variable LOGGER_TO_KIBANA_CONFIG\n\n## commands\n\nThe current available commands are:\n\n### process\n\nProcess a folder and prints out the processed functions/logs in the following format:\n\n```bash\n[{'subfolder': '', 'filename': '', 'function': '', 'type': '', 'query': 'message: \"\"', 'label': ': '}]\n```\n\nTo execute the command run:\n\n```bash\npython main.py process -f \n```\n\nCheck the table under [How does it work] section to get more info about log_type and log_filter.\n\n### process_and_generate\n\nProcess a folder (as shown in the process section) and generates a table visualization for kibana.\n\nTo execute the command run:\n\n```bash\npython main.py process_and_generate -f \n```\n\n### process_generate_and_send\n\nProcess a folder, generates a table visualization for kibana and send it to kibana (currently in localhost:5601)\n\nTo execute the command run:\n\n```bash\npython main.py process_and_generate -f \n```\n\n### How does it work\n\nBy default, it scans for files under the folder specified and with the pattern `app/src/**/*.py`. You can specify another patter in the FilesMatchFilter in the [settings.ini](settings.ini)\n\nThis program uses different regex `detectors` to filter logs and files to process.\n\nThose can be changed in the [settings.ini](settings.ini) file.\n\nThe current available detectors are:\n\n| Detector | Default Value | Propose |\n|---|---|---|\n| FilesMatchFilter | app/src/**/*.py | Filter the files to process in the provided folder |\n| FunctionMappingDetector | def | Detect a function |\n| FunctionMappingFilter | (?<=def ).*?(?=\\() | Filter the function name |\n| LogDebugDetector | LOG.debug | Detect the log debug message |\n| LogDebugFilter | (?<=LOG.debug\\([\"\\']).*?(?=[\"\\']) | Filter the log debug message |\n| LogInfoDetector | LOG.info | Detect the log info message |\n| LogInfoFilter | (?<=LOG.info\\([\"\\']).*?(?=[\"\\']) | Filter the log info message |\n| LogWarnDetector | LOG.warn | Detect the log warn message |\n| LogWarnFilter | (?<=LOG.warn\\([\"\\']).*?(?=[\"\\']) | Filter the log warn message |\n| LogErrorDetector | LOG.error | Detect the log error message |\n| LogErrorFilter | (?<=LOG.error\\([\"\\']).*?(?=[\"\\']) | Filter the log error message |\n| LogCriticalDetector | LOG.critical | Detect the log critical message |\n| LogCriticalFilter | (?<=LOG.critical\\([\"\\']).*?(?=[\"\\']) | Filter the log critical message |\n| LogExceptionDetector | LOG.exception | Detect the log exception message |\n| LogExceptionFilter | (?<=LOG.exception\\([\"\\']).*?(?=[\"\\']) | Filter the log exception message |\n\nOther configuration available in the settings.ini file are:\n| Type | Value | Propose |\n| -- | -- | -- |\n| BaseUrl | [http://localhost:5601](http://localhost:5601) | Kibana base url |\n| Index | 59676040-e7fd-11e9-9209-1f165c3af176 | Kibana index |\n\n## The process\n\nThe commands for the application are done in the following logical order.\n\n```bash\n process -> generate -> send\n```\n\nAs an example, when processing a file in `tests/unit/resources/example.py` with the content:\n\n```python\ndef lambda_handler(_event: dict, _context):\n LOG.debug('Initialising')\n LOG.info('Processing')\n LOG.warn('Success')\n LOG.error('Failure')\n LOG.critical('Bananas')\n LOG.exception('Exception')\n)\n```\n\nWill return the next object:\n\n```python\n[{'subfolder': 'resources', 'filename': 'example.py', 'function': 'lambda_handler', 'type': 'debug', 'query': 'message: \"Initialising\"', 'label': 'debug: Initialising'},\n{'subfolder': 'resources', 'filename': 'example.py', 'function': 'lambda_handler','type': 'info', 'query': 'message: \"Processing\"', 'label': 'info: Processing'},\n{'subfolder': 'resources', 'filename': 'example.py', 'function': 'lambda_handler', 'type': 'warn', 'query': 'message: \"Success\"', 'label': 'warn: Success'},\n{'subfolder': 'resources', 'filename': 'example.py', 'function': 'lambda_handler', 'type': 'error', 'query': 'message: \"Failure\"', 'label': 'error: Failure'},\n{'subfolder': 'resources', 'filename': 'example.py', 'function': 'lambda_handler', 'type': 'critical', 'query': 'message: \"Bananas\"', 'label': 'critical: Bananas'},\n{'subfolder': 'resources', 'filename': 'example.py', 'function': 'lambda_handler', 'type': 'exception', 'query': 'message: \"Exception\"', 'label': 'exception: Exception'}]\n```\n\nGenerate, will generate a table visualization with filters for all the logs that have found.\n\nThe choise of having a table visualization is for the amount of information available. It is, basically, a good place to start as later we can split the visualizations by file, function or whatever we choose to do so.\n\nYou can change the type of visualization generated by modifying the VisualizationType in the [settings.ini](settings.ini). The current available values are metric or table. The default value is table.\n\nTo finish, it sends the generated visualization to Kibana with the following name format:\n\n[Generated - ]\n\n## Limitations\n\nIt only generates the visualizations and not the dashboards.\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/ismaelmartinez/logger-to-kibana", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "logger-to-kibana", "package_url": "https://pypi.org/project/logger-to-kibana/", "platform": "", "project_url": "https://pypi.org/project/logger-to-kibana/", "project_urls": { "Homepage": "https://github.com/ismaelmartinez/logger-to-kibana" }, "release_url": "https://pypi.org/project/logger-to-kibana/0.4.2/", "requires_dist": null, "requires_python": "", "summary": "Import logger messages from a file and generates a Kibana Visualization", "version": "0.4.2" }, "last_serial": 6001166, "releases": { "0.0.5": [ { "comment_text": "", "digests": { "md5": "64e0bbaa33338e273d6f204ecd320d3d", "sha256": "26bb650134d1c6fe685c63f5f421d5314f7915fe239e8a45f17a8135a1e40d4b" }, "downloads": -1, "filename": "logger_to_kibana-0.0.5-py3-none-any.whl", "has_sig": false, "md5_digest": "64e0bbaa33338e273d6f204ecd320d3d", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13557, "upload_time": "2019-09-30T21:06:43", "url": "https://files.pythonhosted.org/packages/76/63/283e45ef29e944455a6d263931a4d02eea02ddc5d39b9ff53f169a683811/logger_to_kibana-0.0.5-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "94d02a78c954cf61218141a314ded366", "sha256": "91ac7c16872df22c2316d4941a7b18cb4ea3191075061d7e90747b3f77a98cdc" }, "downloads": -1, "filename": "logger_to_kibana-0.0.5.tar.gz", "has_sig": false, "md5_digest": "94d02a78c954cf61218141a314ded366", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7778, "upload_time": "2019-09-30T21:06:46", "url": "https://files.pythonhosted.org/packages/8a/17/8dfbad3a99155f5a0d7d29c2234466882a02b3493fed0cb55873af0d0389/logger_to_kibana-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "21985e6ce04c6e24ad06433494841d52", "sha256": "fca55aca7648e4d090fc6043b4856ec4635e7486c8eea13f25761f0444cb1204" }, "downloads": -1, "filename": "logger_to_kibana-0.0.6-py3-none-any.whl", "has_sig": false, "md5_digest": "21985e6ce04c6e24ad06433494841d52", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13903, "upload_time": "2019-10-04T16:18:31", "url": "https://files.pythonhosted.org/packages/55/70/bf5fcf3dc36b71284497a8a9ea7f72977d854320fe14f6e3d051432226f1/logger_to_kibana-0.0.6-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e452dce19cdfa3da6e98ea3e3e25b068", "sha256": "ce042d65596b00bb892d4c956de8b66d93e00fc3760e6f50517fe417fa474e1e" }, "downloads": -1, "filename": "logger_to_kibana-0.0.6.tar.gz", "has_sig": false, "md5_digest": "e452dce19cdfa3da6e98ea3e3e25b068", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8021, "upload_time": "2019-10-04T16:18:33", "url": "https://files.pythonhosted.org/packages/b5/df/2ac1a071124795ee40e0a7b938543da6d744d29cfa15f0698a4a8a34e030/logger_to_kibana-0.0.6.tar.gz" } ], "0.0.7": [ { "comment_text": "", "digests": { "md5": "ed159fed089499f9b4ef76adbdd29657", "sha256": "fc42025834338bd39b6a1b6d8816fad3c643a75da963b6581236bc540db23f2b" }, "downloads": -1, "filename": "logger_to_kibana-0.0.7-py3-none-any.whl", "has_sig": false, "md5_digest": "ed159fed089499f9b4ef76adbdd29657", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14066, "upload_time": "2019-10-05T23:01:57", "url": "https://files.pythonhosted.org/packages/c1/7d/c68336cd872556ad2e0befd3813e0d4dadbfb0f57057f2fc355828b030f0/logger_to_kibana-0.0.7-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "19bb1b18466dc11c110c33ef08be82fa", "sha256": "afefabe6eb6dfc86ebd9222aec56a72744a637b4fccd47f1c2401172c53c9f6f" }, "downloads": -1, "filename": "logger-to-kibana-0.0.7.tar.gz", "has_sig": false, "md5_digest": "19bb1b18466dc11c110c33ef08be82fa", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8183, "upload_time": "2019-10-05T23:01:59", "url": "https://files.pythonhosted.org/packages/28/cc/49454c43fa3fa224c73d7b707a52f25ea4ee01af16d6d30e510da1f7cc6c/logger-to-kibana-0.0.7.tar.gz" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "88d2994b179e1056b361fca1e9647d74", "sha256": "55a8c1543f2cf697010fab5df27aac7d62480b20c1faa7df159ef739ea5a9f15" }, "downloads": -1, "filename": "logger_to_kibana-0.0.8-py3-none-any.whl", "has_sig": false, "md5_digest": "88d2994b179e1056b361fca1e9647d74", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14089, "upload_time": "2019-10-05T23:08:47", "url": "https://files.pythonhosted.org/packages/d7/0d/61f0c30c48dd414d8fe19231148691bc5189a9dd66357b8d55f744a234e5/logger_to_kibana-0.0.8-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "8861195070e95ab830e08e0f2bb1bb83", "sha256": "8b8ba13f0f2b9a86a944e8aedcd81dd4a6af1fd2abef68d3f3838ebda6df2fa9" }, "downloads": -1, "filename": "logger-to-kibana-0.0.8.tar.gz", "has_sig": false, "md5_digest": "8861195070e95ab830e08e0f2bb1bb83", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8215, "upload_time": "2019-10-05T23:08:48", "url": "https://files.pythonhosted.org/packages/74/b8/88ce04b90bc35d060b7a4a875103cc307a88e4d891c2b2d67423bf16cdd6/logger-to-kibana-0.0.8.tar.gz" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "a4b5dd211028a40729feb261e8796852", "sha256": "b70bd6d90d8bd88dffb6105a088a63bf49e64fc0e2c5ee5512bac8211e789fba" }, "downloads": -1, "filename": "logger_to_kibana-0.0.9-py3-none-any.whl", "has_sig": false, "md5_digest": "a4b5dd211028a40729feb261e8796852", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9571, "upload_time": "2019-10-06T06:41:13", "url": "https://files.pythonhosted.org/packages/7d/a9/450283c41775babc4dcdaa4d54ad1e7e72c13f3889f2f262b680e1b5b195/logger_to_kibana-0.0.9-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "32bca88852eae6118e578d19b69d06b9", "sha256": "ffb50f3e64afba126678e3dfcc21b3bad73af65fb35e54693e764059f5ed5fa1" }, "downloads": -1, "filename": "logger-to-kibana-0.0.9.tar.gz", "has_sig": false, "md5_digest": "32bca88852eae6118e578d19b69d06b9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6002, "upload_time": "2019-10-06T06:41:14", "url": "https://files.pythonhosted.org/packages/03/c5/92cc7f8b4f64a6809ab795efd580555c80b60a0d3569a47515aaf605d9bc/logger-to-kibana-0.0.9.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "245d75fe81379581097293c4b62dad2b", "sha256": "555539d4df99bbe075a4839d8bd200cd9cc1f2366832cd3c8059a6bacd2ca994" }, "downloads": -1, "filename": "logger_to_kibana-0.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "245d75fe81379581097293c4b62dad2b", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9535, "upload_time": "2019-10-10T20:55:58", "url": "https://files.pythonhosted.org/packages/5f/85/0f3c97d7908f67f1fb8ce2cf0c1ec6abf1b2091928b3678f1e8b1bc6f20c/logger_to_kibana-0.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9fab22d109d435caf0bf549840b28605", "sha256": "1360370c829592c98b8c1a61f0fd051b8c1677a63f621655c365d589b0ef3dae" }, "downloads": -1, "filename": "logger-to-kibana-0.3.1.tar.gz", "has_sig": false, "md5_digest": "9fab22d109d435caf0bf549840b28605", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5980, "upload_time": "2019-10-10T20:56:00", "url": "https://files.pythonhosted.org/packages/fe/9a/017e76abad64afae31d2859e6bf4523bb11d9c0da1c6bc1ee88e5d6fdad4/logger-to-kibana-0.3.1.tar.gz" } ], "0.4.0": [ { "comment_text": "", "digests": { "md5": "0c44654dcf80649743b4a8f6619b5810", "sha256": "106983d43de51dd118167babf8c13619617c9c3363b285eb8e568075a4f64292" }, "downloads": -1, "filename": "logger_to_kibana-0.4.0-py3-none-any.whl", "has_sig": false, "md5_digest": "0c44654dcf80649743b4a8f6619b5810", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9679, "upload_time": "2019-10-13T20:32:16", "url": "https://files.pythonhosted.org/packages/c7/63/2f8847bd3baf82c8f4f78fdbe1c8337237d31c21c866284f3a1f57b742bc/logger_to_kibana-0.4.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "6e7bc5e798f3bec21b248ed0e18e4b30", "sha256": "63b7adedbaae240a1f2660841ba4a44c4cfedd5a5f154297cb261fa93b8fe0fb" }, "downloads": -1, "filename": "logger-to-kibana-0.4.0.tar.gz", "has_sig": false, "md5_digest": "6e7bc5e798f3bec21b248ed0e18e4b30", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6088, "upload_time": "2019-10-13T20:32:17", "url": "https://files.pythonhosted.org/packages/9d/38/6fadc98f18bb6f413d1e2db5a72e23c949a10269045ec1b5bded13b2911a/logger-to-kibana-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "2f4c3548b63fd354bfc9d7a47b174fd1", "sha256": "c48a721663791e61348ccc5634d262e71837bd74e868592bb0913f68090e91bd" }, "downloads": -1, "filename": "logger_to_kibana-0.4.1-py3-none-any.whl", "has_sig": false, "md5_digest": "2f4c3548b63fd354bfc9d7a47b174fd1", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 9678, "upload_time": "2019-10-19T21:55:05", "url": "https://files.pythonhosted.org/packages/4e/62/1222f9f9ed24a9424195f6cb31558221ccf13252dcdeceb5c14f98cc8911/logger_to_kibana-0.4.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "639e14d1c1bf49d230923d7392e29c4f", "sha256": "42ce60cc14816d14c70cd681feea69b2ea77ba06209ab3566da8848d372c63f4" }, "downloads": -1, "filename": "logger-to-kibana-0.4.1.tar.gz", "has_sig": false, "md5_digest": "639e14d1c1bf49d230923d7392e29c4f", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6094, "upload_time": "2019-10-19T21:55:06", "url": "https://files.pythonhosted.org/packages/34/82/39eb24b667db2deeb618cc9e464ca8e133852b4b2092b945b1bed84b0c12/logger-to-kibana-0.4.1.tar.gz" } ], "0.4.2": [ { "comment_text": "", "digests": { "md5": "3f6d5d31af7ac9a011d02c4deccb52c6", "sha256": "96fcaf385c2ec52288982090742c01c16eccbfab08dde870840a7f2c3daeec9f" }, "downloads": -1, "filename": "logger_to_kibana-0.4.2-py3-none-any.whl", "has_sig": false, "md5_digest": "3f6d5d31af7ac9a011d02c4deccb52c6", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14561, "upload_time": "2019-10-19T22:00:25", "url": "https://files.pythonhosted.org/packages/76/16/4c31ccf4b37660c528415a69dee1bc2ab69e0174acea88dc46d90c3b4cad/logger_to_kibana-0.4.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ed0bd91a920623328b10bf7c0e9fb623", "sha256": "a5100811710592e6dbef4c92396722688c263136f293861ee8cb9f88a775d9cb" }, "downloads": -1, "filename": "logger-to-kibana-0.4.2.tar.gz", "has_sig": false, "md5_digest": "ed0bd91a920623328b10bf7c0e9fb623", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8700, "upload_time": "2019-10-19T22:00:27", "url": "https://files.pythonhosted.org/packages/23/ef/5e4929cbba37bc2dd20a75c9938967dc131197d58fe37841ba2c7af26d33/logger-to-kibana-0.4.2.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "3f6d5d31af7ac9a011d02c4deccb52c6", "sha256": "96fcaf385c2ec52288982090742c01c16eccbfab08dde870840a7f2c3daeec9f" }, "downloads": -1, "filename": "logger_to_kibana-0.4.2-py3-none-any.whl", "has_sig": false, "md5_digest": "3f6d5d31af7ac9a011d02c4deccb52c6", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 14561, "upload_time": "2019-10-19T22:00:25", "url": "https://files.pythonhosted.org/packages/76/16/4c31ccf4b37660c528415a69dee1bc2ab69e0174acea88dc46d90c3b4cad/logger_to_kibana-0.4.2-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "ed0bd91a920623328b10bf7c0e9fb623", "sha256": "a5100811710592e6dbef4c92396722688c263136f293861ee8cb9f88a775d9cb" }, "downloads": -1, "filename": "logger-to-kibana-0.4.2.tar.gz", "has_sig": false, "md5_digest": "ed0bd91a920623328b10bf7c0e9fb623", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8700, "upload_time": "2019-10-19T22:00:27", "url": "https://files.pythonhosted.org/packages/23/ef/5e4929cbba37bc2dd20a75c9938967dc131197d58fe37841ba2c7af26d33/logger-to-kibana-0.4.2.tar.gz" } ] }