{ "info": { "author": "Nick Gernert", "author_email": "nicholas.gernert@altamiracorp.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# Cluster Logging library for python 3.6\n\nThis is a library for python 3.6 that will help fulfil a\nrequirement for cluster services and logging to elastic search.\n\n## Requirements\n\n- Python 3.6+\n- pipenv\n- Elasticsearch\n- Fluentd\n- Kibana\n\n## Pipenv & Pipfile\n\nPipenv combines package management and virtualenv into one tool. \npipenv: https://docs.pipenv.org/basics\n\nTo install:\n\n```bash\n$ pip install pipenv\n```\n\nPipfile replaces requirements.txt and it will specify both dependencies and dev dependencies in one file.\n\n## Development Installation\n\n```bash\n# install dependencies\npipenv install --dev\n\n# activate virtual environment\npipenv shell\n\n# alternativly you can start your script in a virtual environment context\npipenv run python main.py message that you want to send\n```\n\n## ULM Configuration\n\nThe default configuration uses Environment variables to configure the logger. \nThese values were grabbed from version 2.0 of ULM-logging-common Java implementation. Not every value is implemented yet and changes are upto date in table. \n[Repo](https://bitbucket.di2e.net/projects/PIR/repos/logging-common/browse)\n\n| Variable | Default Value | Description | Implemented |\n| -------- | ------------- | ----------- | ----------: |\n| ULM_FLUENTD_HOST | localhost | host of the FluentD service | \u2714\ufe0f\n| ULM_FLUENTD_PORT | 24224 | port of FluentD tcp \"in_forward\" | \u2714\ufe0f\n| ULM_FLUENTD_LABEL_ENVS | | comma separated list(with or without space) of other ENV vars to add as fields to FluentD logs | \u2714\ufe0f\n| ULM_FLUENTD_TAG | (project).(app) | tag to use when sending logs to fluentd, e.g. the MLA_IDENT string such as \"Myapp.logs\" | \u2714\ufe0f\n| ULM_FLUENTD_BUFFER | 1048576 | an integer of the number of bytes to allow in the buffer. Excess will be dropped (default is 1MiB) | \u274c\n| ULM_FLUENTD_TIMEOUT | 3.0 | an float of the number of ms to allow for communication with FluentD. | \u2714\ufe0f\n| ULM_FLUENTD_CLIENT | async | Whether to send async messages or not. This will send async as long as the value has async somewhere in the string. | \u2714\ufe0f\n| ULM_FLUENTD_ENABLE_TIMEMS_FIELD | False | Enables an additional field to contain the millisecond timestamp | \u274c\n| ULM_FLUENTD_TIMEMS_FIELD | timems | the name of the field to contain the millisecond timestamp | \u274c\n\n## Standalone Usage (Development)\n\nIt may be required to run the following command to get elastic search to run.\n\n```bash\nsudo sysctl -w vm.max_map_count=262144\n```\n\n### Local\n\n```bash\n$ pipenv run python main.py -h\nusage: main.py [-h] [-n HOST] [-p PORT] [-e [ENV [ENV ...]]] [-o PROJECT]\n [-a APP] [-c COUNT]\n\nConsole Application to test Cluster Logging using fluentd.\n\npositional arguments:\n message Message to be logged.\n\noptional arguments:\n -h, --help show this help message and exit\n -n HOST, --host HOST Hostname to listen on. (default=localhost)\n -p PORT, --port PORT Port number to bind to. (default=24224)\n -e [ENV [ENV ...]], --env [ENV [ENV ...]]\n Environment Keys to add to message.\n -o PROJECT, --project PROJECT\n Tag for the project\n -a APP, --app APP Tag for the application\n -c COUNT, --count COUNT\n The number of times to send message.\n```\n\n### Docker\n\n```bash\n$ docker build -t TAG_FOR_THE_BUILD:VERSION .\n\n# all env values can be set with -e\n\n$ docker run -e host=fluentd \\\n -e port=24224 -e msg=\"message you want to send\" TAG_FOR_THE_BUILD:VERSION\n\n# If you want to run the whole environment you can use docker-compose up\n\n$ docker-compose up\n```\n\n### Api\n\nFollows the python logger implementation with logging levels. [python.org](https://docs.python.org/3/library/logging.html)\n\n| Level | Numeric value |\n| -------- | ------------: |\n| CRITICAL | 50 |\n| ERROR | 40 |\n| WARNING | 30 |\n| INFO | 20 |\n| DEBUG | 10 |\n| NOTSET | 0 |\n\nLogging level can be set for logging using:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.WARNING)\n```\n\nSetting the logging level will filter levels below the set logging level. For logging levels error and above the stack trace will be added to the log message.\n\n### Standard Configuration\n\nThese environment variables can be set either system-wide or added to the `.env` file:\n\nThere are 3 Environment variables that will be extracted by default:\n\n- `HOST` - host that the container is running on.\n- `APP_VERSION` - Version of the app.\n- `LOGGER_VERBOSE` - Whether the fluent logger library is verbose or not.\n- `MARATHON_APP_ID` - The ID that is assigned by DCOS.\n- `MARATHON_APP_DOCKER_IMAGE` - The image that the container was built from.\n\nThere are three ways to get application properties into log messages.\n\n- pass a dictionary of key value pairs\n- pass a list of keys that are on the container environment\n- or pass a JSON file with key value pairs.\n\nNOTE:\n\n- The dictionary will overwrite values in environment and JSON file.\n- The Environment will overwrite values in the JSON file.\n\n### Usage\n\nActivate your venv\n\n```bash\n# Linux / MacOS\nsource {NAME_OF_VENV}/bin/activate\n```\n\n```bat\nREM Windows\n{NAME_OF_VENV}\\Scripts\\activate\n```\n\nInstall cluster logger into your venv. Cluster Logger has been installed into pypi.\n\n```bash\n$ pip install cluster_logger\n```\n\nTo Use\n\n```python\nimport cluster_logger\nimport logging\n# Then init the ClusterLogger class\n\nenv_keys = ['MARATHON_APP_LABELS', 'MARATHON_APP_RESOURCE_CPUS']\nprops = {'HOST', '192.168.0.12'}\n\nlogging.basicConfig(level=logging.DEBUG) # set logging level\ncluster_logger.initLogger('Rasters', # Project name\n 'intent_client', # Application name\n 'fluentd', # Fluentd Host name (optional) will default to fluentd if nothing is passed in and HOST isn't set on the environment\n 24224, # Fluentd Port Number(optional)\n '/path/to/settings.json', # JSON settings file (optional)\n env_keys, # Environment Keys (optional)\n props) # Specific Properties (optional)\n\n# in Module where you will log\nimport cluster_logger\n\n# General Logger\nlogger = cluster_logger.getLogger(__name__)\n# Metric Logger\nmetrics = cluster_logger.getLogger(__name__ + '.metrics', is_metric=True)\n\nlogger.exception('Custom Message to send typically an error')\nlogger.log(logging.INFO, {'Duration', 58})\n```\n\n### Contributing\n\n```bash\n# bump the version that is stored in setup.py and cluster_logger/__init__.py\n# assuming pipenv install --dev was ran first\n$ pipenv run bumpversion minor # possible: major / minor / patch\n$ git push\n```\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://bitbucket.di2e.net/projects/PIR/repos/websl-logging", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "cluster-logger", "package_url": "https://pypi.org/project/cluster-logger/", "platform": "", "project_url": "https://pypi.org/project/cluster-logger/", "project_urls": { "Homepage": "https://bitbucket.di2e.net/projects/PIR/repos/websl-logging" }, "release_url": "https://pypi.org/project/cluster-logger/0.5.0/", "requires_dist": [ "fluent-logger" ], "requires_python": "", "summary": "Library for wrapping cluster logging for Altamira projects.", "version": "0.5.0" }, "last_serial": 4003804, "releases": { "0.4.0": [ { "comment_text": "", "digests": { "md5": "5142896637a87a6313bdc06148aca142", "sha256": "063d4e04cf88c7d2d1ec6623d49fffcaea05f15cc9624f80fca4db978efa9774" }, "downloads": -1, "filename": "cluster_logger-0.4.0-py3-none-any.whl", "has_sig": false, "md5_digest": "5142896637a87a6313bdc06148aca142", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6519, "upload_time": "2018-06-11T17:43:15", "url": "https://files.pythonhosted.org/packages/d6/37/635de14b5166d3cbc04e203d79537124993acf06614466163027483d39ec/cluster_logger-0.4.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "fb5d69ce885d2de0060e06ed5576c3b1", "sha256": "f108ec16717ca6f60ca198f20d5a9f7f63ec249099e3ca1bf076d25e974c4301" }, "downloads": -1, "filename": "cluster_logger-0.4.0.tar.gz", "has_sig": false, "md5_digest": "fb5d69ce885d2de0060e06ed5576c3b1", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7103, "upload_time": "2018-06-11T17:43:15", "url": "https://files.pythonhosted.org/packages/d7/d4/b9d3c56cb0963f3cf86271b73cb97f40e413d7903123a629542d6a9010a6/cluster_logger-0.4.0.tar.gz" } ], "0.4.1": [ { "comment_text": "", "digests": { "md5": "00e271356cd27ab9b0faa8c60e4d7575", "sha256": "af52898e197e795b58cb6e53cfec4a9665501431139b39e44e55afd444288294" }, "downloads": -1, "filename": "cluster_logger-0.4.1-py3-none-any.whl", "has_sig": false, "md5_digest": "00e271356cd27ab9b0faa8c60e4d7575", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 6587, "upload_time": "2018-06-13T17:34:50", "url": "https://files.pythonhosted.org/packages/bf/cc/9afed9baed5eb57a56b5c6740c4ea08f12115537f97c4f930bb9ea9be551/cluster_logger-0.4.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "bd03980fe777b7f1e155ed0545ac94ec", "sha256": "00fe3dd49de4ffde6cb70a8cd70fe72f7372e3317ebd7bce3bc73d9fd035e24e" }, "downloads": -1, "filename": "cluster_logger-0.4.1.tar.gz", "has_sig": false, "md5_digest": "bd03980fe777b7f1e155ed0545ac94ec", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7179, "upload_time": "2018-06-13T17:34:51", "url": "https://files.pythonhosted.org/packages/95/48/5cf4a9648ed47ababef752c4c1833fdf2cc7a2f1dd16974c9cfff785be2b/cluster_logger-0.4.1.tar.gz" } ], "0.5.0": [ { "comment_text": "", "digests": { "md5": "469552c980b6f86c581f8e0abc5f67c9", "sha256": "f2ef88d20cb50433a5dd3c9951db8ebf6c40fd1cdfdfd97ab05904769211a4ee" }, "downloads": -1, "filename": "cluster_logger-0.5.0-py3-none-any.whl", "has_sig": false, "md5_digest": "469552c980b6f86c581f8e0abc5f67c9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 8140, "upload_time": "2018-06-26T14:07:43", "url": "https://files.pythonhosted.org/packages/30/5b/fdc51d0427ba7599985bb2c73d50f12116902fe866fe597551ee56f2e592/cluster_logger-0.5.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "50d1f5bd8291c4bee7b8b6ad0b6f3104", "sha256": "7e08b65b32084300d5a3948fa4a4bd453cc076d885a8a2c4b990f7781f033284" }, "downloads": -1, "filename": "cluster_logger-0.5.0.tar.gz", "has_sig": false, "md5_digest": "50d1f5bd8291c4bee7b8b6ad0b6f3104", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8711, "upload_time": "2018-06-26T14:07:44", "url": "https://files.pythonhosted.org/packages/7a/7c/ccfc0c3722da06783e4ed13bd0c47fe830430bd1e242f6aa6ccdffac2a36/cluster_logger-0.5.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "469552c980b6f86c581f8e0abc5f67c9", "sha256": "f2ef88d20cb50433a5dd3c9951db8ebf6c40fd1cdfdfd97ab05904769211a4ee" }, "downloads": -1, "filename": "cluster_logger-0.5.0-py3-none-any.whl", "has_sig": false, "md5_digest": "469552c980b6f86c581f8e0abc5f67c9", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 8140, "upload_time": "2018-06-26T14:07:43", "url": "https://files.pythonhosted.org/packages/30/5b/fdc51d0427ba7599985bb2c73d50f12116902fe866fe597551ee56f2e592/cluster_logger-0.5.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "50d1f5bd8291c4bee7b8b6ad0b6f3104", "sha256": "7e08b65b32084300d5a3948fa4a4bd453cc076d885a8a2c4b990f7781f033284" }, "downloads": -1, "filename": "cluster_logger-0.5.0.tar.gz", "has_sig": false, "md5_digest": "50d1f5bd8291c4bee7b8b6ad0b6f3104", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 8711, "upload_time": "2018-06-26T14:07:44", "url": "https://files.pythonhosted.org/packages/7a/7c/ccfc0c3722da06783e4ed13bd0c47fe830430bd1e242f6aa6ccdffac2a36/cluster_logger-0.5.0.tar.gz" } ] }