{
"info": {
"author": "Joseph Axisa",
"author_email": "jax@looker.com",
"bugtrack_url": null,
"classifiers": [
"License :: OSI Approved :: MIT License",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7"
],
"description": "\n\n\n-----------------\n# Henry: A Looker Cleanup Tool\nHenry is a command line tool that helps determine model bloat in your Looker instance and identify unused content in models and explores. It is meant to help developers cleanup models from unused explores and explores from unused joins and fields, as well as maintain a healthy and user-friendly instance.\n\n## Table of Contents\n- [Henry: A Looker Cleanup Tool](#henry-a-looker-cleanup-tool)\n - [Table of Contents](#table-of-contents)\n - [Status and Support](#status-and-support)\n - [Where to get it](#where-to-get-it)\n - [Usage](#usage)\n - [Storing Credentials](#storing-credentials)\n - [Global Config File](#global-config-file)\n - [API timeout settings](#api-timeout-settings)\n - [Config Path](#config-path)\n - [Global Options that apply to many commands](#global-options-that-apply-to-many-commands)\n - [Suppressing Formatted Output](#suppressing-formatted-output)\n - [Output to File](#output-to-file)\n - [Pulse Command](#pulse-command)\n - [Connection Checks](#connection-checks)\n - [Query Stats](#query-stats)\n - [Scheduled Plans](#scheduled-plans)\n - [Legacy Features](#legacy-features)\n - [Version](#version)\n - [Analyze Command](#analyze-command)\n - [analyze projects](#analyze-projects)\n - [analyze models](#analyze-models)\n - [analyze explores](#analyze-explores)\n - [Vacuum Information](#vacuum-information)\n - [vacuum models](#vacuum-models)\n - [vacuum explores](#vacuum-explores)\n - [Logging](#logging)\n - [Dependencies](#dependencies)\n - [Development](#development)\n - [Authors](#authors)\n - [Contributing](#contributing)\n - [Code of Conduct](#code-of-conduct)\n - [Copyright](#copyright)\n\n\n## Status and Support\nHenry is **NOT** supported or warranted by Looker in any way. Please do not contact Looker support\nfor issues with Henry. Issues can be logged via https://github.com/looker-open-source/henry/issues\n\n\n## Where to get it\nThe source code is currently hosted on GitHub at https://github.com/looker-open-source/henry/. The latest released version can be found on [PyPI](https://pypi.org/project/henry/) and can be installed using:\n\n $ pip install henry\n\nFor development setup, follow the Development setup [below](#development).\n\n\n## Usage\nIn order to display usage information, use:\n\n $ henry --help\n\n\n### Storing Credentials\nAPI3 login credentials can be specified at runtime using various flags or more conveniently, using a `config.yml` having the format shown below.\n\n```\nhosts:\n dev_looker:\n host: devhostname.looker.com\n id: AbCdEfGhIjKlMnOp\n secret: QrStUvWxYz1234567890\n staging_looker:\n host: staginghostname.looker.com\n id: AbCdEfGhIjKlMnOp\n secret: QrStUvWxYz1234567890\n```\n\nMake sure that the `config.yml` file has restricted permissions by running `chmod 600 config.yml`. The tool will also ensure that this is the case every time it writes to the file.\n\nIf `config.yml` resides in the current working directory, then you don't need to do anything. If not, its location needs to be specified at runtime using the `--path` parameter or in the [global config file](#global-config-file). \n\n\n### Global Config File\nA global settings file called `settings.json` can be defined in `~/.henry`. The file can be used to define a number of paramaters to be used at runtime:\n\n```\n{\n \"api_conn_timeout\": x,\n \"config_path\": \"/path/to/api3/credentials/yml/file\"\n\n}\n```\n\n#### API timeout settings\nThe `api_conn_timeout` parameter can be used to specify API call timeout settings. It can take 3 types of values: null, an integer representing\nconnect and read timeouts (in seconds) combined or a list that specifies\nthe connect and read timeouts separately (e.g. \"[5, 15]\").\n\n\n#### Config Path\nThe `config_path` parameter defines the absolute location to the [API3 credentials file](#storing-credentials). \n\nIn order of precedence, these are the ways that are used to define the location of the credentials file path:\n--path, config_path in ~/.henry/settings.json and then the default.\n\n\n### Global Options that apply to many commands\n\n#### Suppressing Formatted Output\nMany commands provide tabular output. For tables the option `--plain` will suppress the table headers and format lines, making it easier to use tools like grep, awk, etc. to retrieve values from the output of these commands.\n\n\n#### Output to File\nUsing the `--output` option allows you to specify a path and a file to save the results to. When combined with `--plain` the format lines will be suppressed. Example usage:\n\n $ henry vacuum models --plain --output=unused_explores.csv\n\nsaves the results to *unused_explores.csv* in the current working directory.\n\n\n### Pulse Command\nThe command `henry pulse` runs a number of tests that help determine the overall instance health. A healthy Looker instance should pass all the tests. Below is a list of tests currently implemented.\n\n#### Connection Checks\nRuns specific tests for each connection to make sure the connection is in working order. If any tests fail, the output will show which tests passed or failed for that particular connection. Example:\n```\n+------------------+------------------------------------------------------------------------------------------------------+\n| Connection | Status |\n|------------------+------------------------------------------------------------------------------------------------------|\n| thelook | -- Failed to create or write to pdt connection registration table tmp.connection_reg_r3 : Connection |\n| | registration error for thelook: max registrations reached for connection thelook |\n| assets_analytics | OK |\n| events_ecommerce | OK |\n+------------------+------------------------------------------------------------------------------------------------------+\n```\n\n#### Query Stats\nChecks how many queries were run over the past 30 days and how many of them errored or got killed as well as some statistics around runtimes times. The IDs of queries that took more than 5 times the average query runtime are also outputted.\n\n#### Scheduled Plans\nDetermines the number of scheduled jobs that ran in the past 30 days, how many were successful, how many ran but did not deliver or failed to run altogether.\n\n#### Legacy Features\nOutputs a list of legacy features that are still in use if any. These are features that have been replaced with improved ones and should be moved away from.\n\n#### Version\nChecks if the latest Looker version is being used. Looker supports only up to 3 releases back.\n\n\n### Analyze Command\nThe `analyze` command is meant to help identify models and explores that have become bloated and use `vacuum` on them in order to trim them.\n\n\n#### analyze projects\nThe `analyze projects` command scans projects for their content as well as checks for the status of quintessential features for success such as the git connection status and validation requirements.\n```\n+-------------------+---------------+--------------+-------------------------+---------------------+-----------------------+\n| project | model_count | view_count | git_connection_status | pull_request_mode | validation_required |\n|-------------------+---------------+--------------+-------------------------+---------------------+-----------------------|\n| marketing | 1 | 13 | OK | links | True |\n| admin | 2 | 74 | OK | off | True |\n| powered_by_looker | 1 | 14 | OK | links | True |\n| salesforce | 1 | 36 | OK | required | False |\n| thelook_event | 1 | 17 | OK | required | True |\n+-------------------+---------------+--------------+-------------------------+---------------------+-----------------------+\n```\n\n\n#### analyze models\nShows the number of explores in each model as well as the number of queries against that model.\n```\n+-------------------+------------------+-----------------+-------------------+-------------------+\n| project | model | explore_count | unused_explores | query_run_count |\n|-------------------+------------------+-----------------+-------------------+-------------------|\n| salesforce | salesforce | 8 | 0 | 39923 |\n| thelook_event | thelook | 10 | 0 | 166307 |\n| powered_by_looker | powered_by | 5 | 0 | 49122 |\n| marketing | thelook_adwords | 3 | 0 | 40869 |\n| admin | looker_base | 0 | 0 | 0 |\n| admin | looker_on_looker | 10 | 9 | 28 |\n+-------------------+------------------+-----------------+-------------------+-------------------+\n```\n\n\n#### analyze explores\nShows explores and their usage. If the `--min_queries` argument is passed, joins and fields that have been used less than the threshold specified will be considered as unused.\n```\n+---------+-----------------------------------------+-------------+-------------------+--------------+----------------+---------------+-----------------+---------------+\n| model | explore | is_hidden | has_description | join_count | unused_joins | field_count | unused_fields | query_count |\n|---------+-----------------------------------------+-------------+-------------------+--------------+----------------+---------------+-----------------+---------------|\n| thelook | cohorts | True | No | 3 | 0 | 19 | 4 | 333 |\n| thelook | data_tool | True | No | 3 | 0 | 111 | 90 | 736 |\n| thelook | order_items | False | No | 7 | 0 | 153 | 16 | 126898 |\n| thelook | events | False | No | 6 | 0 | 167 | 68 | 19372 |\n| thelook | sessions | False | No | 6 | 0 | 167 | 83 | 12205 |\n| thelook | affinity | False | No | 2 | 0 | 34 | 13 | 3179 |\n| thelook | orders_with_share_of_wallet_application | False | No | 9 | 0 | 161 | 140 | 1586 |\n| thelook | journey_mapping | False | No | 11 | 2 | 238 | 228 | 14 |\n| thelook | inventory_snapshot | False | No | 3 | 0 | 25 | 15 | 33 |\n| thelook | kitten_order_items | True | No | 8 | 0 | 154 | 138 | 39 |\n+---------+-----------------------------------------+-------------+-------------------+--------------+----------------+---------------+-----------------+---------------+\n```\n\n\n### Vacuum Information\nThe `vacuum` command outputs a list of unused content based on predefined criteria that a developer can then use to cleanup models and explores.\n\n\n#### vacuum models\nThe `vacuum models` command exposes models and the number of queries against them over a predefined period of time. Explores that are listed here have not had the minimum number of queries against them in the timeframe specified. As a result it is safe to hide them and later delete them.\n```\n+------------------+---------------------------------------------+-------------------------+\n| model | unused_explores | model_query_run_count |\n|------------------+---------------------------------------------+-------------------------|\n| salesforce | None | 39450 |\n| thelook | None | 164930 |\n| powered_by | None | 49453 |\n| thelook_adwords | None | 38108 |\n| looker_on_looker | user_full | 27 |\n| | history_full | |\n| | content_view | |\n| | project_status | |\n| | field_usage_full | |\n| | dashboard_performance_full | |\n| | user_weekly_app_activity_period_over_period | |\n| | pdt_state | |\n| | user_daily_query_activity | |\n+------------------+---------------------------------------------+-------------------------+\n```\n\n\n#### vacuum explores\nThe `vacuum explores` command exposes joins and exposes fields that are below the minimum number of queries threshold (default =0, can be changed using the `--min_queries` argument) over the specified timeframe (default: 90, can be changed using the `--timeframe` argument).\n\nExample: from the analyze function run [above](#analyze_explores), we know that the cohorts explore has 4 fields that haven't been queried once in the past 90 days. Running the following vacuum command:\n\n $ henry vacuum explores --model thelook --explore cohorts\n\n provides the name of the unused fields:\n```\n+---------+-----------+----------------+------------------------------+\n| model | explore | unused_joins | unused_fields |\n|---------+-----------+----------------+------------------------------|\n| thelook | cohorts | N/A | order_items.created_date |\n| | | | order_items.id |\n| | | | order_items.total_sale_price |\n| | | | users.gender |\n+---------+-----------+----------------+------------------------------+\n```\nIt is very important to note that fields vacuumed fields in one explore are not meant to be completely removed from view files altogether because they might be used in other explores or joins. Instead, one should either hide those fields (if they're not used anywhere else) or exclude them from the explore using the _fields_ LookML parameter.\n\n\n## Logging\nThe tool logs activity as it's being used. Log files are stored in `~/.henry/log/` in your home directory. Sensitive information such as your client secret is filtered out for security reasons. Moreover, log files have restricted permissions which allow only the owner to read and write.\n\nThe logging module utilises a rotating file handler which is currently set to rollover when the current log file reaches 500 KB in size. The system saves old log files by adding the suffix '.1', '.2' etc., to the filename. The file being written to is always named `henry.log`. No more than 10 log files are kept at any point in time, ensuring logs do not consume more than 5 MB max.\n\n\n## Dependencies\n- [PyYAML](https://pyyaml.org/): 3.12 or higher\n- [requests](http://docs.python-requests.org/en/master/): 2.18.4 or higher\n- [tabulate](https://bitbucket.org/astanin/python-tabulate): 0.8.2 or higher\n- [tqdm](https://tqdm.github.io/): 4.23.4 or higher\n\n\n## Development\n\nTo install henry in development mode you need to install the dependencies above and clone the project's repo with:\n\n $ git clone git@github.com:looker-open-source/henry.git\n\nYou can then install using:\n\n $ python setup.py develop\n\nAlternatively, you can use `pip` if you want all the dependencies pulled in automatically (the -e option is for installing it in [development mode](https://pip.pypa.io/en/latest/reference/pip_install/#editable-installs)).\n\n $ pip install -e .\n\n\n## Authors\n\nHenry has primarily been developed by [Joseph Axisa](https://github.com/josephaxisa). See [all contributors](https://github.com/looker-open-source/henry/graphs/contributors).\n\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/looker-open-source/henry/issues. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.\n\n\n## Code of Conduct\n\nEveryone interacting in the Henry project\u2019s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/looker-open-source/henry/blob/master/CODE_OF_CONDUCT.md).\n\n\n## Copyright\n\nCopyright (c) 2018 Joseph Axisa for Looker Data Sciences. See [MIT License](LICENSE.txt) for further details.\n\n\n",
"description_content_type": "text/markdown",
"docs_url": null,
"download_url": "",
"downloads": {
"last_day": -1,
"last_month": -1,
"last_week": -1
},
"home_page": "https://github.com/looker-open-source/henry",
"keywords": "",
"license": "MIT",
"maintainer": "",
"maintainer_email": "",
"name": "henry",
"package_url": "https://pypi.org/project/henry/",
"platform": "",
"project_url": "https://pypi.org/project/henry/",
"project_urls": {
"Homepage": "https://github.com/looker-open-source/henry"
},
"release_url": "https://pypi.org/project/henry/0.1.3/",
"requires_dist": [
"requests",
"pyyaml",
"tabulate",
"tqdm"
],
"requires_python": ">=3.6",
"summary": "A Looker Cleanup Tool",
"version": "0.1.3"
},
"last_serial": 5161268,
"releases": {
"0.0.1b2": [
{
"comment_text": "",
"digests": {
"md5": "2cbd61c8fb09dde94fe50f5088c991b0",
"sha256": "88c444c1e5051bb6101f49958ccbb45e2f6083c72263e566a156d40af11550b7"
},
"downloads": -1,
"filename": "henry-0.0.1b2-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "2cbd61c8fb09dde94fe50f5088c991b0",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.6.0",
"size": 26853,
"upload_time": "2018-07-25T15:29:03",
"url": "https://files.pythonhosted.org/packages/1d/f6/a85bf7b374de22b517ba30b8d81749fc704f88a21270db71025e87ca29bb/henry-0.0.1b2-py2.py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "fe2cf3f6c72f3cd99011bf6b975c5dbc",
"sha256": "2217a2ae89270524619d0172e58d40655053dda788d0684cb55d46dc2cc5b6cd"
},
"downloads": -1,
"filename": "henry-0.0.1b2.tar.gz",
"has_sig": false,
"md5_digest": "fe2cf3f6c72f3cd99011bf6b975c5dbc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 26653,
"upload_time": "2018-07-25T15:29:04",
"url": "https://files.pythonhosted.org/packages/47/e9/4b9d5713ece7e6972b7964fdbece014f5d03b493075ef9b6c8dd618f21a7/henry-0.0.1b2.tar.gz"
}
],
"0.1.1": [
{
"comment_text": "",
"digests": {
"md5": "8302c3e94c034f6481aacdd12b020c1d",
"sha256": "b799dab00c3c6ab52c2d82b3142f23654378eb049ac4977f324ca4875947586b"
},
"downloads": -1,
"filename": "henry-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8302c3e94c034f6481aacdd12b020c1d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6.0",
"size": 28848,
"upload_time": "2018-10-12T13:13:11",
"url": "https://files.pythonhosted.org/packages/d9/2f/d57eb5af1a8411e81cadcbf8700c866df43e7bda7f0a5b8df39bc01b82a1/henry-0.1.1-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "96bffda7229ebf8751ffa262fae1412c",
"sha256": "d677a8e270289048a65b545e9d96267dd13496674f44ee34c932a7a5b8b16f77"
},
"downloads": -1,
"filename": "henry-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "96bffda7229ebf8751ffa262fae1412c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6.0",
"size": 27974,
"upload_time": "2018-10-12T13:13:12",
"url": "https://files.pythonhosted.org/packages/e0/de/9a27f7273e15317fe4cffa6f577364d9f6b758d8258dfb162a66edc4cf54/henry-0.1.1.tar.gz"
}
],
"0.1.2": [
{
"comment_text": "",
"digests": {
"md5": "684c1a8191e2d4f5986c26e668d868de",
"sha256": "f4124d7f23eb30b67134fbc087060e44fdc5742fa65e56571f9e9f627149e5cb"
},
"downloads": -1,
"filename": "henry-0.1.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "684c1a8191e2d4f5986c26e668d868de",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 28435,
"upload_time": "2019-04-05T18:28:59",
"url": "https://files.pythonhosted.org/packages/0f/06/fcc12aec42de44e488729339d380b36e285ce61a625e82a28e130a50d27a/henry-0.1.2-py3-none-any.whl"
}
],
"0.1.3": [
{
"comment_text": "",
"digests": {
"md5": "c551b706155bb1196e3b1f67bf2c7f30",
"sha256": "ada109075afbe01a904383308e8ca2d40f3bbc7ea35fa89b7af7e6d21754cb3b"
},
"downloads": -1,
"filename": "henry-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c551b706155bb1196e3b1f67bf2c7f30",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 28458,
"upload_time": "2019-04-18T17:01:34",
"url": "https://files.pythonhosted.org/packages/c8/23/e7bf844fbbf4728178d5dca217ae7d3d6af9cc3b684e48f145ff5dec734f/henry-0.1.3-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "cd7b856611febcc567622ec2a317401b",
"sha256": "d1cb4a8c6e487abb005df4a6b8c67c7679149f77ed094897a9afc67b945f5979"
},
"downloads": -1,
"filename": "henry-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "cd7b856611febcc567622ec2a317401b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 27720,
"upload_time": "2019-04-18T17:01:36",
"url": "https://files.pythonhosted.org/packages/88/38/a9a62ea6003da62c22791466ef1e290c43ba680367009bf43cae2891829d/henry-0.1.3.tar.gz"
}
]
},
"urls": [
{
"comment_text": "",
"digests": {
"md5": "c551b706155bb1196e3b1f67bf2c7f30",
"sha256": "ada109075afbe01a904383308e8ca2d40f3bbc7ea35fa89b7af7e6d21754cb3b"
},
"downloads": -1,
"filename": "henry-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c551b706155bb1196e3b1f67bf2c7f30",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 28458,
"upload_time": "2019-04-18T17:01:34",
"url": "https://files.pythonhosted.org/packages/c8/23/e7bf844fbbf4728178d5dca217ae7d3d6af9cc3b684e48f145ff5dec734f/henry-0.1.3-py3-none-any.whl"
},
{
"comment_text": "",
"digests": {
"md5": "cd7b856611febcc567622ec2a317401b",
"sha256": "d1cb4a8c6e487abb005df4a6b8c67c7679149f77ed094897a9afc67b945f5979"
},
"downloads": -1,
"filename": "henry-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "cd7b856611febcc567622ec2a317401b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 27720,
"upload_time": "2019-04-18T17:01:36",
"url": "https://files.pythonhosted.org/packages/88/38/a9a62ea6003da62c22791466ef1e290c43ba680367009bf43cae2891829d/henry-0.1.3.tar.gz"
}
]
}