{ "info": { "author": "Rana Awais", "author_email": "rana.awais@ebryx.com", "bugtrack_url": null, "classifiers": [ "License :: OSI Approved :: GNU General Public License v2 (GPLv2)", "Operating System :: OS Independent", "Programming Language :: Python :: 3" ], "description": "# Introduction\nIt's a fork of [truffleHog](https://github.com/dxa4481/truffleHog.git), introducing more flexibility to checks. Searches through git repositories for secrets, digging deep into commit history and branches. This is effective at finding secrets accidentally committed.\n\n## Install\n```\npip install forestHog\n```\n\n## foresthog\ntruffleHog previously functioned by running entropy checks on git diffs. This functionality still exists, but high signal regex checks have been added, and the ability to surpress entropy checking has also been added.\n\nThese features help cut down on noise, and makes the tool easier to shove into a devops pipeline.\n\n\n```\nforestHog --regex https://github.com/dxa4481/truffleHog.git\n```\n\nYou can also check a repo directly from your file system:\n\n```\nforestHog file:///user/dxa4481/codeprojects/truffleHog/\n```\n\nTo enable entropy check, use following:\n```\nforestHog --regex --entropy https://github.com/dxa4481/truffleHog.git\n```\n\n![Example](https://i.imgur.com/YAXndLD.png)\n\n### Customizing\n\nCustom regexes can be added with the following flag `--rules /path/to/rules`. You can also add regexes along with default ones using `--add-rules /path/to/rules` flag. It makes it easier to extend the rule checks while using default and custom rules both. File provided by `--rules` or `--add-rules` should be a json file of the following format:\n```\n{\n \"RSA private key\": \"-----BEGIN EC PRIVATE KEY-----\"\n}\n```\nThings like subdomain enumeration, s3 bucket detection, and other useful regexes highly custom to the situation can be added.\n\nFeel free to also contribute high signal regexes upstream that you think will benefit the community. Things like Azure keys, Twilio keys, Google Compute keys, are welcome, provided a high signal regex can be constructed.\n\ntrufflehog's base rule set sources from https://github.com/dxa4481/truffleHogRegexes/blob/master/truffleHogRegexes/regexes.json \n\nYou can also check what regexes will the program check against before actually running it against your repo. This is a helpful check to make sure your custom rules/regexes are detected successfully:\n```\nforestHog --regex --show-regex \n```\nA json response will be returned. A sample is shown below:\n```\n{\n \"Slack Token\": \"(xox[p|b|o|a]-[0-9]{12}-[0-9]{12}-[0-9]{12}-[a-z0-9]{32})\",\n \"RSA private key\": \"-----BEGIN RSA PRIVATE KEY-----\",\n ...\n}\n```\n\nEntropy is checked at a minimum of 20-letter words. You can control the word-length and threshold value for the entropy checks to your liking. \n`--entropy-wc` controls the word-length. [default: 20] \n`--entropy-hex-thresh` controls the threshold for entropy calculated for hex strings. [default: 3.0] \n`--entropy-b64-thresh` controls the threshold for entropy calculated for base64 strings. [default: 4.5] \n\n### How it works\nThis module will go through the entire commit history of each branch, and check each diff from each commit, and check for secrets. This is both by regex and by entropy. For entropy checks, forestHog will evaluate the shannon entropy for both the base64 char set and hexidecimal char set for every blob of text greater than `--entropy-wc` characters comprised of those character sets in each diff. If at any point an entropy crosses the thresholds defined by `--entropy-hex-thresh` and `--entropy-b64-thresh` for a string greater than `--entropy-wc` characters, it will print to the screen.\n\n### Help Dialog\n\n```\nusage: forestHog [-h] [--json] [--show-regex] [--regex] [--rules RULES]\n [--add-rules ADD_RULES] [--entropy]\n [--entropy-wc ENTROPY_WC]\n [--entropy-b64-thresh ENTROPY_B64_THRESH]\n [--entropy-hex-thresh ENTROPY_HEX_THRESH]\n [--since-commit SINCE_COMMIT] [--max-depth MAX_DEPTH]\n [--branch BRANCH] [--repo-path REPO_PATH] [--cleanup]\n git_url\n\nFind secrets hidden in the depths of git.\n\npositional arguments:\n git_url URL for secret searching\n\noptional arguments:\n -h, --help show this help message and exit\n --json Output in JSON\n --show-regex prints out regexes that will computed against repo\n --regex Enable high signal regex checks\n --rules RULES Ignore default regexes and source from json list file\n --add-rules ADD_RULES\n Adds more regex rules along with default ones from a\n json list file\n --entropy Enable entropy checks\n --entropy-wc ENTROPY_WC\n Segments n-length words to check entropy against\n [default: 20]\n --entropy-b64-thresh ENTROPY_B64_THRESH\n User defined entropy threshold for base64 strings\n [default: 4.5]\n --entropy-hex-thresh ENTROPY_HEX_THRESH\n User defined entropy threshold for hex strings\n [default: 3.0]\n --since-commit SINCE_COMMIT\n Only scan from a given commit hash\n --max-depth MAX_DEPTH\n The max commit depth to go back when searching for\n secrets\n --branch BRANCH Name of the branch to be scanned\n --repo-path REPO_PATH\n Path to the cloned repo. If provided, git_url will not\n be used\n --cleanup Clean up all temporary result files\n```\n\n## git-forest\nAlong with `foresthog` utility that analysis the static code of your committed phase, another handy tool `git-forest` is added to integrate this code analysis with git hooks.\n### Initiate Forest Integration\nYou can start the integration with following command. By default, `post-commit` and `pre-push` events will be integrated.\n```\ngit-forest init\n```\nAfter initialization, you can change the config of your `foresthog` analysis using `git-forest update` sub-utility or by directly editing the file: `/.forest/config.json`. \nA sample of config, in JSON, is as follows:\n```\n{\n \"custom_rules\": {},\n \"entropy_wc\": 20,\n \"entropy_hex_thresh\": 3.0,\n \"entropy_b64_thresh\": 4.5,\n \"enable_default_rules\": true,\n \"enable_regex\": true,\n \"enable_entropy\": true,\n \"pre_push\": true,\n \"post_commit\": true\n}\n```\n\n### Update Forest Configuration\nYou can update them either manually or using `update` command. For example, to disable post-commit trigger:\n```\ngit-forest update --no-post-commit\n```\nSimilarly, you can change the entropy thresholds like:\n```\ngit-forest update -entropy-hex-thresh 3.3\n```\nFor `custom-rules`, you need to provide a JSON file that contain those rules or manually insert them in config file.\n```\ngit-forest update -custom-rules \n``` \n> Git hooks make use of `git-forest run` command and each event triggers a call which reads the config file on the go and analyse the code using `foresthog` utility and passes / fails the code as return value. \n\n\n\n### Destroy Forest Integration\nTo remove the integration, run the following command.\n```\ngit-forest destroy\n```\nIt will remove `.forest` directory from `` and git hooks are disconnected and backed up with `.bkp` extention in `/.git/hooks` directory.\n\n### Help Dialog\n```\nusage: git-forest [-h] [-trigger {pre-push,post-commit}]\n [-custom-rules CUSTOM_RULES] [-entropy-wc ENTROPY_WC]\n [-entropy-hex-thresh ENTROPY_HEX_THRESH]\n [-entropy-b64-thresh ENTROPY_B64_THRESH] [--no-regex]\n [--no-default-rules] [--no-entropy] [--no-pre-push]\n [--no-post-commit]\n {init,run,update,destroy}\n\npositional arguments:\n {init,run,update,destroy}\n defines foresthog operations.\n\noptional arguments:\n -h, --help show this help message and exit\n -trigger {pre-push,post-commit}\n runs forestHog for certain git event.\n -custom-rules CUSTOM_RULES\n custom rules files to check the git repo against.\n -entropy-wc ENTROPY_WC\n sets string length for calculating entropy.\n -entropy-hex-thresh ENTROPY_HEX_THRESH\n sets entropy threshold for hex strings.\n -entropy-b64-thresh ENTROPY_B64_THRESH\n sets entropy threshold for base64 strings.\n --no-regex disables regex checks for git repo.\n --no-default-rules disables default rule checks for git repo.\n --no-entropy disables entropy checks for git repo.\n --no-pre-push disables pre-push hook to git repo.\n --no-post-commit disables post-commit hook to git repo.\n```\n\n### Things to Keep in Mind\n1. `post-commit` hook should always be enabled. It will help you figure out which commit failed the code analysis and revert the commit right there. Using only `pre-push` still prevents your code to go online but you might have to revert multiple commits to get rid of sensitive data.\n2. Use custom rules, along with default ones, to cater to your specific needs. Default rules cover most widely used key patterns but it may not necessary fulfill what you actually need. If default rules are too restrictive, you can customize to ignore them and use only your custom rules / regexes.\n3. Entropy is helpful to get noisy strings from the code which can potentially be a critical information in your code. You might need to tweak the threshold values to your needs.\n\n> For organizations, it's a best practice to regularly review their public repo for leakage of critical information. `foresthog` utility can easily analyse an online repo.\n\n## Reverting Git Commits\nTo revert a last commit without losing any changes on files, it's best to use:\n```\ngit reset HEAD~\ngit reset --hard HEAD~ # changes will vanish as well.\n```\nTo revert changes on a file to last committed state, you can use:\n```\ngit checkout \n```\nTo revert last push from online repo, use the following:\n```\ngit push +HEAD^:\ngit push origin +HEAD^:master\n```\nTo filter out entire git tree, you can use:\n```\ngit filter-branch --tree-filter \ngit filter-branch --tree-filter \"rm -f \"\n```\nCheck out the documentation to learn more about this amazing utility: [filter-branch](https://git-scm.com/docs/git-filter-branch) \nAnother great tool to rewrite history is [bfg](https://github.com/rtyley/bfg-repo-cleaner). By giving it a file having regex of things you want to replace, it will spread the effect to entire git history.\n\n### Futility of Rewrites\nIf your repo had been public, there is a chance of it being cloned by someone and your critical data being disclosed, if there was any. Rewriting history / commits will prevent any leaking in future but you can't do anything about data leaked with cloned repos. \nRewriting history may sound cool but if done carelessly, it can mess up your entire git tree structure making it unable to be used any further.\n>*Use it with great care and use it only as a last resort.*\n\nLastly, rewriting history means you're making a change in a branch. You will have to notify your coworkers to have them clean their branches as well to make a seemless git structure. Otherwise, they will face errors upon merging and pushing code.\n\n## Features Wish-list\nFollowing improvements can be done in future. \n- `pre-commit` hook should be added to avoid committing sensitive data. Currently, `post-commit` and `pre-push` are supported.\n- Some sort of whitelisting mechanism should be introduced so that certain regex patterns, even if found, can be whitelisted forcibly.\n\n\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/EbryxLabs/forestHog", "keywords": "", "license": "GNU", "maintainer": "", "maintainer_email": "", "name": "forestHog", "package_url": "https://pypi.org/project/forestHog/", "platform": "", "project_url": "https://pypi.org/project/forestHog/", "project_urls": { "Homepage": "https://github.com/EbryxLabs/forestHog" }, "release_url": "https://pypi.org/project/forestHog/1.3.6/", "requires_dist": [ "GitPython (==2.1.1)", "truffleHogRegexes (==0.0.7)" ], "requires_python": "", "summary": "Searches through git repositories for high entropy strings, digging deep into commit history.", "version": "1.3.6" }, "last_serial": 5343607, "releases": { "0.9": [ { "comment_text": "", "digests": { "md5": "51068c0a91333e981ef61a2b971847f1", "sha256": "f9e42bd1ed2b3ab78b7238272f038779d96f2336ddc0ad8587f874865aaaedab" }, "downloads": -1, "filename": "forestHog-0.9-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "51068c0a91333e981ef61a2b971847f1", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 12470, "upload_time": "2019-04-05T11:38:02", "url": "https://files.pythonhosted.org/packages/b9/c0/fc309c14fb76c501c5ddf52592989b0d327b05180f215d1f5df3f4638560/forestHog-0.9-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2a364c27110e53c2afe0128a997c290e", "sha256": "88ddd8d0872ef74d790a03dce4b6a17f5d7ce4c9d9541ac54d2e82f32e4dd3c9" }, "downloads": -1, "filename": "forestHog-0.9.tar.gz", "has_sig": false, "md5_digest": "2a364c27110e53c2afe0128a997c290e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6643, "upload_time": "2019-04-05T11:38:04", "url": "https://files.pythonhosted.org/packages/04/55/5be6489e16af2d3f7d59bcbbf782b992f0577b1e4a30a95bcb68882e8d8a/forestHog-0.9.tar.gz" } ], "1.0": [ { "comment_text": "", "digests": { "md5": "55e210eb3c98300f4435d5fa98c1dc00", "sha256": "934e67bc50da31318d5586fe32928f2fbe826319b3691ab639376fc98da79353" }, "downloads": -1, "filename": "forestHog-1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "55e210eb3c98300f4435d5fa98c1dc00", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 14642, "upload_time": "2019-04-05T11:45:46", "url": "https://files.pythonhosted.org/packages/43/ae/e73572cc3f41143216d8b76c21a9602058336c4ed0d03a5c484c9af2d856/forestHog-1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "150c6abb5345f8029797b88807b01ee3", "sha256": "a86c7f15f93a31c8fcdccae1a982d1f22c6521128d2a1969c9d9cfd97019f9cc" }, "downloads": -1, "filename": "forestHog-1.0.tar.gz", "has_sig": false, "md5_digest": "150c6abb5345f8029797b88807b01ee3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7266, "upload_time": "2019-04-05T11:45:48", "url": "https://files.pythonhosted.org/packages/5f/bf/369f607a20908856a6f8259681acdadc5350c7208c1e18b723004eecc2c2/forestHog-1.0.tar.gz" } ], "1.1": [ { "comment_text": "", "digests": { "md5": "ab08d979df73618899cc0060240bbe2a", "sha256": "da515c7b46c448b082a5ef2b05473ee78f4c04d68e8e5c217b75ed80da054610" }, "downloads": -1, "filename": "forestHog-1.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ab08d979df73618899cc0060240bbe2a", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 14666, "upload_time": "2019-04-05T11:52:03", "url": "https://files.pythonhosted.org/packages/3a/fd/8b5714c89a9add796fb5f710f004429f18d46c8669da597dedf5e077efeb/forestHog-1.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "b08baeb3626b7ca07cb81fb0bb6f3c2c", "sha256": "f1cdea3bbc5fe782e2a0498e8cc9978a4a1e51d7fdcd82ab82b6626c165bbd73" }, "downloads": -1, "filename": "forestHog-1.1.tar.gz", "has_sig": false, "md5_digest": "b08baeb3626b7ca07cb81fb0bb6f3c2c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7301, "upload_time": "2019-04-05T11:52:05", "url": "https://files.pythonhosted.org/packages/56/96/0a414bbb810cbc79074848eb99b9c2feb3967bf1492f82f6e8afa7f60bff/forestHog-1.1.tar.gz" } ], "1.2": [ { "comment_text": "", "digests": { "md5": "1b35685ab3f208ffb4b59534b4c585c7", "sha256": "12f5a21f2bec2d245e777f84c6fe88ca854c06b4c94cea61751ba8dd0d5f0486" }, "downloads": -1, "filename": "forestHog-1.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "1b35685ab3f208ffb4b59534b4c585c7", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19089, "upload_time": "2019-04-06T15:36:05", "url": "https://files.pythonhosted.org/packages/7b/b6/de9366d1c8c14934a6deec835905e8418665eaeaecc324ccc684190101d9/forestHog-1.2-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "3e5df4891beda1aadf6968e8367f45bc", "sha256": "c44a29c72a2f7c68fbe1c84bcaf5a2632f28fbc71a7aaa34334e78622d5030b4" }, "downloads": -1, "filename": "forestHog-1.2.tar.gz", "has_sig": false, "md5_digest": "3e5df4891beda1aadf6968e8367f45bc", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15454, "upload_time": "2019-04-06T15:36:06", "url": "https://files.pythonhosted.org/packages/7e/f9/1945d3295138f725a086deb401aa33fdfffc6ae0a3d3af4ae3d634f1e2f4/forestHog-1.2.tar.gz" } ], "1.3": [ { "comment_text": "", "digests": { "md5": "85ca2eb73e2ce0c2cb53be9c17f9d2bf", "sha256": "cc065eeb8f33c6b41da0627bca0bd1c25a42b8098af09a3a493f1537e7fcafd0" }, "downloads": -1, "filename": "forestHog-1.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "85ca2eb73e2ce0c2cb53be9c17f9d2bf", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19310, "upload_time": "2019-05-31T13:44:19", "url": "https://files.pythonhosted.org/packages/01/90/1a7c327507a1d62bcd0c63cf64792738f55ed63e1a88d6b2fbf723f71ee2/forestHog-1.3-py2.py3-none-any.whl" } ], "1.3.1": [ { "comment_text": "", "digests": { "md5": "9456197c21c1e1d57afb8a0f10e47d28", "sha256": "f301486c2cdcf01ebab0c8c907ee03b7bb17fcc706c4a566ec5b4bf667c4f830" }, "downloads": -1, "filename": "forestHog-1.3.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "9456197c21c1e1d57afb8a0f10e47d28", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19387, "upload_time": "2019-05-31T13:57:28", "url": "https://files.pythonhosted.org/packages/97/8b/b2541319102cbcb3f5936739435edb4843bbf6d918b2c768a8198a31715d/forestHog-1.3.1-py2.py3-none-any.whl" } ], "1.3.2": [ { "comment_text": "", "digests": { "md5": "752be63d8e8fe92335c6c3fb9c7aa1f6", "sha256": "f3a4a91b754ef354ba02e84bfc957021e171375845a06fa084275735e6af043b" }, "downloads": -1, "filename": "forestHog-1.3.2-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "752be63d8e8fe92335c6c3fb9c7aa1f6", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19392, "upload_time": "2019-05-31T14:31:34", "url": "https://files.pythonhosted.org/packages/53/6e/30bd9571c64a46c2a311bce630596d58d1cb70877ee1cffa41dba8dc3f7b/forestHog-1.3.2-py2.py3-none-any.whl" } ], "1.3.3": [ { "comment_text": "", "digests": { "md5": "a89b9a125ac986da573374cbaa288cdd", "sha256": "3520a7e519d1c51047c2b052aa38d3de8c2be9609f837a58ebd734a03fafbe8a" }, "downloads": -1, "filename": "forestHog-1.3.3-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "a89b9a125ac986da573374cbaa288cdd", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19370, "upload_time": "2019-05-31T14:47:13", "url": "https://files.pythonhosted.org/packages/77/fd/fae39e410de453faf08cf6b96e7a17f7c7c16aff007c45c9947b3b5b90fe/forestHog-1.3.3-py2.py3-none-any.whl" } ], "1.3.4": [ { "comment_text": "", "digests": { "md5": "7d4664e7b9a34fcc2b5dbb61386b860a", "sha256": "a7df26dd6f3721ca43d04e4a2d759ce096d0d546c7f4c60fe9e50384380f2aff" }, "downloads": -1, "filename": "forestHog-1.3.4-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "7d4664e7b9a34fcc2b5dbb61386b860a", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19383, "upload_time": "2019-05-31T14:55:44", "url": "https://files.pythonhosted.org/packages/17/54/47d344c0cf8dd8bc03f1c54363e860f4acd2b9ff44607ec2f4e9380d12e1/forestHog-1.3.4-py2.py3-none-any.whl" } ], "1.3.5": [ { "comment_text": "", "digests": { "md5": "cd9e3864d5b057c1ed8a88224b10ae9d", "sha256": "d7df8e52487a5896ee716315316a86ed8d0d48b6bc6c7cbec705f04388b89486" }, "downloads": -1, "filename": "forestHog-1.3.5-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "cd9e3864d5b057c1ed8a88224b10ae9d", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19406, "upload_time": "2019-05-31T16:03:04", "url": "https://files.pythonhosted.org/packages/6f/a9/5363c9ea8fd25c1b98f9647427dcdcd097e7d987827e43ee86714ec5ec48/forestHog-1.3.5-py2.py3-none-any.whl" } ], "1.3.6": [ { "comment_text": "", "digests": { "md5": "b736d3326ac2f116a8762395480c548b", "sha256": "0dd12de29eb4fcf19350b71b7f4478b88a8ece9be10ff186445558019ef04c0d" }, "downloads": -1, "filename": "forestHog-1.3.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b736d3326ac2f116a8762395480c548b", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19419, "upload_time": "2019-05-31T16:39:14", "url": "https://files.pythonhosted.org/packages/e0/42/9e5c0a1b2ca774ae2863d934673f6fadfd7095acbeebcba3447a39446aca/forestHog-1.3.6-py2.py3-none-any.whl" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "b736d3326ac2f116a8762395480c548b", "sha256": "0dd12de29eb4fcf19350b71b7f4478b88a8ece9be10ff186445558019ef04c0d" }, "downloads": -1, "filename": "forestHog-1.3.6-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "b736d3326ac2f116a8762395480c548b", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 19419, "upload_time": "2019-05-31T16:39:14", "url": "https://files.pythonhosted.org/packages/e0/42/9e5c0a1b2ca774ae2863d934673f6fadfd7095acbeebcba3447a39446aca/forestHog-1.3.6-py2.py3-none-any.whl" } ] }