{ "info": { "author": "Will Rubel", "author_email": "willrubel@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Natural Language :: English", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6" ], "description": "Cloudformation-Validator\n========================\n\nFeatures\n========\n\ncloudformation\\_validator provides type checking and other base\nfunctionality out of the box and is designed to be non-blocking and\neasily extensible, allowing for custom validation. It has no\ndependencies and is thoroughly tested under Python 2.7, Python 3.3,\nPython 3.4, Python 3.5, Python 3.6.\n\nFunding\n=======\n\ncloudformation\\_validator is a open source, collaboratively funded\nproject. If you run a business and are using cloudformation\\_validator\nin a revenue-generating product, it would make business sense to sponsor\nits development: it ensures the project that your product relies on\nstays healthy and actively maintained. Individual users are also welcome\nto make a recurring pledge or a one time donation if cfn-validator has\nhelped you in your work or personal projects.\n\nEvery single sign-up makes a significant impact towards making\ncloudformation\\_validator possible.\n\nWant Custom Rules and Support For Your Application\n==================================================\n\nSubmit an issue on my github page if you would like additional custom\nrules and I will try and get them added as soon as possible.\n\nI you would like other functionality, just submit an issue and I will\nsee what I can do to get it added.\n\nInstallation\n============\n\ncloudformation-validator is on PyPI so all you need is:\n\n``` {.sourceCode .console}\n$ pip install cfn-validator\n```\n\n\nDemonstration\n=============\n\n
\n\n\nJust run:\n\n``` {.sourceCode .console\n$ pip install virtualenv\n$ which python\n$ virtualenv ~/virtualenvs/my_project -p /home/example_username/opt/python-3.6.2/bin/python3\n$ git clone https://github.com/rubelw/cloudformation-validator.git\n$ cd cloudformation-validator\n$ pip install -r requirements-dev.txt\n$ python setup.py install --force\n$ python setup.py test}\n```\n\nOr you can use tox to run the tests under all supported Python versions.\nMake sure the required python versions are installed and run:\n\n``` {.sourceCode .console\n$ pip install virtualenv\n$ which python\n$ virtualenv ~/virtualenvs/my_project -p /home/example_username/opt/python-3.6.2/bin/python3\n$ git clone https://github.com/rubelw/cloudformation-validator.git\n$ cd cloudformation-validator\n$ pip install -r requirements-dev.txt\n$ python setup.py install --force\n$ pip install tox # first time only\n$ tox}\n```\n\nListing Rules\n=============\n\n``` {.sourceCode .console}\n$ cfn-validator dump_rules\n##################################\n########## WARNINGS ##############\n##################################\n{'id': 'F4', 'type': 'VIOLATION::WARNING', 'message': 'IAM policy should not allow * action'}\n{'id': 'W1', 'type': 'VIOLATION::WARNING', 'message': 'Specifying credentials in the template itself is probably not the safest thing'}\n...\n```\n\nExample\n=======\n\nGetting help\n\n``` {.sourceCode .console}\n$ cfn-validator validate --help\nUsage: cfn-validator validate [OPTIONS]\n\n primary function for validating a template :param template_path: :param\n template_file: :param debug: :param rules_directory: :param profile_path:\n :param allow_suppression: :param print_suppression: :param\n parameter_values_path: :param isolate_custom_rule_exceptions: :param\n version: :return:\n\nOptions:\n -s, --suppress-errors Whether to suppress misc errors to get hash only\n -t, --template-path TEXT base directory to search for templates\n -f, --template-file TEXT single_template_file\n --debug Turn on debugging\n -r, --rules-directory TEXT Extra rule directory\n -o, --profile-path TEXT Path to a profile file\n --allow-suppression / --no-allow-suppression\n Allow using Metadata to suppress violations\n -p, --print-suppression Emit suppressions to stderr\n -m, --parameter-values-path TEXT\n Path to a JSON file to pull Parameter values\n from\n -i, --isolate-custom-rule-exceptions\n Isolate custom rule exceptions - just emit\n the exception without stack trace and keep\n chugging\n -v, --version Print version and exit\n --help Show this message and exit.\n```\n\nValidate a file\n\n``` {.sourceCode .console}\n$cfn-validator validate -f cloudfront_distribution_without_logging.json\n\nEvaluating: cloudfront_distribution_without_logging.json\n[\n {\n 'failure_count': '0',\n 'filename': 'cloudfront_distribution_without_logging.json',\n 'file_results': [\n {\n 'id': 'W10',\n 'type': 'VIOLATION::WARNING',\n 'message': 'CloudFront Distribution should enable access logging',\n 'logical_resource_ids': [\n 'rDistribution2'\n ]\n }\n ]\n }\n]\n```\n\nValidate all files in a path\n\n``` {.sourceCode .console}\n$cfn-validator validate -f /projects\n...\n```\n\nProgrammatically call cfn-validator to analyze a file\n\n``` {.sourceCode .console}\nfrom cloudformation_validator.ValidateUtility import ValidateUtility\n\nconfig_dict = {}\nconfig_dict['template_file'] = '/tmp/template.json'\nvalidator = ValidateUtility(config_dict)\nreal_result = validator.validate()\nprint(real_result)\n\n[\n {\n 'failure_count': '0',\n 'filename': '/tmp/template.json',\n 'file_results': [\n {\n 'id': 'W1',\n 'type': 'VIOLATION::WARNING',\n 'message': 'Specifying credentials in the template itself is probably not the safest thing',\n 'logical_resource_ids': [\n 'EC2I4LBA1'\n ]\n }\n ]\n }\n]\n```\n\nI you get some errors and warnings in your out put, you can pass-in the\nflag to suppress all errors\n\n``` {.sourceCode .console}\nfrom cloudformation_validator.ValidateUtility import ValidateUtility\n\nconfig_dict = {}\nconfig_dict['suppress_errors'] = True\nconfig_dict['template_file'] = '/tmp/template.json'\nvalidator = ValidateUtility(config_dict)\nreal_result = validator.validate()\nprint(real_result)\n\n[\n {\n 'failure_count': '0',\n 'filename': '/tmp/template.json',\n 'file_results': [\n {\n 'id': 'W1',\n 'type': 'VIOLATION::WARNING',\n 'message': 'Specifying credentials in the template itself is probably not the safest thing',\n 'logical_resource_ids': [\n 'EC2I4LBA1'\n ]\n }\n ]\n }\n]\n```\n\nWriting your own rules\n\n> - Utilize the format for existing rules in the\n> /cloudformation\\_validator/custom\\_rules directory\n> - Places the files in a new directory\n> - The \\_\\_init\\_\\_, rule\\_text, rule\\_type and rule\\_id methods\n> should be amount the same, just change of the rule, the text for a\n> failure, and the type to either \\'VIOLATION::FAILING\\_VIOLATION\\'\n> or VIOLATION::WARNNING\\'\n> - Set the id to \\'W\\' for warnings, and \\'F\\' for failure. Pick a\n> number not utilized elsewhere\\...\n> - NOTE: Currently working on functionality for controlling and\n> listing rules\n> - For the audit\\_impl function - portion with will test the resource\n> objects, you will need to review the object model for the resource\n> to see what objects are available, and then review the parser for\n> the resource. Also, look at other similar rules for the resource,\n> and model after them. The basic concept of the function is to\n> identify resources which apply, iterate over the selected\n> resources, and identify specific aspects to evaluate in the rule\n> - pass in the \\--rules-directory /directory in the command line, and\n> the extra rules directory will get added to the existing rules\n\n``` {.sourceCode .console}\ndef audit_impl(self):\n\n violating_rules = []\n\n # This defines which type of resource we are going to test\n resources = self.cfn_model.resources_by_type('AWS::SQS::QueuePolicy')\n\n if len(resources)>0:\n for resource in resources:\n if hasattr(resource, 'policy_document'):\n if resource.policy_document:\n if resource.policy_document.wildcard_allowed_actions():\n violating_rules.append(resource.logical_resource_id)\n\n return violating_rules\n```\n\nExample of writing a rule which requires custom tags for EC2 instances\n======================================================================\n\n- Create a directory to store your custom rule\n- Create the custom rule\n\n``` {.sourceCode .console}\nmkdir ~/custom_validator_rules\n```\n\n``` {.sourceCode .console}\nfrom __future__ import absolute_import, division, print_function\nimport inspect\nimport sys\nfrom builtins import (str)\nfrom cloudformation_validator.custom_rules.BaseRule import BaseRule\nfrom collections import Iterable\nfrom six import StringIO, string_types\nfrom builtins import (str)\n\nclass Ec2CustomTagsRule(BaseRule):\n\n def __init__(self, cfn_model=None, debug=None):\n '''\n Initialize Ec2HasTagsRule\n :param cfn_model:\n '''\n BaseRule.__init__(self, cfn_model, debug=debug)\n\n def rule_text(self):\n '''\n Returns rule text\n :return:\n '''\n if self.debug:\n print('rule_text')\n return 'EC2 instance does not have the required tags'\n\n def rule_type(self):\n '''\n Returns rule type\n :return:\n '''\n self.type= 'VIOLATION::FAILING_VIOLATION'\n return 'VIOLATION::FAILING_VIOLATION'\n\n def rule_id(self):\n '''\n Returns rule id\n :return:\n '''\n if self.debug:\n print('rule_id')\n self.id ='F86'\n return 'F86'\n\n def tags_to_dict(self, aws_tags):\n \"\"\" Convert a list of AWS tags into a python dict \"\"\"\n return {str(tag['Key']): str(tag['Value']) for tag in self.ensure_list(aws_tags)}\n\n def ensure_list(self, value):\n \"\"\"\n Coerces a variable into a list; strings will be converted to a singleton list,\n and `None` or an empty string will be converted to an empty list.\n Args:\n value: a list, or string to be converted into a list.\n\n Returns:\n :py:class:`list`\n \"\"\"\n ret_value = value\n if not value:\n ret_value = []\n elif not isinstance(value, Iterable) or isinstance(value, string_types):\n ret_value = [value]\n return ret_value\n\n\n def audit_impl(self):\n '''\n Audit\n :return: violations\n '''\n if self.debug:\n print('Ec2HasTagsRule - audit_impl')\n\n violating_volumes = []\n\n resources = self.cfn_model.resources_by_type('AWS::EC2::Instance')\n\n if len(resources) > 0:\n\n for resource in resources:\n if self.debug:\n print('resource: ' + str(resource))\n print('vars: '+str(vars(resource)))\n\n if hasattr(resource, 'tags'):\n tags_dict = self.tags_to_dict(resource.cfn_model['Properties']['Tags'])\n required_tags = ('Name', 'ResourceOwner','DeployedBy','Project')\n if not set(required_tags).issubset(tags_dict):\n violating_volumes.append(str(resource.logical_resource_id))\n else:\n if self.debug:\n print('does not tags property')\n violating_volumes.append(str(resource.logical_resource_id))\n\n else:\n if self.debug:\n print('no violating_volumes')\n\n return violating_volumes\n```\n\n- Test the rule by creating a cloudformation template without the\n necessary tags and testing\n\n``` {.sourceCode .console}\n{\n \"Parameters\": {\n \"subnetId\": {\n \"Type\": \"String\",\n \"Default\": \"subnet-4fd01116\"\n }\n },\n\n \"Resources\": {\n \"EC2I4LBA1\": {\n \"Type\": \"AWS::EC2::Instance\",\n \"Properties\": {\n \"ImageId\": \"ami-6df1e514\",\n \"InstanceType\": \"t2.micro\",\n \"SubnetId\": {\n \"Ref\": \"subnetId\"\n }\n },\n \"Metadata\": {\n \"AWS::CloudFormation::Authentication\": {\n \"testBasic\" : {\n \"type\" : \"basic\",\n \"username\" : \"biff\",\n \"password\" : \"badpassword\",\n \"uris\" : [ \"http://www.example.com/test\" ]\n }\n }\n }\n }\n }\n}\n```\n\n- Run the test\n\n``` {.sourceCode .console\ncfn-validator validate --template-file=/tmp/template.json --rules-directory=/home/user/custom_validator_rules}\n```\n\n- You should receive the following violations\n\n``` {.sourceCode .console}\n{\n 'failure_count': '1',\n 'filename': '/tmp/template.json',\n 'file_results': [\n {\n 'id': 'F86',\n 'type': 'VIOLATION::FAILING_VIOLATION',\n 'message': 'EC2 instance does not have the required tags',\n 'logical_resource_ids': [\n 'EC2I4LBA1'\n ]\n },\n {\n 'id': 'W1',\n 'type': 'VIOLATION::WARNING',\n 'message': 'Specifying credentials in the template itself is probably not the safest thing',\n 'logical_resource_ids': [\n 'EC2I4LBA1'\n ]\n }\n ]\n}\n```\n\n- No add tags property to the cloudformation template and run again\n\n``` {.sourceCode .console\n{\n\"Parameters\": {\n\"subnetId\": {\n\"Type\": \"String\",\n\"Default\": \"subnet-4fd01116\"\n}\n},}\n\"Resources\": {\n \"EC2I4LBA1\": {\n \"Type\": \"AWS::EC2::Instance\",\n \"Properties\": {\n \"ImageId\": \"ami-6df1e514\",\n \"InstanceType\": \"t2.micro\",\n \"SubnetId\": {\n \"Ref\": \"subnetId\"\n },\n \"Tags\" : [\n {\"Key\" : \"Name\", \"Value\":\"value\"},\n {\"Key\":\"ResourceOwner\",\"Value\":\"resourceowner\"},\n {\"Key\":\"DeployedBy\",\"Value\":\"deployedby\"},\n {\"Key\":\"Project\",\"Value\":\"project\"}\n ]\n },\n \"Metadata\": {\n \"AWS::CloudFormation::Authentication\": {\n \"testBasic\" : {\n \"type\" : \"basic\",\n \"username\" : \"biff\",\n \"password\" : \"badpassword\",\n \"uris\" : [ \"http://www.example.com/test\" ]\n }\n }\n }\n }\n}\n```\n\n> }\n\n- You should receive the following violations\n\n``` {.sourceCode .console\n{\n'failure_count': '0',\n'filename': '/tmp/template.json',\n'file_results': [\n{\n'id': 'W1',\n'type': 'VIOLATION::WARNING',\n'message': 'Specifying credentials in the template itself is probably not the safest thing',\n'logical_resource_ids': [\n'EC2I4LBA1'\n]\n}\n]\n}}\n```\n\nUnit Testing\n============\n\nRun unit tests\n\n``` {.sourceCode .console}\n(python3) => pytest\n================================================ test session starts =================================================\ncollected 22 items\n\ntest/test_cloudfront_distribution.py . [ 4%]\ntest/test_ec2_instance.py . [ 9%]\ntest/test_ec2_volume.py .. [ 18%]\ntest/test_elasticloadbalancing_loadbalancer.py . [ 22%]\ntest/test_iam_user.py . [ 27%]\ntest/test_lambda_permission.py . [ 31%]\ntest/test_rds_instance.py ... [ 45%]\ntest/test_s3_bucket.py . [ 50%]\ntest/test_s3_bucket_policy.py . [ 54%]\ntest/test_security_group.py ........ [ 90%]\ntest/test_sns_policy.py . [ 95%]\ntest/test_sqs_policy.py . [100%]\n```\n\nSource\n======\n\nI am just getting started on this, so any suggestions would be welcome.\n\\<