{ "info": { "author": "Evan Borgstrom", "author_email": "eborgstrom@nerdwallet.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Topic :: Software Development :: Libraries", "Topic :: Software Development :: Pre-processors", "Topic :: System :: Systems Administration" ], "description": "Terrafompy\n==========\n\nTerraformpy is a library and command line tool to supercharge your Terraform configs using a full fledged Python environment!\n\n`Terraform`_ is an amazing tool. Like, really amazing. When working with code that is managing third-party service definitions, and actually applying changes to those definitions by invoking APIs, a high-degree of confidence in the change process is a must-have, and that's where Terraform excels. The work flow it empowers allow teams to quickly make changes across a large (and ever growing) footprint in multiple providers/regions/technologies/etc.\n\nBut as your definitions grow the `HCL`_ syntax very quickly leaves a lot to be desired, and is it ever verbose... So many definitions of variables and outputs need to be repeated, over and over, as you compose more modules that use each other.\n\nSince `HCL`_ is \"fully JSON compatible\" and Python is great at generating JSON data, we built Terraformpy to provide a more productive environment to build and maintain complex Terraform configs. It has been used daily in production at `NerdWallet`_ since 2016 and has proven very valuable in accelerating our adoption of Terraform across our engineering organization.\n\n.. _Terraform: https://www.terraform.io\n.. _HCL: https://github.com/hashicorp/hcl\n.. _NerdWallet: https://www.nerdwallet.com\n\n\nInstalling Terraformpy\n----------------------\n\nThe recommended way to install and use Terraformpy is via `Pipenv`_\n\nAn example would look like:\n\n.. code-block:: bash\n\n $ mkdir my-terraform-project\n $ cd my-terraform-project\n $ pipenv install --two terraformpy\n\nYou can then run Terraformpy using ``pipenv run``:\n\n.. code-block:: bash\n\n $ pipenv run terraformpy ...\n\nOr you can use ``pipenv shell`` to activate the virtualenv so you don't need to use ``pipenv run``. The rest of this document assumes that you've run ``pipenv shell`` and can just run ``terraformpy`` directly.\n\n.. _Pipenv: https://docs.pipenv.org/en/latest/\n\nUsing the CLI tool\n------------------\n\nThe ``terraformpy`` command line tool operates as a shim for the underlying ``terraform`` tool. When invoked it will first find all ``*.tf.py`` files in the current directory, loading them using the `imp`_ module, generate a file named ``main.tf.json``, and then invoke underlying tool.\n\n.. code-block:: bash\n\n # just replace terraform in your regular workflow\n terraformpy plan -out=tf.plan\n\n # review changes...\n\n # apply them!\n # since we're going to operate on the generated plan here, we don't event need to use terraformpy anymore\n terraform apply tf.plan\n\n\nEach of the ``*.tf.py`` files uses a declarative syntax, using objects imported from this library. You don't need to define a main function, you just create instances of classes (anonymous or otherwise) in the root of the module (you're building regular Python code here). Since you're in a full blown Python environment there is no limit on what you can do -- import things, connect to databases, etc.\n\n.. _imp: https://docs.python.org/3/library/imp.html\n\n\nWriting ``.tf.py`` files\n------------------------\n\nThe ``terraformpy`` name space provides a number of classes that map directly to things you declare in normal ``.tf.`` files. To write your definitions simply import these classes and begin creating instances of them. Below is the first example from the `Terraform getting start guide`_.\n\n.. _Terraform getting start guide: https://learn.hashicorp.com/terraform/getting-started/build.html#configuration\n\n.. code-block:: python\n\n from terraformpy import Provider, Resource\n\n Provider(\n 'aws',\n profile='default',\n region='us-east-1'\n )\n\n Resource(\n 'aws_instance', 'example',\n ami='ami-2757f631'\n instance_type='t2.micro'\n )\n\n\nThings you can import from ``terraformpy``:\n\n* ``Provider``\n* ``Variable``\n* ``Data``\n* ``Resource``\n* ``Output``\n\nSee the ``examples/`` dir for fully functional examples.\n\n\nInterpolation\n-------------\n\nSo far, we've only used terraformpy anonymously, but the returned instances of the ``Data`` and ``Resource`` classes offer handy interpolation attributes. For example, a common task is using the ``Data`` class to fetch remote data:\n\n.. code-block:: python\n\n ami = Data(\n 'aws_ami', 'ecs_ami',\n most_recent=True,\n filter=[\n dict(name='name', values=['\\*amazon-ecs-optimized']),\n dict(name='owner-alias', values=['amazon'])\n ]\n )\n\n Resource(\n 'aws_instance', 'example',\n ami=ami.id,\n instance_type='m4.xlarge'\n )\n\nHere we simply refer to the id attribute on the ami object when creating the ``aws_instance``. During the compile phase it would be converted to the correct syntax: ``\"${data.aws_ami.ecs_ami.id}\"``.\n\nThis works by having a custom ``__getattr__`` function on our ``Data`` and ``Resource`` objects that will turn any attribute access for an attribute name that doesn't exist into the Terraform interpolation syntax.\n\n\nModules\n-------\n\nModules have been explicitly excluded from this implementation because they aim to solve the same problem -- building reusable blocks in your Terraform configs.\n\nWith all the features of Python at your disposal building reusable units is straightforward without using the native modules from Terraform, but do see Resource Collections (next) for some helper scaffolding!\n\n\nResource Collections\n--------------------\n\nA common pattern when building configs using Python is to want to abstract a number of different resources under the guise of a single object -- which is the same pattern native Terraform modules aim to solve. In terraformpy we provide a ``ResourceCollection`` base class for building objects that represent multiple resources.\n\nYou can use `Schematics`_ to define the fields and perform validation.\n\nAs an example, when provisioning an RDS cluster you may want to have a standard set of options that you ship with all your clusters. You can express that with a resource collection:\n\n\n.. _Schematics: https://schematics.readthedocs.io/en/latest/\n\n.. code-block:: python\n\n from schematics import types\n from schematics.types import compound\n from terraformpy import Resource, ResourceCollection\n\n\n class RDSCluster(ResourceCollection):\n\n # Defining attributes of your resource collection is like defining a Schematics Model, in fact the\n # ResourceCollection class is just a specialized subclass of the Schematics Model class.\n #\n # Each attribute becomes a field on the collection, and can be provided as a keyword when constructing\n # an instance of your collection.\n #\n # Validation works the same as in Schematics. You can attach validators to the fields themselves and\n # also define \"validate_field\" functions.\n\n name = types.StringType(required=True)\n azs = compound.ListType(types.StringType, required=True)\n instance_class = types.StringType(required=True, choices=('db.r3.large', ...))\n\n # The create_resources function is invoked once the instance has been created and the kwargs provided have been\n # processed against the inputs. All of the instance attributes have been converted to the values provided, so\n # if you access self.name in create_resources you're accessing whatever value was provided to the instance\n\n def create_resources(self):\n self.param_group = Resource(\n 'aws_rds_cluster_parameter_group', '{0}_pg'.format(self.name),\n family='aurora5.6',\n parameter=[\n {'name': 'character_set_server', 'value': 'utf8'},\n {'name': 'character_set_client', 'value': 'utf8'}\n ]\n )\n\n self.cluster = Resource(\n 'aws_rds_cluster', self.name,\n cluster_identifier=self.name,\n availability_zones=self.azs,\n database_name=self.name,\n master_username='root',\n master_password='password',\n db_cluster_parameter_group_name=self.param_group.id\n )\n\n self.instances = Resource(\n 'aws_rds_cluster_instance', '{0}_instances'.format(self.name),\n count=2,\n identifier='{0}-${{count.index}}'.format(self.name),\n cluster_identifier=self.cluster.id,\n instance_class=self.instance_class\n )\n\n\nThat definition can then be imported and used in your terraformpy configs.\n\n.. code-block:: python\n\n from modules.rds import RDSCluster\n\n\n cluster1 = RDSCluster(\n name='cluster1',\n azs=['us-west-2a','us-west-2b','us-west-2c'],\n instance_class='db.r3.large'\n )\n\n # you can then refer to the resources themselves, for interpolation, through the attrs\n # i.e. cluster1.cluster.id\n\n\nVariants\n--------\n\nResource definitions that exist across many different environments often only vary slightly between each environment. To facilitate the ease of definition for these differences you can use variant grouping.\n\nFirst create the folders: ``configs/stage/``, ``configs/prod/``, ``configs/shared/``. Inside each of them place a ``__init__.py`` to make them packages.\n\nNext create the file ``configs/shared/instances.py``:\n\n.. code-block:: python\n\n from terraformpy import Resource\n\n Resource(\n 'aws_instance', 'example',\n ami=ami.id,\n prod_variant=dict(\n instance_type='m4.xlarge'\n ),\n stage_variant=dict(\n instance_type='t2.medium'\n )\n )\n\nThen create ``configs/stage/main.tf.py``:\n\n.. code-block:: python\n\n from terraformpy import Variant\n\n with Variant('stage'):\n import configs.shared.instances\n\nSince the import of the instances file happens inside of the Variant context then the Resource will be created as if it had been defined like:\n\n.. code-block:: python\n\n from terraformpy import Resource\n\n Resource(\n 'aws_instance', 'example',\n ami=ami.id,\n instance_type='t2.medium'\n )\n\n\nMultiple providers\n------------------\n\nDepending on your usage of Terraform you will likely end up needing to use multiple providers at some point in time. To use `multiple providers in Terraform`_ you define them using aliases and then reference those aliases in your resource definitions.\n\nTo make this pattern easier you can use the Terraformpy ``Provider`` object as a context manager, and then any resources created within the context will automatically have that provider aliases referenced:\n\n.. code-block:: python\n\n from terraformpy import Resource, Provider\n\n with Provider(\"aws\", region=\"us-west-2\", alias=\"west2\"):\n sg = Resource('aws_security_group', 'sg', ingress=['foo'])\n\n assert sg.provider == 'aws.west2'\n\n.. _multiple providers in Terraform: https://www.terraform.io/docs/configuration/providers.html#alias-multiple-provider-instances\n\n\nUsing file contents\n-------------------\n\nOften times you will want to include the contents of a file that is located alongside your Python code, but when running ``terraform`` along with the ``${file('myfile.json')}`` interpolation function pathing will be relative to where the compiled ``main.tf.json`` file is and not where the Python code lives.\n\nTo help with this situation a function named ``relative_file`` inside of the ``terraformpy.helpers`` namespace is provided.\n\n.. code-block:: python\n\n from terraformpy import Resource\n from terraformpy.helpers import relative_file\n\n Resource(\n 'aws_iam_role', 'role_name',\n name='role-name',\n assume_role_policy=relative_file('role_policy.json')\n )\n\nThis would produce a definition that leverages the ``${file(...)}`` interpolation function with a path that reads the ``role_policy.json`` file from the same directory as the Python code that defined the role.\n\n\nNotes and Gotchas\n=================\n\nSecurity Group Rules and ``self``\n----------------------------------\n\nWhen creating ``aws_security_group_rule`` ``Resource`` objects you cannot pass ``self=True`` to the object since Python already passes a ``self`` argument into the constructor. In this case you'll need to specify it directly in the ``_values``:\n\n.. code-block:: python\n\n sg = Resource(\n 'aws_security_group_rule', 'my_rule',\n _values=dict(self=True),\n vpc_id=vpc.id,\n ...\n )\n\nRelease Steps\n=================\n1. Create an issue, check out a branch, and make your code changes.\n2. Push to run CircleCI tests.\n3. Create Pull Request to Master including VERSION bump.\n4. Merge PR after Approval.\n5. Add tag like v1.0.0 that matches new version and push.\n\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "", "keywords": "", "license": "", "maintainer": "", "maintainer_email": "", "name": "terraformpy", "package_url": "https://pypi.org/project/terraformpy/", "platform": "", "project_url": "https://pypi.org/project/terraformpy/", "project_urls": null, "release_url": "https://pypi.org/project/terraformpy/1.1.1/", "requires_dist": [ "schematics (<3.0,>=2.0)", "six (<2,>=1.11)", "pytest (<4.7,>=4.6) ; (python_version <= \"2.7\") and extra == 'dev'", "pytest (<6,>=5.0) ; (python_version > \"3\") and extra == 'dev'" ], "requires_python": "", "summary": "Terraformpy is a library and command line tool to supercharge your Terraform configs using a full fledged Python environment!", "version": "1.1.1" }, "last_serial": 5827648, "releases": { "1.1.0": [ { "comment_text": "", "digests": { "md5": "b888c30f4f540a1195643a4b6d56aef1", "sha256": "048f3686b640a53f95dee7eab9e67dcd0caa9dd263d0e98d57d4093b10e77e97" }, "downloads": -1, "filename": "terraformpy-1.1.0-py2-none-any.whl", "has_sig": false, "md5_digest": "b888c30f4f540a1195643a4b6d56aef1", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 15160, "upload_time": "2019-08-19T13:05:37", "url": "https://files.pythonhosted.org/packages/d6/b4/32da244f136c8295dfbe10abc1ebf3836e7c816321680cde2ab1a5e799df/terraformpy-1.1.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "e815addcd0e0fc46eef49b603903d83d", "sha256": "1ce6caa7e666f24d57c97262713d3b406a81d4d1d4136a653c34a0dacf2666f5" }, "downloads": -1, "filename": "terraformpy-1.1.0.tar.gz", "has_sig": false, "md5_digest": "e815addcd0e0fc46eef49b603903d83d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16928, "upload_time": "2019-08-19T13:05:40", "url": "https://files.pythonhosted.org/packages/bf/e7/3f03a868d6a5be253af9608a2be99d15b4b38886cfa4145f2c6ef016da0f/terraformpy-1.1.0.tar.gz" } ], "1.1.1": [ { "comment_text": "", "digests": { "md5": "188dfff93b47b6f62c075823bd8b0e39", "sha256": "c614268ef39e10cd9d266b856268ee96620ab124da80a216410c1c95227f4038" }, "downloads": -1, "filename": "terraformpy-1.1.1-py2-none-any.whl", "has_sig": false, "md5_digest": "188dfff93b47b6f62c075823bd8b0e39", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 19449, "upload_time": "2019-09-13T20:29:24", "url": "https://files.pythonhosted.org/packages/fb/17/6ebfe45a5b25a1eb23ccd7a53108752d483ef771c6550c2e339646918548/terraformpy-1.1.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "688e2565b6c58fb4da47ba0a44e667bf", "sha256": "d97debf0f7d93e854acc36dd772155ab23bf7e9d983277175e81276510a4ae8b" }, "downloads": -1, "filename": "terraformpy-1.1.1.tar.gz", "has_sig": false, "md5_digest": "688e2565b6c58fb4da47ba0a44e667bf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17497, "upload_time": "2019-09-13T20:29:26", "url": "https://files.pythonhosted.org/packages/bb/88/0fabb7f22d8673e6dbfa6113a498959873c0c1af665aa0eeecf7f4df81da/terraformpy-1.1.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "188dfff93b47b6f62c075823bd8b0e39", "sha256": "c614268ef39e10cd9d266b856268ee96620ab124da80a216410c1c95227f4038" }, "downloads": -1, "filename": "terraformpy-1.1.1-py2-none-any.whl", "has_sig": false, "md5_digest": "188dfff93b47b6f62c075823bd8b0e39", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 19449, "upload_time": "2019-09-13T20:29:24", "url": "https://files.pythonhosted.org/packages/fb/17/6ebfe45a5b25a1eb23ccd7a53108752d483ef771c6550c2e339646918548/terraformpy-1.1.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "688e2565b6c58fb4da47ba0a44e667bf", "sha256": "d97debf0f7d93e854acc36dd772155ab23bf7e9d983277175e81276510a4ae8b" }, "downloads": -1, "filename": "terraformpy-1.1.1.tar.gz", "has_sig": false, "md5_digest": "688e2565b6c58fb4da47ba0a44e667bf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17497, "upload_time": "2019-09-13T20:29:26", "url": "https://files.pythonhosted.org/packages/bb/88/0fabb7f22d8673e6dbfa6113a498959873c0c1af665aa0eeecf7f4df81da/terraformpy-1.1.1.tar.gz" } ] }