{ "info": { "author": "Hinnerk Haardt, HIT Information-Control GmbH", "author_email": "haardt@information-control.de", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: BSD License", "Programming Language :: Python :: 2", "Topic :: System :: Installation/Setup", "Topic :: System :: Systems Administration" ], "description": "Templates and Management for AWS CloudFormation\n===============================================\n\n|Build Status|\n\n- **``cftpl``**: A simple Python library to build and manage\n CloudFormation Stacks. STacks are built using templates and can use\n variables, loops and reuse existing code.\n- **``cfmgr``:** The executable to manage CloudFormation Stacks from\n the command line. Based on ``cftpl``, has actual password management.\n\nWhy?\n====\n\n- The syntax becomes shorter and much more readable. Using YAML allows\n comment lines within the stack template.\n- An expressive template language enables loops and code reuse (``if``,\n ``for``, ``include``, ``extends``, ``macro``)\n- Use all this with a simple command line tool.\n\nFeatures\n========\n\n- Build AWS CloudFormation stack templates using JSON, YAML and the\n Jinja2 template language.\n- Create, update and delete CloudFormation stacks (uses\n `boto `__).\n- Use as a standalone executable or as a Python library.\n- Provides state of the art password management.\n\nInstallation\n============\n\nThe Lazy:\n---------\n\n::\n\n sudo pip install cftpl\n\n``cfmgr`` should be available now:\n\n::\n\n $ cfmgr\n\n cfmgr \n\n Where is one of:\n\n list: list all stacks in AWS\n create: create or update the stack (same as 'update')\n update: create or update the stack (same as 'create')\n show: displays the generated template in the base format (usually YAML) without converting it to JSON\n test: builds, tests and displays the generated stack template (JSON), does not change any resources\n delete: delete the stack on AWS\n convert: convert a JSON file to YAML syntax\n\n And it the path of a config.py file.\n\nEverybody else:\n---------------\n\n.. code:: sh\n\n git clone https://github.com/hinnerk/cftpl.git\n cd cftpl\n python bootstrap.py\n ./bin/buildout\n\n``cfmgr`` is available as ``bin/cfmgr``. No system packages are\nmodified, all dependencies are installed within the top level directory.\n\nExample\n=======\n\nFirst Step: Configuration\n-------------------------\n\nGo to the ``example`` directory or download the example files to an\notherwise empty directory:\n\n- `config.py `__\n- `example.yaml `__\n\nIn ``config.py``:\n\n1. Set the ``ACCOUNT`` to your actual AWS access key id. Keep your\n secret access key handy, but do not enter it anywhere yet.\n2. Set ``EMAIL`` to your email address. It will be used to inform you\n about started and stopped instances.\n\nSecond Step: Validate and Create Stack\n--------------------------------------\n\nTest your Stack\n~~~~~~~~~~~~~~~\n\nThe command ``cfmgr test config.py`` will validate the configuration\nonline. Because of that you will need to enter your secret access key.\nThis key will be stored in your operating systems encrypted key store\n(Keychain on Mac OS X, the Linux Secret Service, the Windows Credential\nVault) and only used to authenticate AWS API calls.\n\nAdditionally it will display the configuration of the stack and all\nfiles defined in the configuration file.\n\n::\n\n $ ../bin/cfmgr test config.py\n Please enter password for \"ABCDEF1234567890\":\n Validating template... successfull.\n This is the configuration we'd use if this was for real:\n\n\n {\n \"AWSTemplateFormatVersion\": \"2010-09-09\",\n [...]\n }\n\n\n Here are all the files you can use:\n \"salt-installer\": \"install-salt.sh\"\n\nCreate your Stack\n~~~~~~~~~~~~~~~~~\n\n**Warning: This will cost your money.** Because this step creates\nressources on AWS charges will occur. Do not proceed if you do not\nunderstand what that means.\n\nYou might want to open the `AWS CloudFormation Management\nConsole `__ and watch\nclosely while your stack is created.\n\n::\n\n $ ../bin/cfmgr create config.py\n Validating template... successfull.\n Creating Stack CFTPL-Example-Stack... done.\n Showing status messages for stack None:\n 2014-05-21 12:01:21.997000: AWS::CloudFormation::Stack CREATE_IN_PROGRESS CFTPL-Example-Stack User Initiated\n [...]\n\nThe command will show all log messages until the stack creation is\ncomplete. You can break this any time with Ctrl-C. Please note that this\ncancels the display of messages only; the stack creation will continue\nin the background. Again, it is a good idea to watch this in the `AWS\nCloudFormation Management\nConsole `__ until you are\nfamiliar with the system.\n\n**Important: Do not forget to confirm your SNS topic subscription.** The\nexample configuration creates an Amazon Simple Notification Service\nTopic and subscribes your email to it so you receive emails whenever an\nEC2 instance is created or deleted.\n\nUpdate your stack\n~~~~~~~~~~~~~~~~~\n\n**Warning: This will create a running EC2 instance. Do not forget to\ndelete the stack later to prevent charges from piling up.**\n\nUp to now the stack has not created any EC2 instances. While we can just\ngo ahead and add one to the stack, it is usually more serviceable to let\nAWS handle the creation and destruction of actual instances. This is\ndone through the modification of AutoScalingGroups.\n\nChange the lines 16 and 17 of the example configuration.\n``MaxSize: '0'`` to ``MaxSize: '1'`` and ``DesiredCapacity: '0'`` to\n``DesiredCapacity: '1'``.\n\nFrom this:\n\n.. code:: YAML\n\n MinSize: 0\n MaxSize: '0'\n DesiredCapacity: '0'\n\nTo this:\n\n.. code:: YAML\n\n MinSize: 0\n MaxSize: '1'\n DesiredCapacity: '1'\n\nBe careful to keep the indention intact.\n\nNow update the running stack:\n\n::\n\n $ ../bin/cfmgr update config.py\n Validating template... successfull.\n Updating Stack CFTPL-Example-Stack... done.\n done.\n Showing status messages for stack CFTPL-Example-Stack:\n [...]\n 2014-05-21 12:38:38.201000: AWS::CloudFormation::Stack UPDATE_IN_PROGRESS CFTPL-Example-Stack User Initiated\n 2014-05-21 12:38:48: AWS::AutoScaling::AutoScalingGroup UPDATE_IN_PROGRESS DemoServerASGroup None\n 2014-05-21 12:38:49: AWS::AutoScaling::AutoScalingGroup UPDATE_COMPLETE DemoServerASGroup None\n 2014-05-21 12:38:52.014000: AWS::CloudFormation::Stack UPDATE_COMPLETE_CLEANUP_IN_PROGRESS CFTPL-Example-Stack None\n 2014-05-21 12:38:53.679000: AWS::CloudFormation::Stack UPDATE_COMPLETE CFTPL-Example-Stack None\n Status: UPDATE_COMPLETE\n\nThe AutoScalingGroup now knows that it is supposed to create an EC2\nInstance. You can see that instance being created in the `AWS EC2\nManagement Console `__. Additionally\nyou should receive an email that informs you about the new instance.\n\nThe number of running instances is managed by AutoScaling. Thus when you\ndestroy the running instance using Actions => Terminate in the EC2\nManagement Console, another one is created and you should receive\nseparate emails about the destruction of the old and the creation of a\nnew instance.\n\nTo stop all instances from running either change ``MaxSize: '1'`` to\n``MaxSize: '0'`` and ``DesiredCapacity: '1'`` back to\n``DesiredCapacity: '0'`` and update the stack, or just delete the whole\nstack.\n\nDelete your Stack\n~~~~~~~~~~~~~~~~~\n\nThis deletes the stack and most of its associated ressources. Some\nressources, like database backups and S3 buckets are not necessarily\nremoved to prevent data loss. So deleting a stack may not stop all\ncharges. Please read the AWS documentation to find out which ressources\nwill persist.\n\n::\n\n $ ../bin/cfmgr delete config.py\n arn:aws:cloudformation:eu-west-1:001408880364:stack/CFTPL-Example-Stack/83772a30-e0ee-11e3-8185-50014118ec7c CFTPL-Example-Stack CREATE_COMPLETE\n Should I delete CFTPL-Example-Stack? yes\n Deleting stack CFTPL-Example-Stack...done.\n Showing status messages for stack CFTPL-Example-Stack:\n [...]\n 2014-05-21 13:49:27.413000: AWS::CloudFormation::Stack DELETE_IN_PROGRESS CFTPL-Example-Stack User Initiated\n 2014-05-21 13:49:43: AWS::AutoScaling::AutoScalingGroupDELETE_IN_PROGRESS DemoServerASGroup None\n 2014-05-21 13:49:44: AWS::AutoScaling::AutoScalingGroupDELETE_COMPLETE DemoServerASGroup None\n 2014-05-21 13:49:45: AWS::SNS::Topic DELETE_IN_PROGRESS DemoAutoScalingSNSTopic None\n 2014-05-21 13:49:45: AWS::AutoScaling::LaunchConfigurationDELETE_IN_PROGRESS DemoServerLaunchConfig None\n 2014-05-21 13:49:46: AWS::AutoScaling::LaunchConfigurationDELETE_COMPLETE DemoServerLaunchConfig None\n 2014-05-21 13:49:46: AWS::SNS::Topic DELETE_COMPLETE DemoAutoScalingSNSTopic None\n 2014-05-21 13:49:48: AWS::EC2::SecurityGroup DELETE_IN_PROGRESS DemoServerSecurityGroup None\n 2014-05-21 13:49:49: AWS::EC2::SecurityGroup DELETE_COMPLETE DemoServerSecurityGroup None\n Status: DELETE_COMPLETE\n\nAdditional Commands\n===================\n\nConverting JSON to YAML\n-----------------------\n\n``cfmgr convert config.py`` converts an existing JSON into an YAML\ntemplate:\n\n- Copy your existing JSON stack template file into the directory of the\n example configuration. Make sure that the file ends with ``.json``.\n- In the configuration change the value of ``TEMPLATE`` to the name of\n that file.\n\nThen call ``cfmgr convert config.py``:\n\n::\n\n $ ../bin/cfmgr convert config.py\n Reading JSON from:\n example/my-old-stack.json\n YAML written to:\n example/my-old-stack.yaml\n DONE\n\n $ ls -la\n -rw-r--r-- 1 user staff 188 May 16 15:50 config.py\n -rw-r--r-- 1 user staff 25984 May 16 16:13 my-old-stack.json\n -rw-r--r-- 1 user staff 12562 May 16 16:13 my-old-stack.yaml\n\nYou can now change ``TEMPLATE = 'my-old-stack.json.yaml'``. Do not\nforget to version control it prior to editing it. ;)\n\nShow Template YAML\n------------------\n\nSometimes, especially when developing macros, it's nice to see the\ngenerated YAML code prior to the further export to JSON.\n``cfmgr show config.py`` does just that:\n\n::\n\n $ ../bin/cfmgr show config.py\n This is the rendered template:\n\n\n AWSTemplateFormatVersion: '2010-09-09'\n Description: CFTPL Example.\n Resources:\n [...]\n\nList all Stacks\n---------------\n\n``cfmgr list config.py`` returns a list of all existing running and\ndeleted stacks:\n\n::\n\n $ ../bin/cfmgr list config.py\n\n Region: eu-west-1\n Endpoint: cloudformation.eu-west-1.amazonaws.com\n\n NAME | CFTPL-Example-Stack\n -------------+-------------------------------------\n STATUS | DELETE_COMPLETE\n REASON |\n [...]\n\nUse as a library\n================\n\n.. code:: python\n\n import cftpl\n\n config = cftpl.get_settings('path/to/config.py')\n template = cftpl.CFTemplate(config)\n stack = cftpl.CFStack(config)\n\nTODO: Add documentation here.\n\nPassword Management\n===================\n\nStand Alone Executable\n----------------------\n\nOn the first start the executable will ask for the password and store it\nin the local encrypted password storage of the operating system. That's\nKeychain on Mac OS X, the Linux Secret Service and the Windows\nCredential Vault.\n\nThis password will be reused on further calls, so you don't have to type\nit while it is still stored adequately secure.\n\nLibrary Calls\n-------------\n\nWhenever the library is running on an AWS instance, `Temporary Security\nCredentials `__\nare the solution you're looking for.\n\nCftpl will use temporary security credentials by default whenever\navailable.\n\nIf your company has implemented some kind of password management, you\ncan call ``CFStack()`` with a password:\n\n.. code:: python\n\n from cftpl import CFStack\n\n password = ge_password()\n config = get_config()\n stack = CFStack(config, password=password)\n\n stack.update()\n\nLastly one could create boto configuration files and leave them lie\nabout, so they can be read by an attacker or accidentally checked into a\npublic repository. Honestly, AWS has a perfectly serviceable key\nmanagement right built in, so why not use it?\n\nJSON vs. YAML + Jinja2\n======================\n\nThis is part of a template we actually use. Being written in YAML and\nusing a Jinja2 ``for`` loop it's quite readable. Note, that YAML allows\ncomments:\n\n.. code:: YAML\n\n SOMEDNS:\n Type: AWS::Route53::RecordSetGroup\n Properties:\n Comment: TEST CASE Zone Records\n HostedZoneName: test.local.\n # just to show off: YAML does comments\n RecordSets:\n {% for name in ('calendar', 'chat', 'docs', 'mail', 'start') %}\n - Name: {{ name }}.test.local.\n Type: CNAME\n TTL: \"43200\"\n ResourceRecords: [\"ghs.google.com.\"]\n {% endfor %}\n\nThis is the same as above in the format CloudFormation accepts (JSON).\nPlease note that while it's quite a bit longer, JSON does not allow\ncomments.:\n\n.. code:: JSON\n\n \"SOMEDNS\": {\n \"Type\": \"AWS::Route53::RecordSetGroup\",\n \"Properties\": {\n \"Comment\": \"TEST CASE Zone Records\",\n \"HostedZoneName\": \"test.local.\",\n \"RecordSets\": [\n {\n \"ResourceRecords\": [\n \"ghs.google.com.\"\n ],\n \"Type\": \"CNAME\",\n \"Name\": \"calendar.test.local.\",\n \"TTL\": \"43200\"\n },\n {\n \"ResourceRecords\": [\n \"ghs.google.com.\"\n ],\n \"Type\": \"CNAME\",\n \"Name\": \"chat.test.local.\",\n \"TTL\": \"43200\"\n },\n {\n \"ResourceRecords\": [\n \"ghs.google.com.\"\n ],\n \"Type\": \"CNAME\",\n \"Name\": \"docs.test.local.\",\n \"TTL\": \"43200\"\n },\n {\n \"ResourceRecords\": [\n \"ghs.google.com.\"\n ],\n \"Type\": \"CNAME\",\n \"Name\": \"mail.test.local.\",\n \"TTL\": \"43200\"\n },\n {\n \"ResourceRecords\": [\n \"ghs.google.com.\"\n ],\n \"Type\": \"CNAME\",\n \"Name\": \"start.test.local.\",\n \"TTL\": \"43200\"\n }\n ]\n }\n },\n\n.. |Build Status| image:: https://travis-ci.org/hinnerk/cftpl.svg?branch=master\n :target: https://travis-ci.org/hinnerk/cftpl", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/hinnerk/cftpl", "keywords": "Amazon Web Services,CloudFormation,AWS", "license": "(c) 2014 HIT Information-Control GmbH", "maintainer": null, "maintainer_email": null, "name": "cftpl", "package_url": "https://pypi.org/project/cftpl/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/cftpl/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/hinnerk/cftpl" }, "release_url": "https://pypi.org/project/cftpl/1.0/", "requires_dist": null, "requires_python": null, "summary": "Templates and Management for AWS CloudFormation.", "version": "1.0" }, "last_serial": 1119164, "releases": { "1.0": [ { "comment_text": "", "digests": { "md5": "e32e815719572452e46a49a42a2d05e3", "sha256": "c44345910293222e3d115e721f51f4042f168c485105f9fb16f5a3de2c48484d" }, "downloads": -1, "filename": "cftpl-1.0.tar.gz", "has_sig": false, "md5_digest": "e32e815719572452e46a49a42a2d05e3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15110, "upload_time": "2014-06-09T13:06:04", "url": "https://files.pythonhosted.org/packages/e4/6f/82c338e45ad342c3a68acfad2437a2ec0443f029438bb0556363df83a294/cftpl-1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "e32e815719572452e46a49a42a2d05e3", "sha256": "c44345910293222e3d115e721f51f4042f168c485105f9fb16f5a3de2c48484d" }, "downloads": -1, "filename": "cftpl-1.0.tar.gz", "has_sig": false, "md5_digest": "e32e815719572452e46a49a42a2d05e3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15110, "upload_time": "2014-06-09T13:06:04", "url": "https://files.pythonhosted.org/packages/e4/6f/82c338e45ad342c3a68acfad2437a2ec0443f029438bb0556363df83a294/cftpl-1.0.tar.gz" } ] }