{
    "info": {
        "author": "Andrew Love",
        "author_email": "DREWNCREW@GMAIL.COM",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 2.7",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "# TOIL\n\nHard working python framework with eyes and boots on the ground for your cloud.\n<br>\n```\n*  /-\\       ~~~~~~~~\n* |< >|       ~~~~~~\n*  \\@/       ~~~~~~~~\n*  /|\\\n*   | \\\n*  / \\ \\\n*    ___\\___\n*    | | | |\n```\n\n## Description\n\nThe toil Python framework is for cloud administrators or programmers\nrequiring access to cloud services.  Provides a simplistic pattern to utilize\nand develop sdks and services,and help you get off the ground and into the\ncloud quickly.\n\nAs an enterprise cloud administrator I work with multiple cloud vendors.\nI routinely write scripts that aggregate data across the cloud vendors, and\nfind it bothersome and unproductive to securely configure config files\nwith multiple profiles, authentication and authorization.\n\n**Toil** gives you a simple centralized config file in JSON format, allowing\nmultiple profiles per SDK, and encrypting sensitive values.\n\nBuilt in providers include OCI, AWS, CHEF, SCALR, and RELATIONAL DATA SOURCES.\n\nThe framework is extensible by allowing you to add your own services, and\nfollows the \"Convention over Configuration\" design paradigm. Services are\ncompletely exposed so you do not have to worry about the framework hiding\nimplementation details.\n\n\nEncryption, method decorators for timing and retry, and other utilities are\navailable.\n\nGet the code at https://github.com/drewncrew/toil-py\n\n## Whats New?\n- 2018-05-08 improved oci library and added vertical instance scaling example\n\n- 2018-05-07 config console tool\n```\n\n    After installing toil-py with pip use the config tool to create, encrypt and decrypt the json config.\n    usage: toil-config [--init CONFIG_DIR_NAME]\n        create directory, create config.json, create key\n\n         [--create FILE_NAME]\n         create config file with file name\n\n         [--create_key FILE_NAME]\n         create encryption key with file name\n\n         [--encrypt CONFIG_FILE_NAME]\n         encrypt config file\n\n         [--decrypt CONFIG_FILE_NAME]\n         decrypt config file\n\n    To get started try this:\n\n    toil-config --init /path/.toil\n        creates a config.json file in your directory and an encryption key\n\n    toil-config -k /path/.toil/key --encrypt /path/.toil/config.json\n        create the file /path/.toil/config.json.encrypted where all values are encrypted\n\n    toil-config -k /path/.toil/key --decrypt /Users/aclove/.toil/config.json.encrypted\n        create the file /path/.toil/config.json.encrypted.decrypted where all values are decrypted\n```\n- 2018-05-04 toil is available via pip. https://pypi.org/project/toil-py\n```\npip install toil-py\n```\n- 2018-05-04 Second release 1.1.0. Restructured project\n- 2018-05-03 Console application toil-config will be released soon in 1.1.0\n- 2018-04-26 First release 1.0.0\n\n\n## Usage\nHere is a quick glance demonstrating how easy it is:\n\n## What format is the config file?\n\nThe config file is JSON, may provide YAML in future.\n\nEach service may have multiple profiles defined.  For instance, you may have\nan OCI admin profile, OCI basic profile and OCI read only profile.  You may specify\ndifferent credentials in each profile.\n\nA 'default' profile is used in all method calls if no profile name is provided.\n\n``` json\n{\n  \"toil\": {\n    \"datasources\": {\n      \"default\": {\n        \"log_level\": \"info\",\n        \"provider\": \"toil.provider.datasource.AlchemyMySQLDatasourceProvider\",\n        \"proxy\": \"\",\n        \"adapter\": \"mysql+pymysql\",\n        \"database\": \"db_name\",\n        \"env\": \"db_env\",\n        \"host\": \"host_name\",\n        \"port\": \"3306\",\n        \"user\": \"user\",\n        \"password\": \"password\"\n      }\n    },\n    \"services\": {\n      \"oci\": {\n        \"alias\": \"oci\",\n        \"log_level\": \"debug\",\n        \"provider\": \"toil.provider.oci_sdk.OciSdkLib\",\n        \"proxy\": \"\",\n        \"default\": {\n          \"user\": \"ocid1.user.oc1...\",\n          \"fingerprint\": \"...\",\n          \"key_file\": \"~/.oci/oci_api_key.pem\",\n          \"tenancy\": \"ocid1.tenancy.oc1...\",\n          \"region\": \"us-ashburn-1\"\n        },\n        \"oci_prod_admin\": {\n          \"user\": \"ocid1.user.oc1...\",\n          \"fingerprint\": \"...\",\n          \"key_file\": \"~/.oci/oci_api_key.pem\",\n          \"tenancy\": \"ocid1.tenancy.oc1...\",\n          \"region\": \"us-ashburn-1\"\n        },\n        \"oci_prod_readonly\": {\n          \"user\": \"ocid1.user.oc1...\",\n          \"fingerprint\": \"...\",\n          \"key_file\": \"~/.oci/oci_api_key.pem\",\n          \"tenancy\": \"ocid1.tenancy.oc1...\",\n          \"region\": \"us-ashburn-1\"\n        },\n        \"oci_dev_readonly\": {\n          \"user\": \"ocid1.user.oc1...\",\n          \"fingerprint\": \"...\",\n          \"key_file\": \"~/.oci/oci_api_key.pem\",\n          \"tenancy\": \"ocid1.tenancy.oc1...\",\n          \"region\": \"us-ashburn-1\"\n        }\n      },\n      \"aws\": {\n        \"alias\": \"aws\",\n        \"log_level\": \"info\",\n        \"provider\": \"toil.provider.aws.AwsLib\",\n        \"proxy\": \"proxy-host-name:proxy-port\",\n        \"default\": {\n          \"access_key_id\": \"access_key_id\",\n          \"account_number\": \"account_number\",\n          \"region\": \"us-east-1\",\n          \"role_arn\": \"arn:aws:iam::#########:role/role-name\",\n          \"role_session_name\": \"role_session_name\",\n          \"secret_access_key\": \"secret_access_key\"\n        },\n        \"aws_profile_1\": {\n          \"access_key_id\": \"access_key_id\",\n          \"account_number\": \"account_number\",\n          \"region\": \"us-east-1\",\n          \"role_arn\": \"arn:aws:iam::#########:role/role-name\",\n          \"role_session_name\": \"role_session_name\",\n          \"secret_access_key\": \"secret_access_key\"\n        }\n      },\n      \"openstack\": {\n        \"alias\": \"openstack\",\n        \"log_level\": \"info\",\n        \"provider\": \"toil.provider.openstack.OpenStackLib\",\n        \"proxy\": \"\",\n        \"default\": {\n          \"auth_url\": \"auth_url\",\n          \"domain\": \"domain\",\n          \"password\": \"password\",\n          \"project\": \"project\",\n          \"region\": \"region\",\n          \"user\": \"user\",\n          \"user_domain\": \"user_domain\"\n        }\n      }\n    }\n  }\n}\n```\n\n## Example Code\n\n### Call program from bash\n```\npython oci_list_compartment.py -c /data/files/config/toil.json -k YPALQ0g7pIOCCHg0hLL1qi7oRzdWk8Vj3Cr8-HsUoy0=\n```\n\n### Create Framework\n```\n# process args - get the config file location and encryption key\nargs = toil.parm.handle.handle_parms(['c', 'k'])\n\n# create cloud framework\nframework = toil.framework.create(**args)\n```\n\n### Use a profile:\n```\nframework.oci.session('oci_prod_admin')\n```\n\n### OCI (Oracle cloud infrastructure)\n```\nsession = framework.oci.session(env)\ncompute_client = session.client('compute')\ninstances = session.paginate(compute_client.list_instances, session.config()['compartment_id'])\n\nfor instance in instances:\nlogger.info(instance)\n```\n\n### OpenStack\n```\nsession = framework.openstack.session()\nopenstack_connection = session.connect()\n\nfor server in openstack_connection.compute.servers():\nmeta = server.metadata.get('some-key')\n```\n\n### Scalr\n```\nsession = framework.scalr.session()\nscalr_envs = session.list('/account/environments/')\n```\n\n### AWS\n```\ninstance = framework.aws.resource('ec2', 'default').Instance('i-xxxxxxx')\ntags = instance.tags\n```\n\n### AWS S3 upload directory\n```\nframework.aws.upload_to_s3('some-bucket', '/path to dir', 'folder name')\n```\n\n### AES enryption\n```\nkey = framework.encryptor.generate_key()\nconfidential_data = \"this is an encryption test\"\nencrypted_data = framework.encryptor.encrypt(confidential_data, encryption_key=key)\ndecrypted_data = framework.encryptor.decrypt(encrypted_data, encryption_key=key)\n```\n\n### Your own service you provide becomes a property of the library.  Nice!\n```\nmy_service_session = framework.your_service_name.session()\nmy_service_session.your_method()\n```\n\n### Method execution metrics\n```\n@toil.util.decorator.timeit(loops=1)\ndef process(toil):\n...\n```\n\n### Retry if an exception occurs\n```\n@toil.util.decorator.retry(3, requests.exceptions.RequestException)\ndef get(self, url, **kwargs):\n...\n```\n\n### execute a sql statement\n```\n# execute a query\n---------------\nq = \"\"\"\nselect\n*\nfrom\nsome_table\nwhere\nsome_column like :i\n\"\"\"\n\n### simple query\n---------------\nds_session = framework.datasource.session()\n\nfor row in ds_session.exec_sql_query(q, **{\"i\": \"i-30%\"}):\nprint(row['some_column'])\n```\n\n## Framework Standard Parameters\n\n### Standard parameters include:\n-  -c = config file\n-  -e = environment\n-  -k = encryption key\n-  -o = options\n-  -v = verbose\n\n\n## How do I create a config file?\n- generate encryption keys (optional)\n- create a config file - sample code in project\n- add credentials and passwords to config\n- encrypt the config file. (optional)\n- use the config file\n\n```\nconfig_file = 'c:/temp/toil.json'\nencrypted_file = 'c:/temp/toil_enc.json'\ndecrypted_file = 'c:/temp/toil_denc.json'\n\n### generate a config file\ntoil.config.util.generate_config_file(config_file)\n\n### now update the file with credentials and passwords.\n\n### encrypt file (optional)\nkey = framework.encryptor.generate_key('/path/toil_key.txt')\nframework.encrypt_config_file(config_file, encrypted_file, encryption_key=key)\n\n### decrypt file (optional)\nframework.decrypt_config_file(encrypted_file, decrypted_file, encryption_key=key)\n```\n\n## How do I add my own service?\n### create a class inherits from toil.provider.base.BaseProvider\n### implement the session method\n```\n# -*- coding: utf-8 -*-\n\"\"\"\nExample custom service\n\"\"\"\nimport logging\nimport toil.util.decorator\nimport toil.provider.base\nimport toil.framework\n\nlogger = logging.getLogger(__name__)\n\n\nclass ExampleLib(toil.provider.base.BaseProvider):\n\"\"\"\nClass example\n\nProperties :\nconfig: dict with configuration data\n\"\"\"\n\ndef __init__(self, cloud_provider, config):\nsuper(ExampleLib, self).__init__(cloud_provider, config)\n\ndef session(self, profile='default'):\n\"\"\"\nCreate session.\n\nArgs:\nprofile (str): the profile defined in config to use\n\nReturns:\nSession\n\"\"\"\nif profile in self.config:\nself.configure_proxy()\nsession = ExampleSession(self,\napi_url=self.config[profile]['api_url'],\napi_key_id=self.config[profile]['api_key_id'],\napi_key_secret=self.config[profile]['api_key_secret']\n)\nreturn session\nelse:\nraise toil.CloudException(\n\"profile '{profile}' not defined in config {config}\".format(profile=profile, config=self.config))\n\n\nclass ExampleSession(object):\n\"\"\"\nexample\n\"\"\"\ndef __init__(self, client, api_url, api_key_id, api_key_secret):\nself.client = client\nself.api_url = api_url\nself.api_key_id = api_key_id\nself.api_key_secret = api_key_secret\nsuper(ExampleSession, self).__init__()\n\ndef list(self, path, **kwargs):\nreturn []\n\ndef create(self, *args, **kwargs):\nreturn {}\n\ndef fetch(self, *args, **kwargs):\nreturn {}\n\ndef delete(self, *args, **kwargs):\nreturn {}\n\ndef post(self, *args, **kwargs):\nreturn {}\n\n@toil.util.decorator.retry(3, Exception)\ndef get(self, url, **kwargs):\nreturn {}\n```\n\n### Add service to config file.  Alias is used as the property name on the library.\n``` json\n\"services\": {\n  \"example_service\": {\n    \"provider\": \"toil.provider.example.ExampleLib\",\n    \"alias\": \"example_service\",\n    \"log_level\": \"info\",\n    \"proxy\": \"\",\n    \"default\": {\n      \"account_number\": \"123\",\n      \"access_key_id\": \"456\",\n      \"secret_access_key\": \"789\",\n    }\n    \"profile2\": {\n      \"account_number\": \"abc\",\n      \"access_key_id\": \"def\",\n      \"secret_access_key\": \"ghi\",\n    }\n  }\n}\n...\n```\n\n## Use your service\n```\nsession = toil.example_service.session( )\nsession.fetch( )\n```\n\n## Examples\nReview the exmple package.\nGet the code at https://github.com/drewncrew/toil-py\n\n\n## Installation\n\n### Linux\n\n### create a projects directory\n```\nmkdir ~/projects\ncd ~/projects\n```\n\n\n### Ensure python 2.7 is installed (Only do this if installing python)\n```\nsudo yum groupinstall -y 'development tools'\nsudo yum install -y python-devel\nsudo yum install -y xz-libs\nsudo yum install -y zlib-devel\nsudo yum install -y openssl-devel\nsudo yum install -y openldap-devel\nsudo yum install -y mysql-devel.x86_64\nsudo yum install -y mysql-connector-python\n```\n\n### Get python source (Only do this if installing python)\n```\nwget http://www.python.org/ftp/python/2.7.12/Python-2.7.12.tar.xz\n# decode (-d) the XZ encoded tar archive:\nxz -d Python-2.7.12.tar.xz\n# extraction:\ntar -xvf Python-2.7.12.tar\n```\n\n### Build python 2.7 (Only do this if installing python)\n```\ncd Python-2.7.12\n./configure\nsudo make\nsudo make altinstall\n#python2.7 should now be in /usr/local/bin/\n```\n\n### Ensure pip is installed\n```\nwget https://bootstrap.pypa.io/get-pip.py\nsudo /usr/local/bin/python2.7 get-pip.py\n#pip2.7 should now be in /usr/local/bin/\n```\n\n\n### Install virtual environment\n```\ncd ~/projects\nsudo /usr/local/bin/pip2.7 install virtualenv\n```\n\n### Activate virtual environment\n```\nvirtualenv --python=/usr/local/bin/python2.7 toilpython27\nsource ~/projects/bin/activate\n```\n\n### Finally install the library\n```\n#get the current source distribution\nwget -O /dist/toil-0.0.1.tar.gz\npip2.7 install -I /home/user/projects/myproject/toil-0.0.1.tar.gz\n```\n\n### Create a directory for your project\n```\nmkdir ~/projects/myproject\ncd ~/projects/myproject/\n#makesure your virtual env is active\nsource ~/projects/bin/activate\nwhich python #should return ~/projects/toilpython27/bin/python\n```\n\n## Notes\n- get the 2.7 release from https://www.python.org/downloads/\n- install .whl by ```pip install some-package.whl```\n- http://dev.mysql.com/downloads/connector/python/\n- get the distribution with pip. see https://pypi.org/project/toil-py/\n- pip install -I toil-0.0.1.tar.gz\n- Create a source distribution with command python setup.py sdist --formats=gztar\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/drewncrew/toil",
        "keywords": "cloud oci aws scalr mysql oracle framework openstack",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "toil-py",
        "package_url": "https://pypi.org/project/toil-py/",
        "platform": "",
        "project_url": "https://pypi.org/project/toil-py/",
        "project_urls": {
            "Homepage": "https://github.com/drewncrew/toil"
        },
        "release_url": "https://pypi.org/project/toil-py/1.3.1/",
        "requires_dist": [
            "nose",
            "urllib3[secure]",
            "PyChef",
            "PyMySQL",
            "cryptography (>=2.1.3)",
            "SQLAlchemy",
            "requests (>=2.5.1)",
            "oci",
            "openstacksdk",
            "boto3",
            "check-manifest; extra == 'dev'",
            "coverage; extra == 'test'"
        ],
        "requires_python": "",
        "summary": "toil - Hard working python framework with eyes and boots on the ground for your cloud. Built in workers include OCI, AWS, CHEF, SCALR, REALTIONAL DATA SOURCES.",
        "version": "1.3.1"
    },
    "last_serial": 3844797,
    "releases": {
        "1.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0ecd6ff87967659f25902f9bc0745b44",
                    "sha256": "5d7e47a3369e26091fcffd22801d2c293b2f76be7fc02d460ed78dcce0fc5b44"
                },
                "downloads": -1,
                "filename": "toil_py-1.1.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0ecd6ff87967659f25902f9bc0745b44",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 42463,
                "upload_time": "2018-05-04T15:55:05",
                "url": "https://files.pythonhosted.org/packages/bb/d0/5d62f1be3d0d99257b5d9d349bb4f178e73306ad31f148fadbb581f36775/toil_py-1.1.0-py2.py3-none-any.whl"
            }
        ],
        "1.2.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "18d39f5d3a54a4e402f63f5db22909a3",
                    "sha256": "8b458e33a9a66a3b419bc865224ec09a123061c9330dae48b0c0861f86ff209d"
                },
                "downloads": -1,
                "filename": "toil_py-1.2.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "18d39f5d3a54a4e402f63f5db22909a3",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 44931,
                "upload_time": "2018-05-08T16:33:24",
                "url": "https://files.pythonhosted.org/packages/3a/c7/98fb67479da82de67b469896a5346cbb46856d80432f61f5cf01868ddaca/toil_py-1.2.0-py2.py3-none-any.whl"
            }
        ],
        "1.3.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "66127365cac21a28552c08e775008ed3",
                    "sha256": "a4abdd32889193839167fd6e2b30cd3f0c7beb155ab9f75e7727f6e61f73a3e3"
                },
                "downloads": -1,
                "filename": "toil_py-1.3.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "66127365cac21a28552c08e775008ed3",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 39376,
                "upload_time": "2018-05-08T16:37:37",
                "url": "https://files.pythonhosted.org/packages/f0/73/d141fbb1e68a83e701740628bfe2223f4d0f2b59d9a4c8f6f46cf078280e/toil_py-1.3.1-py2.py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "66127365cac21a28552c08e775008ed3",
                "sha256": "a4abdd32889193839167fd6e2b30cd3f0c7beb155ab9f75e7727f6e61f73a3e3"
            },
            "downloads": -1,
            "filename": "toil_py-1.3.1-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "66127365cac21a28552c08e775008ed3",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 39376,
            "upload_time": "2018-05-08T16:37:37",
            "url": "https://files.pythonhosted.org/packages/f0/73/d141fbb1e68a83e701740628bfe2223f4d0f2b59d9a4c8f6f46cf078280e/toil_py-1.3.1-py2.py3-none-any.whl"
        }
    ]
}