{ "info": { "author": "Demand Media", "author_email": "opensource@demandmedia.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Natural Language :: English", "Programming Language :: Python", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7" ], "description": "# Ice Pick\n\nA Python library that allows easy access to AWS billing data collected by the Netflix OSS Ice tool,\nas well as manipulation of the configured Application Groups.\n\n\n## Installation\n\n pip install ice_pick\n\n## Getting Started\n\nUsing ice_pick is simple. Once you have an instance of the Netflix OSS Ice application configured for your AWS account, you can just import the APIRequest, point to your Ice url, and start querying.\n\n### Getting the Summary Table Data\n\n from ice_pick.api import APIRequest\n \n ice_url = 'http://example.com/ice/' # URL to your Ice instance\n api_request = APIRequest(ice_url)\n data = api_request.get_data()\n \n print data\n \n {\n 'data': {\n 'AWS Data Pipeline': [0.25148882999999966],\n 'Alexa Web Information Service': [0.11130000000000001],\n 'aggregated': [48884.90918908445],\n 'cloudfront': [3241.575401989994],\n .\n .\n .\n 'vpc': [296.7824844800026]\n },\n 'groupBy': 'Product',\n 'hours': [588],\n 'start': 1391212800000,\n 'stats': {\n 'AWS Data Pipeline': {\n 'average': 0.25148882999999966,\n 'max': 0.25148882999999966,\n 'total': 0.25148882999999966\n },\n 'Alexa Web Information Service': {\n 'average': 0.11130000000000001,\n 'max': 0.11130000000000001,\n 'total': 0.11130000000000001\n },\n 'aggregated': {\n 'average': 48884.90918908445,\n 'max': 48884.90918908445,\n 'total': 48884.90918908445\n },\n 'cloudfront': {\n 'average': 3241.575401989994,\n 'max': 3241.575401989994,\n 'total': 3241.575401989994\n },\n .\n .\n .\n 'vpc': {\n 'average': 296.7824844800026,\n 'max': 296.7824844800026,\n 'total': 296.7824844800026\n }\n },\n 'status': 200,\n 'time': [1391212800000]\n }\n\nBy default **ice-pick** uses the following filters:\n\n {\n 'aggregate': 'data',\n 'breakdown': True,\n 'consolidate': 'monthly',\n 'end': '2014-02-25 08PM',\n 'factorsps': False,\n 'groupBy': 'Product',\n 'isCost': True,\n 'showsps': False,\n 'start': '2014-02-01 12AM'\n }\n \nThe *start* and *end* dates are based on current (UTC) time by default.\n\n### Filtering by Products\n\n from ice_pick.filters import products as _products\n products = [_products.EC2, _products.EC2_INSTANCE]\n api_request.set_products(products)\n data = api_request.get_data()\n \n print data\n \n {\n 'data': {\n 'aggregated': [27643.726958229963],\n 'ec2': [1663.472958229997],\n 'ec2_instance': [25980.253999999964]\n },\n 'groupBy': 'Product',\n 'hours': [588],\n 'start': 1391212800000,\n 'stats': {\n 'aggregated': {\n 'average': 27643.726958229963,\n 'max': 27643.726958229963,\n 'total': 27643.726958229963\n },\n 'ec2': {\n 'average': 1663.472958229997,\n 'max': 1663.472958229997,\n 'total': 1663.472958229997\n },\n 'ec2_instance': {\n 'average': 25980.253999999964,\n 'max': 25980.253999999964,\n 'total': 25980.253999999964\n }\n },\n 'status': 200,\n 'time': [1391212800000]\n }\n\n### Filtering by Regions\n\n from ice_pick.filters import regions as _regions\n regions = [_regions.US_WEST_1, _regions.US_WEST_2]\n api_request.set_products(regions)\n data = api_request.get_data()\n \n print data\n \n {\n 'data': {\n 'aggregated': [3901.7434038699694],\n 'ec2': [100.57340387000029],\n 'ec2_instance': [3801.169999999969]\n },\n 'groupBy': 'Product',\n 'hours': [588],\n 'start': 1391212800000,\n 'stats': {\n 'aggregated': {\n 'average': 3901.7434038699694,\n 'max': 3901.7434038699694,\n 'total': 3901.7434038699694\n },\n 'ec2': {\n 'average': 100.57340387000029,\n 'max': 100.57340387000029,\n 'total': 100.57340387000029\n },\n 'ec2_instance': {\n 'average': 3801.169999999969,\n 'max': 3801.169999999969,\n 'total': 3801.169999999969\n }\n }, \n 'status': 200,\n 'time': [1391212800000]\n }\n \n \n \n### More Filters\n\n # Filtering by date time\n import datetime\n start = datetime.datetime(2014, 01, 01)\n end = datetime.datetime.now()\n \n api_request.set_start(start)\n api_request.set_end(start)\n \n \n # Filtering by Usage Type\n from ice_pick.filters import usage_types as _usage_types\n usage_types = [_usage_types.LOAD_BALANCER_USAGE, _usage_types.M1_XLARGE]\n \n api_request.set_usage_types(usage_types)\n \n\n### Initializing And Overriding Default Filtering\n\nYou can pass all the filters you want to apply at the moment you initialize the APIRequest.\n\n from ice_pick.filters import consolidate as _consolidate\n from ice_pick.filters import group_by as _group_by\n from ice_pick.filters import operations as _operations\n\n filters = {\n APIFilters.ACCOUNTS: ['012345678900', '009876543210'],\n APIFilters.REGIONS: [_regions.US_WEST_1, _regions.US_WEST_2],\n APIFilters.BREAKDOWN: True,\n APIFilters.CONSOLIDATE: _consolidate.MONTHLY,\n APIFilters.FACTOR_SPS: False,\n APIFilters.GROUP_BY: _group_by.PRODUCT,\n APIFilters.IS_COST: True,\n APIFilters.OPERATIONS: [_operations.CREATE_BUCKET, _operations.CREATE_SNAPSHOT],\n APIFilters.SHOW_SPS: False,\n APIFilters.USAGE_TYPES: USAGE_TYPES\n .\n .\n .\n }\n\n api_request = APIRequest(ice_url, **filters)\n\n\n### Getting The Current Filters\n\nYou can check which filters were active on the last request, and which filters will be used for the next request.\n\n api_request.get_filters()\n\n### Using HTTP Request Authentication\n\nIf your Ice installation is behind HTTP authentication (i.e. being served by a proxy with\nHTTP authentication enabled), you can use any form of authentication that is supported\nby the [requests library](http://docs.python-requests.org/en/latest/user/authentication/);\nsimply call the ``set_http_auth()`` method with the same parameter that needs to be\npassed to the requests object.\n\nHTTP Basic authentication:\n\n from ice_pick.api import APIRequest\n \n ice_url = 'http://example.com/ice/' # URL to your Ice instance\n api_request = APIRequest(ice_url)\n api_request.set_http_auth(('myuser', 'mypass'))\n data = api_request.get_data()\n\nHTTP Digest authentication:\n\n from ice_pick.api import APIRequest\n from requests.auth import HTTPDigestAuth\n \n ice_url = 'http://example.com/ice/' # URL to your Ice instance\n api_request = APIRequest(ice_url)\n api_request.set_http_auth(HTTPDigestAuth('user', 'pass'))\n data = api_request.get_data()\n\n## Working With Application Groups\n\nice_pick now includes a Groups class to query and manipulate Application Groups\nand related data. This is useful if you have ``ice.customTags`` set in\n``ice.properties`` and want to automatically create Application Groups for them.\n\n### Query Accounts, Products and Resource Groups\n\n >>> from pprint import pprint\n >>> from ice_pick.groups import Groups\n >>> g = Groups('http://ice.example.com/')\n >>> accounts = g.get_account_names()\n >>> pprint(accounts)\n [u'123456789012', u'987654321098']\n >>> regions = g.get_regions_for_account(accounts)\n >>> pprint(regions)\n [u'ap-northeast-1',\n u'ap-southeast-1',\n u'ap-southeast-2',\n u'eu-west-1',\n u'sa-east-1',\n u'us-east-1',\n u'us-west-1',\n u'us-west-2']\n >>> products = g.get_products(accounts, regions)\n >>> pprint(products)\n [u'ebs',\n u'ec2',\n u'ec2_instance',\n u'rds',\n u's3']\n >>> resource_groups = g.get_all_resource_groups(accounts, regions, products)\n >>> pprint(resource_groups)\n [u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Bar_Test',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test',\n u'ebs',\n u'ec2',\n u'ec2_instance',\n u'rds',\n u's3']\n >>> rg_lists = g.get_resource_group_lists()\n >>> pprint(rg_lists)\n {u'AWS Key Management Service': [],\n u'ebs': [u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test',\n u'ebs'],\n u'ec2': [u'someapp',\n u'someapp_Prod',\n u'someapp_Test',\n u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Bar_Test',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test',\n u'ec2'],\n u'ec2_instance': [u'someapp',\n u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test',\n u'ec2_instance'],\n u'rds': [u'rds',\n u'someapp',\n u'someapp_Prod',\n u'someapp_Test'],\n u'redshift': [],\n u's3': [u's3',\n u'someapp_Prod'],\n u'ses': [],\n u'simpledb': [],\n u'sns': [],\n u'sqs': [],\n u'storage_gateway': [],\n u'sws': [],\n u'vpc': []}\n >>> app_groups = g.get_application_group_names()\n >>> pprint(app_groups)\n [u'SomeService',\n u'Foobar',\n u'AMIs']\n\n### Query an Existing Application Group\n\n >>> from pprint import pprint\n >>> from ice_pick.groups import Groups\n >>> g = Groups('http://ice.example.com/')\n >>> grp = g.get_application_group('Foobar')\n >>> pprint(grp)\n {'name': u'Foobar',\n 'owner': u'me@example.com',\n 'products': {u'ec2': [u'someapp', u'someapp_Prod', u'someapp_Test'],\n u'ec2_instance': [u'someapp'],\n u'rds': [u'someapp', u'someapp_Prod', u'someapp_Test'],\n u's3': [u'someapp_Prod']}}\n\n### Create or Update an Application Group\n\nApplication group of all untagged resources:\n\n >>> from pprint import pprint\n >>> from ice_pick.groups import Groups\n >>> g = Groups('http://ice.example.com/')\n >>> rg_lists = g.get_resource_group_lists()\n >>> \n >>> untagged = {}\n >>> for product in rg_lists:\n ... for rg in rg_lists[product]:\n ... if product == rg:\n ... untagged[product] = [rg]\n ... \n >>> pprint(untagged)\n {u'cloudfront': [u'cloudfront'],\n u'cloudwatch': [u'cloudwatch'],\n u'ebs': [u'ebs'],\n u'ec2': [u'ec2'],\n u'ec2_instance': [u'ec2_instance'],\n u'elasticache': [u'elasticache'],\n u'rds': [u'rds'],\n u'route53': [u'route53'],\n u's3': [u's3']}\n >>> g.set_application_group('Untagged', untagged, 'me@example.com')\n {}\n\nApplication group for all Resource Groups (tags)\nstarting with 'SomeService':\n\n >>> from collections import defaultdict\n >>> from pprint import pprint\n >>> from ice_pick.groups import Groups\n >>> g = Groups('http://ice.example.com/')\n >>> rg_lists = g.get_resource_group_lists()\n >>> \n >>> SomeService = defaultdict(list)\n >>> for product in rg_lists:\n ... for rg in rg_lists[product]:\n ... if rg.startswith('SomeService'):\n ... SomeService[product].append(rg)\n ... \n >>> pprint(dict(SomeService))\n {u'ebs': [u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test'],\n u'ec2': [u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Bar_Test',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test'],\n u'ec2_instance': [u'SomeService/Foo_Prod',\n u'SomeService/Foo_Test',\n u'SomeService/Foo_Unknown',\n u'SomeService/Bar_Dev',\n u'SomeService/Bar_Prod',\n u'SomeService/Baz_Dev',\n u'SomeService/Baz_Prod',\n u'SomeService/Baz_Test']}\n >>> g.set_application_group('SomeService', SomeService, 'me@example.com')\n {}\n\n## License\n\nCopyright 2014 Demand Media\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/demandmedia/ice_pick", "keywords": null, "license": "Apache 2.0", "maintainer": null, "maintainer_email": null, "name": "ice_pick", "package_url": "https://pypi.org/project/ice_pick/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/ice_pick/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/demandmedia/ice_pick" }, "release_url": "https://pypi.org/project/ice_pick/0.6/", "requires_dist": null, "requires_python": null, "summary": "Python data access interface for Netflix OSS Ice Tool", "version": "0.6" }, "last_serial": 2079388, "releases": { "0.1": [ { "comment_text": "", "digests": { "md5": "8791e5f0171be967b2a47ef21b22c9ac", "sha256": "5df8012b26534127768a6e211ed787f265724a6bae63da0d26fcd739e89df358" }, "downloads": -1, "filename": "ice_pick-0.1.tar.gz", "has_sig": false, "md5_digest": "8791e5f0171be967b2a47ef21b22c9ac", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11602, "upload_time": "2014-05-23T18:31:17", "url": "https://files.pythonhosted.org/packages/26/a4/a7bff7d22d6ad23b8552a0fd8aa4333927387f9c1180b27aa5922ce5ae47/ice_pick-0.1.tar.gz" } ], "0.2": [ { "comment_text": "", "digests": { "md5": "03fdfe40b8b2acc026345c81c5fd3a8a", "sha256": "bc6b7c074a2aa34f7ab6dcc121f1b5cbb23fa087ecb977f7560cb5187d1b9264" }, "downloads": -1, "filename": "ice_pick-0.2.tar.gz", "has_sig": false, "md5_digest": "03fdfe40b8b2acc026345c81c5fd3a8a", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 11601, "upload_time": "2014-05-23T18:55:13", "url": "https://files.pythonhosted.org/packages/c5/bc/62aca1987453a8cd019a27c9fb927195cd43a7096b3eb433546c6a78d641/ice_pick-0.2.tar.gz" } ], "0.3": [ { "comment_text": "", "digests": { "md5": "0702c1a7767adedc3e6600d36116fd9b", "sha256": "72d9e8a7c5dcd9e93cbcd57a0cdace785581ff1bf1f1ebc2b53d7d96f06496e1" }, "downloads": -1, "filename": "ice_pick-0.3.tar.gz", "has_sig": false, "md5_digest": "0702c1a7767adedc3e6600d36116fd9b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17586, "upload_time": "2015-07-06T21:01:50", "url": "https://files.pythonhosted.org/packages/90/7f/cbc5c47e26ab339a725f87061d26fb53fcc613bbbe840d114341c0e9578d/ice_pick-0.3.tar.gz" } ], "0.4": [ { "comment_text": "", "digests": { "md5": "e01a7f2e095b5a4d2cfd64b36260a5f3", "sha256": "a6c6cac2838f7ae5915e09386604508261829594d3af2efc77aa9c0edb654a8a" }, "downloads": -1, "filename": "ice_pick-0.4.tar.gz", "has_sig": false, "md5_digest": "e01a7f2e095b5a4d2cfd64b36260a5f3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17538, "upload_time": "2015-08-21T20:24:40", "url": "https://files.pythonhosted.org/packages/ad/f1/bf33442442839cbbf5206e713e37aa8ae6364f6c117e0b8d668fc19441b9/ice_pick-0.4.tar.gz" } ], "0.5": [ { "comment_text": "", "digests": { "md5": "bdc6aa91adcc32879919de13662e6d9b", "sha256": "35ab083b609844e6036fc65b2fbad25c371a555e8f0579fc289963132d5a7072" }, "downloads": -1, "filename": "ice_pick-0.5.tar.gz", "has_sig": false, "md5_digest": "bdc6aa91adcc32879919de13662e6d9b", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17671, "upload_time": "2015-08-21T20:33:44", "url": "https://files.pythonhosted.org/packages/c7/6c/d21e93936f663d64b4ca9bbaa656c008c0d71f1a0bc76aa1f7d117289ecf/ice_pick-0.5.tar.gz" } ], "0.6": [ { "comment_text": "", "digests": { "md5": "7b84880ffe9dfceae50a8ad37a0b0146", "sha256": "a719720c88953be358c9f682803973e53b46724e9a24c6f043b15d3f365d7e30" }, "downloads": -1, "filename": "ice_pick-0.6.tar.gz", "has_sig": false, "md5_digest": "7b84880ffe9dfceae50a8ad37a0b0146", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17693, "upload_time": "2016-04-22T23:10:19", "url": "https://files.pythonhosted.org/packages/27/33/25d50ae70a40cb3d1f055c43516b68542f8f578b479d4bf0adb4d21a07db/ice_pick-0.6.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "7b84880ffe9dfceae50a8ad37a0b0146", "sha256": "a719720c88953be358c9f682803973e53b46724e9a24c6f043b15d3f365d7e30" }, "downloads": -1, "filename": "ice_pick-0.6.tar.gz", "has_sig": false, "md5_digest": "7b84880ffe9dfceae50a8ad37a0b0146", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 17693, "upload_time": "2016-04-22T23:10:19", "url": "https://files.pythonhosted.org/packages/27/33/25d50ae70a40cb3d1f055c43516b68542f8f578b479d4bf0adb4d21a07db/ice_pick-0.6.tar.gz" } ] }