{ "info": { "author": "Open Knowledge International", "author_email": "", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.6", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# AWS Extensions for datapackage-pipelines\n\n[![Build Status](https://travis-ci.org/frictionlessdata/datapackage-pipelines-aws.svg?branch=master)](https://travis-ci.org/frictionlessdata/datapackage-pipelines-aws)\n\n## Install\n\n```\n# clone the repo and install it wit pip\n\ngit clone https://github.com/frictionlessdata/datapackage-pipelines-aws.git\npip install -e .\n```\n\n## Usage\n\nYou can use datapackage-pipelines-aws as a plugin for [dpp](https://github.com/frictionlessdata/datapackage-pipelines#datapackage-pipelines). In pipeline-spec.yaml it will look like this\n\n```yaml\n ...\n - run: aws.dump.to_s3\n```\n\nYou will need AWS credentials to be set up. See [the guide to set up the credentials](http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html)\n\n### dump.to_s3\n\nSaves the DataPackage to AWS S3.\n\n_Parameters:_\n\n* `bucket` - Name of the bucket where DataPackage will be stored (should already be created!)\n* `acl` - ACL to provide the uploaded files. Default is 'public-read' (see [boto3 docs](http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.put_object) for more info).\n* `path` - Path (key/prefix) to the DataPackage. May contain format string available for `datapackage.json` Eg: `my/example/path/{owner}/{name}/{version}`\n* `content_type` - content type to use when storing files in S3. Defaults to text/plain (usual S3 default is binary/octet-stream but we prefer text/plain).\n* `endpoint_url` - api endpoint to allow using S3 compatible services (e.g. 'https://ams3.digitaloceanspaces.com')\n\n_Example:_\n\n```yaml\ndatahub:\n title: datahub-to-s3\n pipeline:\n -\n run: load_metadata\n parameters:\n url: http://example.com/my-datapackage/datapackage.json\n -\n run: load_resource\n parameters:\n url: http://example.com/my-datapackage/datapackage.json\n resource: my-resource\n -\n run: aws.dump.to_s3\n parameters:\n bucket: my.bucket.name\n path: path/{owner}/{name}/{version}\n -\n run: aws.dump.to_s3\n parameters:\n bucket: my.another.bucket\n path: another/path/{version}\n acl: private\n```\n\nExecuting pipeline above will save DataPackage in the following directories on S3:\n* my.bucket.name/path/my-name/py-package-name/latest/...\n* my.bucket.name/another/path/latest/...\n\n\n### change_acl\n\nChanges ACL of object in given Bucket with given path aka prefix.\n\n_Parameters:_\n\n* `bucket` - Name of the bucket where objects are stored\n* `acl` - Available options `'private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control'`\n* `path` - Path (key/prefix) to the DataPackage.\n\n_Example:_\n\n```yaml\ndatahub:\n title: datahub-to-s3\n pipeline:\n -\n run: load_metadata\n parameters:\n url: http://example.com/my-datapackage/datapackage.json\n -\n run: load_resource\n parameters:\n url: http://example.com/my-datapackage/datapackage.json\n resource: my-resource\n -\n run: aws.dump.to_s3\n parameters:\n bucket: my.bucket.name\n path: path/{owner}/{name}/{version}\n -\n run: aws.change_acl\n parameters:\n bucket: my.another.bucket\n path: path/\n acl: private\n```\n\nExecuting pipeline above will save DataPackage on S3 and change ACL to private to all keys prefixed `path`:\n\n**Note:** If path parameter is not set this will change ACL for all object in given bucket", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/frictionlessdata/datapackage_pipelines_aws", "keywords": "data", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "datapackage-pipelines-aws", "package_url": "https://pypi.org/project/datapackage-pipelines-aws/", "platform": "", "project_url": "https://pypi.org/project/datapackage-pipelines-aws/", "project_urls": { "Homepage": "https://github.com/frictionlessdata/datapackage_pipelines_aws" }, "release_url": "https://pypi.org/project/datapackage-pipelines-aws/0.0.23/", "requires_dist": null, "requires_python": "", "summary": "{{ DESCRIPTION }}", "version": "0.0.23" }, "last_serial": 4065482, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "a015f0e86e15d2a69e70519133253542", "sha256": "f04e18591148ff398021f7f5fdbca7cb8b01d4348eec00aa01ee211d583ef81b" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.1.tar.gz", "has_sig": false, "md5_digest": "a015f0e86e15d2a69e70519133253542", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3034882, "upload_time": "2017-07-11T06:45:16", "url": "https://files.pythonhosted.org/packages/65/ba/faf3ae2eda85a9cc8afbcc46290c6e3f6597e2a71c65214eb0a4112cf9c5/datapackage-pipelines-aws-0.0.1.tar.gz" } ], "0.0.10": [ { "comment_text": "", "digests": { "md5": "138bac85f9824c8c8fa67dec6ae7c994", "sha256": "3bbe9cdcc1dbafd9c9113e8b2cf3d8bc4126e895a963535535925fa4a62195ca" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.10.tar.gz", "has_sig": false, "md5_digest": "138bac85f9824c8c8fa67dec6ae7c994", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5308, "upload_time": "2017-11-04T14:51:22", "url": "https://files.pythonhosted.org/packages/58/20/a23ae9e48c3dd68d2b555a758b2bc4416d433e275c6596ecfb1bab3c4b54/datapackage-pipelines-aws-0.0.10.tar.gz" } ], "0.0.11": [ { "comment_text": "", "digests": { "md5": "f383212716219f7db1a596fff7d4c461", "sha256": "d80cbf16d804e49e46fcfd08c900c02962477195a8d6ae48b362327f8cc96c84" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.11.tar.gz", "has_sig": false, "md5_digest": "f383212716219f7db1a596fff7d4c461", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5310, "upload_time": "2017-11-07T17:25:17", "url": "https://files.pythonhosted.org/packages/70/32/b42754a397963838ea47542825ab8639424aa23e577049f7784f90250be2/datapackage-pipelines-aws-0.0.11.tar.gz" } ], "0.0.12": [ { "comment_text": "", "digests": { "md5": "72d6761c38e21034018cd07687d2e513", "sha256": "fe51baf578832ed8b3db9bd1f801c3e72164fb67fbe71d9eb34ca418ac9721f4" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.12.tar.gz", "has_sig": false, "md5_digest": "72d6761c38e21034018cd07687d2e513", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5360, "upload_time": "2017-11-07T18:04:28", "url": "https://files.pythonhosted.org/packages/1e/cd/77c00f104cbbf31da33fc2dbf5cfab2962adda42cb4aa081e82b8994cb15/datapackage-pipelines-aws-0.0.12.tar.gz" } ], "0.0.13": [ { "comment_text": "", "digests": { "md5": "16fcccad4458cb4ba309a603747a4ec3", "sha256": "0e3a555bf80cdf83b82782030a78f9efcd15aaa0841639e07feec3e952a57680" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.13.tar.gz", "has_sig": false, "md5_digest": "16fcccad4458cb4ba309a603747a4ec3", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5963, "upload_time": "2017-11-20T10:24:36", "url": "https://files.pythonhosted.org/packages/52/ac/386529abda4e262a05e71f0b33618970b223ea2570adb8d24a5138e3e8d8/datapackage-pipelines-aws-0.0.13.tar.gz" } ], "0.0.14": [ { "comment_text": "", "digests": { "md5": "ec7f21668e481766b640777115625209", "sha256": "d6f781b8bacff310ce7718c800543ce924086dc5e11cf7de8d1b71dc14a4533c" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.14.tar.gz", "has_sig": false, "md5_digest": "ec7f21668e481766b640777115625209", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5972, "upload_time": "2017-11-21T09:45:21", "url": "https://files.pythonhosted.org/packages/12/1c/1293941c983df37d935e878a46099a58156c12a11f9cd951390f71e9d328/datapackage-pipelines-aws-0.0.14.tar.gz" } ], "0.0.15": [ { "comment_text": "", "digests": { "md5": "2c60d54f5a134a035b0d26e0239a3833", "sha256": "e5a8d6356a41fab95ba277cae79ab896819739e032a4a65752e19dc52b292c7c" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.15.tar.gz", "has_sig": false, "md5_digest": "2c60d54f5a134a035b0d26e0239a3833", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6015, "upload_time": "2017-11-21T12:59:24", "url": "https://files.pythonhosted.org/packages/ec/07/72ebfc85222e1e3b947850e54c3f6c5623d419f3daf16aa58261c270a702/datapackage-pipelines-aws-0.0.15.tar.gz" } ], "0.0.16": [ { "comment_text": "", "digests": { "md5": "041debeb012cc460c84799f80d5f051e", "sha256": "13a9db0de756b8084628e0d193f939eefd9f04db6b48423af9625b118a8beca5" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.16.tar.gz", "has_sig": false, "md5_digest": "041debeb012cc460c84799f80d5f051e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6007, "upload_time": "2017-11-21T13:07:24", "url": "https://files.pythonhosted.org/packages/83/18/dfcd159293a3aa543b1a42643fd0cf91a81ba5cb6180bff695735eb27158/datapackage-pipelines-aws-0.0.16.tar.gz" } ], "0.0.17": [ { "comment_text": "", "digests": { "md5": "f96cd657450a1188af07febb6993a2bb", "sha256": "6d23aa2980a8f79595457982546b840ddb1370f5b88089a0e7b68c441e315dac" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.17.tar.gz", "has_sig": false, "md5_digest": "f96cd657450a1188af07febb6993a2bb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6036, "upload_time": "2017-12-10T16:19:08", "url": "https://files.pythonhosted.org/packages/69/98/810c1a678433e088879106b867e560d4aa59024c4d818235227ae7de54f5/datapackage-pipelines-aws-0.0.17.tar.gz" } ], "0.0.18": [ { "comment_text": "", "digests": { "md5": "d498946fe2847c2a7c245ec47fc3a509", "sha256": "7133e9e2d4d1351fe5c15e87a0cf2c3b7542b5405dc019e8d7d514841c8715a9" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.18.tar.gz", "has_sig": false, "md5_digest": "d498946fe2847c2a7c245ec47fc3a509", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6037, "upload_time": "2017-12-10T18:43:08", "url": "https://files.pythonhosted.org/packages/6a/ed/17b3a10d78ca0f48a3a26c0f429856ae961fed4a0e81479af1ca0cd2f155/datapackage-pipelines-aws-0.0.18.tar.gz" } ], "0.0.19": [ { "comment_text": "", "digests": { "md5": "d35668e0ae02127c73bfd9bec43d7c2d", "sha256": "f06a2375c28032d7bcee9f5b0a7934c5347c3195847d6c03a20957adf764c329" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.19.tar.gz", "has_sig": false, "md5_digest": "d35668e0ae02127c73bfd9bec43d7c2d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6067, "upload_time": "2017-12-11T07:38:35", "url": "https://files.pythonhosted.org/packages/cf/bf/0063500a6b82313b2ec69771d38c4f3a68c159e6270c32485f439bf3b8bb/datapackage-pipelines-aws-0.0.19.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "cbf293c783084800cac68b8d2aa87637", "sha256": "f0da40257884519214539a9875481b47a8e732c70b48e97ebd751de1dad11be3" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.2.tar.gz", "has_sig": false, "md5_digest": "cbf293c783084800cac68b8d2aa87637", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4285, "upload_time": "2017-07-12T20:29:34", "url": "https://files.pythonhosted.org/packages/a7/3c/60e624a85c345d55a2905a10da693c930ba1ee6c3c4d697c6117144754fc/datapackage-pipelines-aws-0.0.2.tar.gz" } ], "0.0.20": [ { "comment_text": "", "digests": { "md5": "9f7699f080e22f924ff80dae2f1d18c8", "sha256": "63622b0ef4f0b5cff24f384ec4768b58162040c9791b2cb74f3463319e9bf7ac" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.20.tar.gz", "has_sig": false, "md5_digest": "9f7699f080e22f924ff80dae2f1d18c8", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6106, "upload_time": "2018-01-22T18:47:45", "url": "https://files.pythonhosted.org/packages/bb/3d/fb4e3e2adaf5574105e6e3e9a4cc0df5b0ee3000df9e6dc6eda89dd7eaa1/datapackage-pipelines-aws-0.0.20.tar.gz" } ], "0.0.21": [ { "comment_text": "", "digests": { "md5": "335c53e8dc01eb73eaeae7182812f550", "sha256": "618a3a71bd71ad65d91402022e57dac95f0d6e37461b807faca5d2635bba83a3" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.21.tar.gz", "has_sig": false, "md5_digest": "335c53e8dc01eb73eaeae7182812f550", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6193, "upload_time": "2018-07-06T10:56:28", "url": "https://files.pythonhosted.org/packages/1a/1b/9acb0c54712f6428aa6787d88f65e6c0c3a358e5a2e7db9eb56a887a7d73/datapackage-pipelines-aws-0.0.21.tar.gz" } ], "0.0.22": [ { "comment_text": "", "digests": { "md5": "dd0b44e10e4228bff90abeeb37dd0fc9", "sha256": "9d7fbead17a72eea972e23fed81650b691b258008300aa8ad7a50e81d7c4e9c6" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.22.tar.gz", "has_sig": false, "md5_digest": "dd0b44e10e4228bff90abeeb37dd0fc9", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6191, "upload_time": "2018-07-06T11:13:58", "url": "https://files.pythonhosted.org/packages/41/26/1ecee65f560c17ee0b4464cc41c3224f62324b47db8d904a0ff79874f56c/datapackage-pipelines-aws-0.0.22.tar.gz" } ], "0.0.23": [ { "comment_text": "", "digests": { "md5": "6104b539425464393b1c3620e23cc8e7", "sha256": "7bb5ddeaa3c3608090f8cf8d94a1665e52f985cf1221b33e3919ab478927b161" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.23.tar.gz", "has_sig": false, "md5_digest": "6104b539425464393b1c3620e23cc8e7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6275, "upload_time": "2018-07-16T12:04:20", "url": "https://files.pythonhosted.org/packages/17/f7/e19175c57e765cc481baad64043497e3b13c2b3d74583a78bd8f17ec1d34/datapackage-pipelines-aws-0.0.23.tar.gz" } ], "0.0.3": [ { "comment_text": "", "digests": { "md5": "b0e61b8c513db6f8468e4e6917677c97", "sha256": "391dbcf86f71ba2d7b7684fd437a6cddd6ed0eaf2b8f99c9b94b9ce0cb184cb0" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.3.tar.gz", "has_sig": false, "md5_digest": "b0e61b8c513db6f8468e4e6917677c97", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4603, "upload_time": "2017-07-13T04:51:40", "url": "https://files.pythonhosted.org/packages/ec/3d/bb375658a07f0ad83f753b5d2da503aee919d1422660153a93827a7bf243/datapackage-pipelines-aws-0.0.3.tar.gz" } ], "0.0.4": [ { "comment_text": "", "digests": { "md5": "f3eac84ca42f3ccaf35f9932501d6d90", "sha256": "ad6fb438ab3b103df9738a35fcdfc06fb43e9d91668bbcd22a535d04e65d6db6" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.4.tar.gz", "has_sig": false, "md5_digest": "f3eac84ca42f3ccaf35f9932501d6d90", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 3044790, "upload_time": "2017-07-19T07:20:08", "url": "https://files.pythonhosted.org/packages/89/2a/aa1cf21d22b8037df86e3ba3c74ca73df6aa581dafe732ee04637361aac0/datapackage-pipelines-aws-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "de15fa5049d90f2b9cda176427e5ec88", "sha256": "37e7005cd13d2d0afc10de136438661c79139c0561992e7d64c7c943ff77c797" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.5.tar.gz", "has_sig": false, "md5_digest": "de15fa5049d90f2b9cda176427e5ec88", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4804, "upload_time": "2017-07-21T17:19:04", "url": "https://files.pythonhosted.org/packages/4a/45/1129524f8752915389b9c1b54a90cb9d16e2e62ef5b5414004e336de7a68/datapackage-pipelines-aws-0.0.5.tar.gz" } ], "0.0.6": [ { "comment_text": "", "digests": { "md5": "7633d7ac9e50bceb4f8f4444d9c90afe", "sha256": "628ec819af16ca08bc9f9682cf7a112c70f6c56237f24b8a37dd1807616808ee" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.6.tar.gz", "has_sig": false, "md5_digest": "7633d7ac9e50bceb4f8f4444d9c90afe", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4762, "upload_time": "2017-08-05T16:42:39", "url": "https://files.pythonhosted.org/packages/ea/20/fe47f7ccd8f85ba0e6d28be65e10f2db3dd662723f1c5effc327d0e99125/datapackage-pipelines-aws-0.0.6.tar.gz" } ], "0.0.8": [ { "comment_text": "", "digests": { "md5": "258300351758a414343f89b66d11cbdb", "sha256": "c03ab12f0c573205c9beba25bfd4de21b9e6e4f38c520435051a6508ceaa7e44" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.8.tar.gz", "has_sig": false, "md5_digest": "258300351758a414343f89b66d11cbdb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4953, "upload_time": "2017-08-05T20:18:14", "url": "https://files.pythonhosted.org/packages/3f/88/14a2f5d7ed1b1a539a69290fbe4009e4ec528aeb9c11ae82580a6e195320/datapackage-pipelines-aws-0.0.8.tar.gz" } ], "0.0.9": [ { "comment_text": "", "digests": { "md5": "2cf29f82003d8768bed69600663d49a0", "sha256": "42a94817d18ef26e349a0f75dfeb9751789d792f7f75ed274870651d94b8ff6d" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.9.tar.gz", "has_sig": false, "md5_digest": "2cf29f82003d8768bed69600663d49a0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5269, "upload_time": "2017-10-25T13:24:48", "url": "https://files.pythonhosted.org/packages/ad/8e/52edd3db7425a9580f1211e07e5ec44bda70315f94f1c3941beab15d7a74/datapackage-pipelines-aws-0.0.9.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "6104b539425464393b1c3620e23cc8e7", "sha256": "7bb5ddeaa3c3608090f8cf8d94a1665e52f985cf1221b33e3919ab478927b161" }, "downloads": -1, "filename": "datapackage-pipelines-aws-0.0.23.tar.gz", "has_sig": false, "md5_digest": "6104b539425464393b1c3620e23cc8e7", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 6275, "upload_time": "2018-07-16T12:04:20", "url": "https://files.pythonhosted.org/packages/17/f7/e19175c57e765cc481baad64043497e3b13c2b3d74583a78bd8f17ec1d34/datapackage-pipelines-aws-0.0.23.tar.gz" } ] }