{ "info": { "author": "Open Knowledge Foundation", "author_email": "info@okfn.org", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Programming Language :: Python :: 3.7", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "# dataflows-aws\n\n[![Travis](https://travis-ci.org/frictionlessdata/dataflows-aws.svg?branch=master)](https://travis-ci.org/frictionlessdata/dataflows-aws)\n[![Coveralls](http://img.shields.io/coveralls/frictionlessdata/dataflows-aws.svg?branch=master)](https://coveralls.io/r/frictionlessdata/dataflows-aws?branch=master)\n\nDataflows's processors to work with AWS\n\n## Features\n\n- `dump_to_s3` processor\n- `change_acl_on_s3` processor\n\n## Contents\n\n\n\n - [Getting Started](#getting-started)\n - [Installation](#installation)\n - [Examples](#examples)\n - [Documentation](#documentation)\n - [dump_to_s3](#dump_to_s3)\n - [change_acl_on_s3](#change_acl_on_s3)\n - [Contributing](#contributing)\n - [Changelog](#changelog)\n\n\n\n## Getting Started\n\n### Installation\n\nThe package use semantic versioning. It means that major versions could include breaking changes. It's recommended to specify `package` version range in your `setup/requirements` file e.g. `package>=1.0,<2.0`.\n\n```bash\n$ pip install dataflows-aws\n```\n\n### Examples\n\nThese processors have to be used as a part of data flow. For example:\n\n```python\nflow = Flow(\n load('data/data.csv'),\n dump_to_s3(\n bucket=bucket,\n acl='private',\n path='my/datapackage',\n endpoint_url=os.environ['S3_ENDPOINT_URL'],\n ),\n)\nflow.process()\n```\n\n## Documentation\n\n### dump_to_s3\n\nSaves the DataPackage to AWS S3.\n\n#### Parameters\n\n- `bucket` - Name of the bucket where DataPackage will be stored (should already be created!)\n- `acl` - ACL to provide the uploaded files. Default is 'public-read' (see [boto3 docs](http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.put_object) for more info).\n- `path` - Path (key/prefix) to the DataPackage. May contain format string available for `datapackage.json` Eg: `my/example/path/{owner}/{name}/{version}`\n- `content_type` - content type to use when storing files in S3. Defaults to text/plain (usual S3 default is binary/octet-stream but we prefer text/plain).\n- `endpoint_url` - api endpoint to allow using S3 compatible services (e.g. 'https://ams3.digitaloceanspaces.com')\n\n### change_acl_on_s3\n\nChanges ACL of object in given Bucket with given path aka prefix.\n\n#### Parameters\n\n- `bucket` - Name of the bucket where objects are stored\n- `acl` - Available options `'private'|'public-read'|'public-read-write'|'authenticated-read'|'aws-exec-read'|'bucket-owner-read'|'bucket-owner-full-control'`\n- `path` - Path (key/prefix) to the DataPackage.\n- `endpoint_url` - api endpoint to allow using S3 compatible services (e.g. 'https://ams3.digitaloceanspaces.com')\n\n## Contributing\n\nThe project follows the [Open Knowledge International coding standards](https://github.com/okfn/coding-standards).\n\nThe recommended way to get started is to create and activate a project virtual environment.\nTo install package and development dependencies into your active environment:\n\n```\n$ make install\n```\n\nTo run tests with linting and coverage:\n\n```bash\n$ make test\n```\n\nFor linting, `pylama` (configured in `pylama.ini`) is used. At this stage it's already\ninstalled into your environment and could be used separately with more fine-grained control\nas described in documentation - https://pylama.readthedocs.io/en/latest/.\n\nFor example to sort results by error type:\n\n```bash\n$ pylama --sort \n```\n\nFor testing, `tox` (configured in `tox.ini`) is used.\nIt's already installed into your environment and could be used separately with more fine-grained control as described in documentation - https://testrun.org/tox/latest/.\n\nFor example to check subset of tests against Python 2 environment with increased verbosity.\nAll positional arguments and options after `--` will be passed to `py.test`:\n\n```bash\ntox -e py37 -- -v tests/\n```\n\nUnder the hood `tox` uses `pytest` (configured in `pytest.ini`), `coverage`\nand `mock` packages. These packages are available only in tox envionments.\n\n## Changelog\n\nHere described only breaking and the most important changes. The full changelog and documentation for all released versions can be found in the nicely formatted [commit history](https://github.com/frictionlessdata/dataflows-aws/commits/master).\n\n#### v0.x\n\n- an initial processors implementation\n\n", "description_content_type": "text/markdown", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/frictionlessdata/dataflows-aws", "keywords": "frictionless data,open data,json schema,table schema,data package,tabular data package", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "dataflows-aws", "package_url": "https://pypi.org/project/dataflows-aws/", "platform": "", "project_url": "https://pypi.org/project/dataflows-aws/", "project_urls": { "Homepage": "https://github.com/frictionlessdata/dataflows-aws" }, "release_url": "https://pypi.org/project/dataflows-aws/0.2.1/", "requires_dist": [ "six", "boto3", "dataflows", "moto[server] ; extra == 'develop'", "pytest-cov ; extra == 'develop'", "pytest ; extra == 'develop'", "pylama ; extra == 'develop'", "mock ; extra == 'develop'", "tox ; extra == 'develop'" ], "requires_python": "", "summary": "A utility library for working with Table Schema in Python", "version": "0.2.1" }, "last_serial": 5724124, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "daaabe1d156a81c654db53dcb28af287", "sha256": "df0be12540765510ccb2ed7815862d30bfb5ef45a9e9f8c63a9684e806649b5c" }, "downloads": -1, "filename": "dataflows_aws-0.1.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "daaabe1d156a81c654db53dcb28af287", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 2934, "upload_time": "2019-08-22T11:57:54", "url": "https://files.pythonhosted.org/packages/40/7b/a9eb8aac16b37394a57a4877bbeeb2eadd4523e310121a094859dd0443f1/dataflows_aws-0.1.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "58cc2fee225d70738f12c6e8cbb14e70", "sha256": "b3f1d160654f1ed5deaad95dbf5630d4fcf19cf5bcd002acad3045c7565d67ef" }, "downloads": -1, "filename": "dataflows-aws-0.1.0.tar.gz", "has_sig": false, "md5_digest": "58cc2fee225d70738f12c6e8cbb14e70", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 58176, "upload_time": "2019-08-22T11:57:57", "url": "https://files.pythonhosted.org/packages/74/cc/e87b0426f5509776562fc699e881b6467d1a943e7a51280c27cf140af8bf/dataflows-aws-0.1.0.tar.gz" } ], "0.2.0": [ { "comment_text": "", "digests": { "md5": "2ea5cb9f9d31a355a89e33d2a818a9a9", "sha256": "b3f9ba640fc66fa9c7a0c0bc0f0a4efb3925ba989297fd6f906173553cc6750e" }, "downloads": -1, "filename": "dataflows_aws-0.2.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "2ea5cb9f9d31a355a89e33d2a818a9a9", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6928, "upload_time": "2019-08-24T09:44:28", "url": "https://files.pythonhosted.org/packages/da/ad/9544cd1e25825a0a32adef5f5d7dfd2fd805e016333d3e0c408f92b60e0b/dataflows_aws-0.2.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "2a466568afdb5a883e155cfc7a74e39e", "sha256": "47165c33916a1c7b09c97ded1c7a35a32bb802e594ff1849a48e7179b774e931" }, "downloads": -1, "filename": "dataflows-aws-0.2.0.tar.gz", "has_sig": false, "md5_digest": "2a466568afdb5a883e155cfc7a74e39e", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7164198, "upload_time": "2019-08-24T09:44:30", "url": "https://files.pythonhosted.org/packages/45/3f/dae33256ead4a01849a67e89a9931c1eb26aa969bd163a80a3dffe7a0d4c/dataflows-aws-0.2.0.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "69acf14e47da3c27aea7dcdfdb9c4634", "sha256": "a0397cd36d3a52e9fc92a769978481da2d42506e3ceb79771193cf7fda3bdb64" }, "downloads": -1, "filename": "dataflows_aws-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "69acf14e47da3c27aea7dcdfdb9c4634", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6906, "upload_time": "2019-08-24T09:46:45", "url": "https://files.pythonhosted.org/packages/24/3c/85d9b68e41c47c0044cb3c00839ac4481808123f13658dc23d97d46646b0/dataflows_aws-0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5324948f86de2e358108832d5bae9414", "sha256": "5c7a2f79c8ab6840189cdbb873f1916d779cc436263fed2605ed1bf688f66d14" }, "downloads": -1, "filename": "dataflows-aws-0.2.1.tar.gz", "has_sig": false, "md5_digest": "5324948f86de2e358108832d5bae9414", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7163907, "upload_time": "2019-08-24T09:46:47", "url": "https://files.pythonhosted.org/packages/e7/73/8da2db4af7d239c046319e7d0cf34046c8b7aaf3f1243df4f768bb826247/dataflows-aws-0.2.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "69acf14e47da3c27aea7dcdfdb9c4634", "sha256": "a0397cd36d3a52e9fc92a769978481da2d42506e3ceb79771193cf7fda3bdb64" }, "downloads": -1, "filename": "dataflows_aws-0.2.1-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "69acf14e47da3c27aea7dcdfdb9c4634", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": null, "size": 6906, "upload_time": "2019-08-24T09:46:45", "url": "https://files.pythonhosted.org/packages/24/3c/85d9b68e41c47c0044cb3c00839ac4481808123f13658dc23d97d46646b0/dataflows_aws-0.2.1-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "5324948f86de2e358108832d5bae9414", "sha256": "5c7a2f79c8ab6840189cdbb873f1916d779cc436263fed2605ed1bf688f66d14" }, "downloads": -1, "filename": "dataflows-aws-0.2.1.tar.gz", "has_sig": false, "md5_digest": "5324948f86de2e358108832d5bae9414", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7163907, "upload_time": "2019-08-24T09:46:47", "url": "https://files.pythonhosted.org/packages/e7/73/8da2db4af7d239c046319e7d0cf34046c8b7aaf3f1243df4f768bb826247/dataflows-aws-0.2.1.tar.gz" } ] }