{ "info": { "author": "Maxim Tacu", "author_email": "maxtacu95@gmail.com", "bugtrack_url": null, "classifiers": [ "Intended Audience :: System Administrators", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.6", "Topic :: System :: Systems Administration" ], "description": "==============\n AWS-Datapipe\n==============\n\nA tool for creating AWS data pipeline. Currently only for **exporting data from any DynamoDB table to an existing S3 bucket** \nand store the Pipeline Definition as a backup. New features are going to be added (check for updates).\n\n`Just because terraform can't do it yet..`\n\n---------------\n Prerequisites\n---------------\n\n**This tool supports only Python 3 because Python 2.7 will not be maintained past 2020**\n\nAWS-Datapipe uses awscli to configure pipelines in your AWS account. By default, installing aws-datapipe you will also\ninstall all the prerequisites, so you can skip this phase and move to the `Installation step`. But in case if you want to do it manually:\n\nThe easiest way to install aws-cli is to use `pip `_ in a ``virtualenv``::\n\n $ pip install awscli\n\nor, if you are not installing in a ``virtualenv``, to install globally::\n\n $ sudo pip install awscli\n\nor for your user::\n\n $ pip install --user awscli\n\nIf you have the aws-cli installed and want to upgrade to the latest version\nyou can run::\n\n $ pip install --upgrade awscli\n\nBecause aws-datapipe is based on aws-cli, before using aws-datapipe, \nyou need to configure your AWS credentials using aws-cli. \nYou can do this in several ways:\n\n* Environment variables\n* Shared credentials file\n* Config file\n* IAM Role\n\nThe quickest way to get started is to run the ``aws configure`` command::\n\n $ aws configure\n AWS Access Key ID: foo\n AWS Secret Access Key: bar\n Default region name [us-west-2]: us-west-2\n Default output format [None]: json\n\nFor additional info and other ways of aws-cli configuration you can check `here `_\n\n---------------\n Installing\n---------------\n\nThe easiest way to install aws-datapipe is to use `pip `_ in a ``virtualenv``::\n\n $ pip install aws-datapipe\n\nor, if you are not installing in a ``virtualenv``, to install globally::\n\n $ sudo pip install aws-datapipe\n\nor for your user::\n\n $ pip install --user aws-datapipe\n\nIf you have the aws-datapipe installed and want to upgrade to the latest version\nyou can run::\n\n $ pip install --upgrade aws-datapipe\n\n\nEnd with an example of getting some data out of the system or using it for a little demo\n\n---------------\nGetting Started\n---------------\nYou can use AWS Datapipe in two different ways: \n\n* interactive\n* passing arguments\n\nTo list all available arguments and additional info check please the ``--help``.\n\nFor interactive mode just start the tool ``aws-datapipe`` and it will ask you for the required information to build a pipeline.\n\nAfter creating data pipeline in AWS, datapipe will save locally a `pipeline definition file` with the same name as your \npipeline that can be used as a backup. After that you can activate the pipeline directly in the script on the next step:\n\n``Would you like to activate the pipeline now? [y/n]`` \n\nYou can omit activation and manually activate the pipeline later whenever you want from the AWS Console\n\nCurrently ``aws-datapipe`` is creating pipeline which will **run every 14 days after activation**. \n\nSchedule configuration is coming soon.. check for updates\n\n-----------\nVersioning\n-----------\n\nI use `SemVer `_ for versioning. For the versions available, see the `tags on this repository `_. \n\n-------\nAuthor\n-------\n\n* **Maxim Tacu** - Site Reliability Engineer - `OLX Group `_.\n\n-------\nLicense\n-------\n\nThis project is licensed under the MIT License - see the `LICENSE `_ file for details\n\n\nHow to Contribute\n-----------------\n\nContributions are very welcome. The easiest way is to fork this repo, and then\nmake a pull request from your fork.", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/tmxak/aws-datapipe", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "aws-datapipe", "package_url": "https://pypi.org/project/aws-datapipe/", "platform": "", "project_url": "https://pypi.org/project/aws-datapipe/", "project_urls": { "Homepage": "https://github.com/tmxak/aws-datapipe" }, "release_url": "https://pypi.org/project/aws-datapipe/0.1.3/", "requires_dist": null, "requires_python": "", "summary": "Tool for creating AWS data pipeline to export data from any DynamoDB table to an S3 bucket", "version": "0.1.3" }, "last_serial": 4811779, "releases": { "0.1.1": [ { "comment_text": "", "digests": { "md5": "76b0019dcd66fb4d7ef5ed331cc8ec22", "sha256": "4fa77588b8531674469aefce36cd107d79601098cdd62143b2ccc34257df2274" }, "downloads": -1, "filename": "aws-datapipe-0.1.1.tar.gz", "has_sig": false, "md5_digest": "76b0019dcd66fb4d7ef5ed331cc8ec22", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5409, "upload_time": "2019-02-05T10:56:24", "url": "https://files.pythonhosted.org/packages/4a/da/793abfc15fb982115149d02e010838b32b13a4e567dcefa7169cb4a6d668/aws-datapipe-0.1.1.tar.gz" } ], "0.1.2": [ { "comment_text": "", "digests": { "md5": "a7f847180ffb9b933b3c6889ecf8481d", "sha256": "71c6febb40ef3a035966d76aac5c5ab3da094947decdaaa4ef4f75a8a728c1aa" }, "downloads": -1, "filename": "aws-datapipe-0.1.2.tar.gz", "has_sig": false, "md5_digest": "a7f847180ffb9b933b3c6889ecf8481d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 4997, "upload_time": "2019-02-05T18:21:37", "url": "https://files.pythonhosted.org/packages/5d/38/908c99a4af5b45d02577729334826d69f454de32dedb9f9e0b3987250412/aws-datapipe-0.1.2.tar.gz" } ], "0.1.3": [ { "comment_text": "", "digests": { "md5": "1d3a8de75ce4e2b69bf4869cd22566a4", "sha256": "84e90b9a8702cb19f06141a25ed46dde729d3b528c3aa46ede285849df2e1e49" }, "downloads": -1, "filename": "aws-datapipe-0.1.3.tar.gz", "has_sig": false, "md5_digest": "1d3a8de75ce4e2b69bf4869cd22566a4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5913, "upload_time": "2019-02-12T16:41:50", "url": "https://files.pythonhosted.org/packages/17/9a/8b2d2073a6b268243df51df8fb3b76a06c55efe9e7739d4f544359814b0a/aws-datapipe-0.1.3.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "1d3a8de75ce4e2b69bf4869cd22566a4", "sha256": "84e90b9a8702cb19f06141a25ed46dde729d3b528c3aa46ede285849df2e1e49" }, "downloads": -1, "filename": "aws-datapipe-0.1.3.tar.gz", "has_sig": false, "md5_digest": "1d3a8de75ce4e2b69bf4869cd22566a4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5913, "upload_time": "2019-02-12T16:41:50", "url": "https://files.pythonhosted.org/packages/17/9a/8b2d2073a6b268243df51df8fb3b76a06c55efe9e7739d4f544359814b0a/aws-datapipe-0.1.3.tar.gz" } ] }