{ "info": { "author": "Seperman", "author_email": "sep@zepworks.com", "bugtrack_url": null, "classifiers": [ "Environment :: Web Environment", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Topic :: Internet :: WWW/HTTP", "Topic :: Internet :: WWW/HTTP :: Dynamic Content", "Topic :: Software Development" ], "description": "# S3 Utils v0.6.0\n\n![Python Versions](https://img.shields.io/pypi/pyversions/s3utils.svg?style=flat)\n![Doc](https://readthedocs.org/projects/s3utils/badge/?version=latest)\n![License](https://img.shields.io/pypi/l/s3utils.svg?version=latest)\n[![Build Status](https://travis-ci.org/seperman/s3utils.svg?branch=master)](https://travis-ci.org/seperman/s3utils)\n\nUser friendly interface to deal with Amazon S3 bucket and Cloud Front in Python.\nI wrote this since s3cmd is for commandline usage and the other libraries out there seemed to be not maintained anymore.\n\nThe s3utils methods are made to be just like Linux commands so it is easy to remember and use.\n\nBehind the scene it is a light Python wrapper around Amazon Boto library.\n\n## Supported Python versions\n\nPython 2.7, 3.3, 3.4 and 3.5 are supported.\n\n\n## Documentation\n* [Documentations](http://s3utils.readthedocs.org/en/latest/)\n\n\n## Installation\n\nInstall from PyPi:\n\n pip install s3utils\n\n## Setup\n\n## Example setup in Django\n\n\nin your settings file::\n\n```python\nS3UTILS_DEBUG_LEVEL=1\nAWS_ACCESS_KEY_ID = 'your access key'\nAWS_SECRET_ACCESS_KEY = 'your secret key'\nAWS_STORAGE_BUCKET_NAME = 'your bucket name'\n```\n\nin your code::\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils()\n>>> s3utils.cp(\"path/to/folder\",\"/test/\")\ncopying /path/to/myfolder/test2.txt to test/myfolder/test2.txt\ncopying /path/to/myfolder/test.txt to test/myfolder/test.txt\ncopying /path/to/myfolder/hoho/photo.JPG to test/myfolder/hoho/photo.JPG\ncopying /path/to/myfolder/hoho/haha/ff to test/myfolder/hoho/haha/ff\n```\n\n## Example manual setup\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> s3utils.cp(\"path/to/folder\",\"/test/\")\ncopying /path/to/myfolder/test2.txt to test/myfolder/test2.txt\ncopying /path/to/myfolder/test.txt to test/myfolder/test.txt\ncopying /path/to/myfolder/hoho/photo.JPG to test/myfolder/hoho/photo.JPG\ncopying /path/to/myfolder/hoho/haha/ff to test/myfolder/hoho/haha/ff\n```\n\n## Mkdir\n\nCreate a folder on S3.\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> s3utils.mkdir(\"path/to/my_folder\")\nMaking directory: path/to/my_folder\n```\n\n## Cp\n\nCopy a file or folder from local to s3\n\n### Parameters\n\nlocal_path : string\nPath to file or folder. Or if you want to copy only the contents of folder, add /* at the end of folder name\n\ntarget_path : string\nTarget path on S3 bucket.\n\nacl : string, optional\n\nFile permissions on S3. Default is public-read\n\nacl options:\n\n- private: Owner gets FULL_CONTROL. No one else has any access rights.\n- public-read: Owners gets FULL_CONTROL and the anonymous principal is granted READ access.\n- public-read-write: Owner gets FULL_CONTROL and the anonymous principal is granted READ and WRITE access.\n- authenticated-read: Owner gets FULL_CONTROL and any principal authenticated as a registered Amazon S3 user is granted READ access\n\ndel_after_upload : boolean, optional\n\ndelete the local file after uploading. This is effectively like moving the file.\nYou can use s3utils.mv instead of s3utils.cp to move files from local to S3.\nIt basically sets this flag to True.\ndefault = False\n\noverwrite : boolean, optional\n\noverwrites files on S3 if set to True. Default is True\n\ninvalidate : boolean, optional\n\ninvalidates the CDN (a.k.a Distribution) cache if the file already exists on S3\ndefault = False\nNote that invalidation might take up to 15 minutes to take place. It is easier and faster to use cache buster\nto grab lastest version of your file on CDN than invalidation.\n\nExamples\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> s3utils.cp(\"path/to/folder\",\"/test/\")\ncopying /path/to/myfolder/test2.txt to test/myfolder/test2.txt\ncopying /path/to/myfolder/test.txt to test/myfolder/test.txt\ncopying /path/to/myfolder/hoho/photo.JPG to test/myfolder/hoho/photo.JPG\ncopying /path/to/myfolder/hoho/haha/ff to test/myfolder/hoho/haha/ff\n\n>>> # When overwrite is set to False:\n>>> s3utils.cp(\"path/to/folder\",\"/test/\", overwrite=False)\ntest/myfolder/test2.txt already exist. Not overwriting.\ntest/myfolder/test.txt already exist. Not overwriting.\ntest/myfolder/hoho/photo.JPG already exist. Not overwriting.\ntest/myfolder/hoho/haha/ff already exist. Not overwriting.\n\n>>> # To overwrite the files on S3 and invalidate the CDN (cloudfront) cache so the new file goes on CDN:\n>>> s3utils.cp(\"path/to/folder\",\"/test/\", overwrite=True, invalidate=True)\ncopying /path/to/myfolder/test2.txt to test/myfolder/test2.txt\ncopying /path/to/myfolder/test.txt to test/myfolder/test.txt\ncopying /path/to/myfolder/hoho/photo.JPG to test/myfolder/hoho/photo.JPG\ncopying /path/to/myfolder/hoho/haha/ff to test/myfolder/hoho/haha/ff\n```\n\n## Mv\n\nMove the file to the S3 and deletes the local copy\n\nIt is basically s3utils.cp that has del_after_upload=True\n\nExamples\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> s3utils.mv(\"path/to/folder\",\"/test/\")\nmoving /path/to/myfolder/test2.txt to test/myfolder/test2.txt\nmoving /path/to/myfolder/test.txt to test/myfolder/test.txt\nmoving /path/to/myfolder/hoho/photo.JPG to test/myfolder/hoho/photo.JPG\nmoving /path/to/myfolder/hoho/haha/ff to test/myfolder/hoho/haha/ff\n```\n\n## Chmod\n\nsets permissions for a file on S3\n\nParameters\n\ntarget_file : string\nPath to file on S3\n\nacl : string, optional\nFile permissions on S3. Default is public-read\n\noptions:\n\n- private: Owner gets FULL_CONTROL. No one else has any access rights.\n- public-read: Owners gets FULL_CONTROL and the anonymous principal is granted READ access.\n- public-read-write: Owner gets FULL_CONTROL and the anonymous principal is granted READ and WRITE access.\n- authenticated-read: Owner gets FULL_CONTROL and any principal authenticated as a registered Amazon S3 user is granted READ access\n\n\nExamples\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> s3utils.chmod(\"path/to/file\",\"private\")\n```\n\n## Ls\n\ngets the list of file names (keys) in a s3 folder\n\nParameters\n\nfolder : string\nPath to file on S3\n\nnum: integer, optional\nnumber of results to return, by default it returns all results.\n\nbegin_from_file: string, optional\nwhich file to start from on S3.\nThis is usedful in case you are iterating over lists of files and you need to page the result by\nstarting listing from a certain file and fetching certain num (number) of files.\n\n\nExamples\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> print s3utils.ls(\"test/\")\n[u'test/myfolder/', u'test/myfolder/em/', u'test/myfolder/hoho/', u'test/myfolder/hoho/.DS_Store', u'test/myfolder/hoho/haha/', u'test/myfolder/hoho/haha/ff', u'test/myfolder/hoho/haha/photo.JPG']\n```\n\n## LL\n\nGet the list of files and permissions from S3\n\nParameters\n\nfolder : string\nPath to file on S3\n\nnum: integer, optional\nnumber of results to return, by default it returns all results.\n\nbegin_from_file : string, optional\nwhich file to start from on S3.\nThis is usedful in case you are iterating over lists of files and you need to page the result by\nstarting listing from a certain file and fetching certain num (number) of files.\n\nall_grant_data : Boolean, optional\nMore detailed file permission data will be returned.\n\nExamples\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> import json\n>>> # We use json.dumps to print the results more readable:\n>>> my_folder_stuff = s3utils.ll(\"/test/\")\n>>> print json.dumps(my_folder_stuff, indent=2)\n{\n \"test/myfolder/\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n }\n ],\n \"test/myfolder/em/\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n }\n ],\n \"test/myfolder/hoho/\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n }\n ],\n \"test/myfolder/hoho/.DS_Store\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n },\n {\n \"name\": null,\n \"permission\": \"READ\"\n }\n ],\n \"test/myfolder/hoho/haha/\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n }\n ],\n \"test/myfolder/hoho/haha/ff\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n },\n {\n \"name\": null,\n \"permission\": \"READ\"\n }\n ],\n \"test/myfolder/hoho/photo.JPG\": [\n {\n \"name\": \"owner's name\",\n \"permission\": \"FULL_CONTROL\"\n },\n {\n \"name\": null,\n \"permission\": \"READ\"\n }\n ],\n}\n```\n\n## Invalidate CDN\n\nInvalidate the CDN (distribution) cache for a certain file of files. This might take up to 15 minutes to be effective.\n\nYou can check for the invalidation status using check_invalidation_request.\n\nExamples\n\n```python\n>>> from s3utils import S3utils\n>>> s3utils = S3utils(\n... AWS_ACCESS_KEY_ID = 'your access key',\n... AWS_SECRET_ACCESS_KEY = 'your secret key',\n... AWS_STORAGE_BUCKET_NAME = 'your bucket name',\n... S3UTILS_DEBUG_LEVEL = 1, #change it to 0 for less verbose\n... )\n>>> aa = s3utils.invalidate(\"test/myfolder/hoho/photo.JPG\")\n>>> print aa\n('your distro id', u'your request id')\n>>> invalidation_request_id = aa[1]\n>>> bb = s3utils.check_invalidation_request(*aa)\n>>> for inval in bb:\n... print 'Object: %s, ID: %s, Status: %s' % (inval, inval.id, inval.status)\n```\n\n##Author\n\nSeperman (Sep Ehr)\n\n* [Zepworks.com](https://zepworks.com)\n* [Github](https://github.com/seperman)\n* [Linkedin](http://www.linkedin.com/in/sepehr)", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/seperman/s3utils/tarball/master", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/seperman/s3utils", "keywords": "", "license": "Apache License 2.0", "maintainer": "", "maintainer_email": "", "name": "s3utils", "package_url": "https://pypi.org/project/s3utils/", "platform": "UNKNOWN", "project_url": "https://pypi.org/project/s3utils/", "project_urls": { "Download": "https://github.com/seperman/s3utils/tarball/master", "Homepage": "https://github.com/seperman/s3utils" }, "release_url": "https://pypi.org/project/s3utils/0.6.1/", "requires_dist": [ "boto (>=2.39.0)" ], "requires_python": "", "summary": "S3 Utils deals with Amazon S3 buckets", "version": "0.6.1" }, "last_serial": 2045137, "releases": { "0.5": [ { "comment_text": "", "digests": { "md5": "41b4c590221122193f834a5c58464f67", "sha256": "6aee887e92cdea41bf6fb205af3866ee06a741e8cce4f30d0e762de46d99a1fe" }, "downloads": -1, "filename": "s3utils-0.5.tar.gz", "has_sig": false, "md5_digest": "41b4c590221122193f834a5c58464f67", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 118523, "upload_time": "2014-04-30T00:10:52", "url": "https://files.pythonhosted.org/packages/a7/f7/bbf01e5dec34d83f32f91895d28ef430bf9aeaa6f53a71bb49cf128ba80b/s3utils-0.5.tar.gz" } ], "0.5.1": [ { "comment_text": "", "digests": { "md5": "bb8393712ca7e91e8c043f8ddf433625", "sha256": "71de7b4ad552eea50d0b974dd038b60873ce260338b9de988a1ee154cc5de251" }, "downloads": -1, "filename": "s3utils-0.5.1.tar.gz", "has_sig": false, "md5_digest": "bb8393712ca7e91e8c043f8ddf433625", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 118570, "upload_time": "2014-04-30T00:22:35", "url": "https://files.pythonhosted.org/packages/58/a8/e3ce9894f98fe8eb5cc7ea01bb6ed0ccf85b7b7ac4e44682d10cd100ca3c/s3utils-0.5.1.tar.gz" } ], "0.5.2": [ { "comment_text": "", "digests": { "md5": "a1cbc10982f60b1d0446a8d741f7f5fb", "sha256": "bef30f39cd75eefa63761dc5c1b67bdbd5a785c0bbe905cfb9348a38b3e511a1" }, "downloads": -1, "filename": "s3utils-0.5.2.tar.gz", "has_sig": false, "md5_digest": "a1cbc10982f60b1d0446a8d741f7f5fb", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 119096, "upload_time": "2014-05-05T22:08:58", "url": "https://files.pythonhosted.org/packages/5d/f6/0324c10769107568170b8f212869988b14fc68f148da52fc4ec90b72f2de/s3utils-0.5.2.tar.gz" } ], "0.5.3": [ { "comment_text": "", "digests": { "md5": "db0d205e6c96921d675bc1e5f85cb7a4", "sha256": "86c7f120cf30fcb6c8aafeb70324a221a82a3bb53cdb1df01d6ebd6e9d60c316" }, "downloads": -1, "filename": "s3utils-0.5.3.tar.gz", "has_sig": false, "md5_digest": "db0d205e6c96921d675bc1e5f85cb7a4", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 128461, "upload_time": "2015-01-14T20:14:42", "url": "https://files.pythonhosted.org/packages/3f/fc/8764bf983d625b562de51f2d7e5c92063376b1f5d13e4939621c069e294e/s3utils-0.5.3.tar.gz" } ], "0.6.0": [ { "comment_text": "", "digests": { "md5": "c2a152e79b212c74b0c4965fbc887dbf", "sha256": "ae8f071fbf6961b6032bc0feb3f6f5fe74feb02f86c12af5413ac05a9177a4af" }, "downloads": -1, "filename": "s3utils-0.6.0-py2-none-any.whl", "has_sig": false, "md5_digest": "c2a152e79b212c74b0c4965fbc887dbf", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 13667, "upload_time": "2016-04-02T06:48:40", "url": "https://files.pythonhosted.org/packages/6d/d5/5fe85bbe4392f93fd1d0eb777f8c116072c138496d2917cdd48b73eb390c/s3utils-0.6.0-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "dad76f3da7919947074ed890d03d3afd", "sha256": "87dfa2978370f6cf9e8ed505de356b99da8ff990d81b2dc2e18bbc127b87e6ec" }, "downloads": -1, "filename": "s3utils-0.6.0-py3-none-any.whl", "has_sig": false, "md5_digest": "dad76f3da7919947074ed890d03d3afd", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13666, "upload_time": "2016-04-02T06:48:48", "url": "https://files.pythonhosted.org/packages/94/c2/493c4c17ab8b8c1fd1054a9b489524ea61ba0e8b47e1f2a343d2629955bf/s3utils-0.6.0-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "35a3f762b16bacf4f53f101179d1fd16", "sha256": "54e4e47b345f830b294442f8265fde4992e1bd084a683baea15996119a74a1e0" }, "downloads": -1, "filename": "s3utils-0.6.0.tar.gz", "has_sig": false, "md5_digest": "35a3f762b16bacf4f53f101179d1fd16", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 214951, "upload_time": "2016-04-02T06:48:54", "url": "https://files.pythonhosted.org/packages/ff/72/499795935131ae15bf7748363354db1f68995ef3520be2f5caabb93000aa/s3utils-0.6.0.tar.gz" } ], "0.6.1": [ { "comment_text": "", "digests": { "md5": "69d941423f2d38d3a14fde4c8e8f000f", "sha256": "7dad13f42c024ddb02480705cdb2e30ea1f848d684f997d6cdc75414998505e6" }, "downloads": -1, "filename": "s3utils-0.6.1-py2-none-any.whl", "has_sig": false, "md5_digest": "69d941423f2d38d3a14fde4c8e8f000f", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 13917, "upload_time": "2016-04-04T17:37:44", "url": "https://files.pythonhosted.org/packages/36/2f/dcbd96e44d282d51fe8e2b6ec200a67f5b054f91f27bcac1c5a9f882753a/s3utils-0.6.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "081442386bf07915d9e3373a91318d06", "sha256": "d9311156887b5e47f5efb366fcd950fe3166edf2fc5ccde854295ae0bcd96628" }, "downloads": -1, "filename": "s3utils-0.6.1-py3-none-any.whl", "has_sig": false, "md5_digest": "081442386bf07915d9e3373a91318d06", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13919, "upload_time": "2016-04-04T17:37:50", "url": "https://files.pythonhosted.org/packages/68/42/d2b931983e8fb41f5078922868449eddb52e45df7eb61944ff42a6312d80/s3utils-0.6.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c83dc14a9a5486ad1f1a6f25b486779c", "sha256": "79c0ef4fe5b0b5dacb48d98bf3901bed94b482bc55cf2f58e70673d1805223f9" }, "downloads": -1, "filename": "s3utils-0.6.1.tar.gz", "has_sig": false, "md5_digest": "c83dc14a9a5486ad1f1a6f25b486779c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 215229, "upload_time": "2016-04-04T17:37:57", "url": "https://files.pythonhosted.org/packages/1f/ba/b1cddcc04ad110194ab502d3d3077a596fdd6c76e3dbf8df79254d87eeb1/s3utils-0.6.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "69d941423f2d38d3a14fde4c8e8f000f", "sha256": "7dad13f42c024ddb02480705cdb2e30ea1f848d684f997d6cdc75414998505e6" }, "downloads": -1, "filename": "s3utils-0.6.1-py2-none-any.whl", "has_sig": false, "md5_digest": "69d941423f2d38d3a14fde4c8e8f000f", "packagetype": "bdist_wheel", "python_version": "py2", "requires_python": null, "size": 13917, "upload_time": "2016-04-04T17:37:44", "url": "https://files.pythonhosted.org/packages/36/2f/dcbd96e44d282d51fe8e2b6ec200a67f5b054f91f27bcac1c5a9f882753a/s3utils-0.6.1-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "081442386bf07915d9e3373a91318d06", "sha256": "d9311156887b5e47f5efb366fcd950fe3166edf2fc5ccde854295ae0bcd96628" }, "downloads": -1, "filename": "s3utils-0.6.1-py3-none-any.whl", "has_sig": false, "md5_digest": "081442386bf07915d9e3373a91318d06", "packagetype": "bdist_wheel", "python_version": "py3", "requires_python": null, "size": 13919, "upload_time": "2016-04-04T17:37:50", "url": "https://files.pythonhosted.org/packages/68/42/d2b931983e8fb41f5078922868449eddb52e45df7eb61944ff42a6312d80/s3utils-0.6.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "c83dc14a9a5486ad1f1a6f25b486779c", "sha256": "79c0ef4fe5b0b5dacb48d98bf3901bed94b482bc55cf2f58e70673d1805223f9" }, "downloads": -1, "filename": "s3utils-0.6.1.tar.gz", "has_sig": false, "md5_digest": "c83dc14a9a5486ad1f1a6f25b486779c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 215229, "upload_time": "2016-04-04T17:37:57", "url": "https://files.pythonhosted.org/packages/1f/ba/b1cddcc04ad110194ab502d3d3077a596fdd6c76e3dbf8df79254d87eeb1/s3utils-0.6.1.tar.gz" } ] }