{
    "info": {
        "author": "Jacob Alheid",
        "author_email": "jake@about.me",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "s3peat\n======\n\ns3peat is a Python module to help upload directories to S3 using parallel\nthreads.\n\nThe source is hosted at `<http://github.com/shakefu/s3peat/>`_.\n\n.. image:: http://shakefu.s3.amazonaws.com/s3peat/s3peat.jpg\n\n\nInstalling\n----------\n\ns3peat can be installed from PyPI to get the latest release. If you'd like\ndevelopment code, you can check out the git repo.\n\n.. code-block:: bash\n\n   # Install from PyPI\n   $ pip install s3peat\n\n   # Install from GitHub\n   $ git clone http://github.com/shakefu/s3peat.git\n   $ cd s3peat\n   $ python setup.py install\n\nCommand line usage\n------------------\n\nWhen installed via ``pip`` or ``python setup.py install``, a command called\n``s3peat`` will be added. This command can be used to upload files easily.\n\n.. code-block:: text\n\n   $ s3peat --help\n   usage: s3peat [--prefix] --bucket [--key] [--secret] [--concurrency]\n         [--exclude] [--include] [--dry-run] [--verbose] [--version]\n         [--help] directory\n\n   positional arguments:\n     directory            directory to be uploaded\n\n   optional arguments:\n     --prefix , -p        s3 key prefix\n     --bucket , -b        s3 bucket name\n     --key , -k           AWS key id\n     --secret , -s        AWS secret\n     --concurrency , -c   number of threads to use\n     --exclude , -e       exclusion regex\n     --include , -i       inclusion regex\n     --private, -r        do not set ACL public\n     --dry-run, -d        print files matched and exit, do not upload\n     --verbose, -v        increase verbosity (-vvv means more verbose)\n     --version            show program's version number and exit\n     --help               display this help and exit\n\n**Example**:\n\n.. code-block:: bash\n\n   $ s3peat -b my/bucket -p my/s3/key/prefix -k KEY -s SECRET my-dir/\n\nConfiguring\n\"\"\"\"\"\"\"\"\"\"\"\n\nThis library is based around `boto <http://docs.pythonboto.org/>`_. Your *AWS\nAccess Key Id* and *AWS Secret Access Key* do not have to be passed on the\ncommand line - they may be configured using any method that boto supports,\nincluding environment variables and the ``~/.boto`` config..\n\n**Example using environment variables**:\n\n.. code-block:: bash\n\n   $ export AWS_ACCESS_KEY_ID=ABCDEFabcdef01234567\n   $ export AWS_SECRET_ACCESS_KEY=ABCDEFabcdef0123456789ABCDEFabcdef012345\n   $ s3peat -b my/bucket -p s3/prefix -c 25 some_dir/\n\n**Example ``~/.boto`` config**:\n\n.. code-block:: config\n\n   # File: ~/.boto\n   [Credentials]\n   aws_access_key_id = ABCDEFabcdef01234567\n   aws_secret_access_key = ABCDEFabcdef0123456789ABCDEFabcdef012345\n\n\nIncluding and excluding files\n\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\n\nUsing the ``--include`` and ``--exclude`` (``-i`` or ``-e``) parameters, you\ncan specify regex patterns to include or exclude from the list of files to be\nuploaded.\n\nThese regexes are Python regexes, as applied by ``re.search()``, so if you want\nto match the beginning or end of a filename (including the directory), make\nsure to use the ``^`` or ``$`` metacharacters.\n\nThese parameters can be specified multiple times, for example:\n\n.. code-block:: bash\n\n   # Upload all .txt and .py files, excluding the test directory\n   $ s3peat -b my-bucket -i '.txt$' -i '.py$' -e '^test/' .\n\nDoing a Dry-run\n\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\n\nIf you're unsure what exactly is in the directory to be uploaded, you can do a\ndry run with the ``--dry-run`` or ``-d`` option.\n\nBy default, dry runs only output the number of files found and an error message\nif it cannot connect to the specified S3 bucket. As you increase verbosity,\nmore information will be output. See below for examples.\n\n.. code-block:: bash\n\n   $ s3peat -b my-bucket . -e '\\.git' --dry-run\n   21 files found.\n\n   $ s3peat -b foo . -e '\\.git' --dry-run\n   21 files found.\n   Error connecting to S3 bucket 'foo'.\n\n   $ s3peat -b my-bucket . -e '\\.git' --dry-run -v\n   21 files found.\n   Connected to S3 bucket 'my-bucket' OK.\n\n   $ s3peat -b foo . -e '\\.git' --dry-run -v\n   21 files found.\n   Error connecting to S3 bucket 'foo'.\n       S3ResponseError: 403 Forbidden\n\n   $ s3peat -b my-bucket . -i 'rst$|py$|LICENSE' --dry-run\n   5 files found.\n\n   $ s3peat -b my-bucket . -i 'rst$|py$|LICENSE' --dry-run -vv\n   Finding files in /home/s3peat/github.com/s3peat ...\n\n   ./LICENSE\n   ./README.rst\n   ./setup.py\n   ./s3peat/__init__.py\n   ./s3peat/scripts.py\n\n   5 files found.\n\n   Connected to S3 bucket 'my-bucket' OK.\n\nConcurrency\n\"\"\"\"\"\"\"\"\"\"\"\n\ns3peat is designed to upload to S3 with high concurrency. The only limits are\nthe speed of your uplink and the GIL. Python is limited in the number of\nthreads that will run concurrently on a single core.\n\nTypically, it seems that more than 50 threads do not add anything to the upload\nspeed, but your experiences may differ based on your network and CPU speeds.\n\nIf you want to try to tune your concurrency for your platfrom, I suggest using\nthe ``time`` command.\n\n**Example**:\n\n.. code-block:: bash\n\n   $ time s3peat -b my-bucket -p my/key/ --concurrency 50 my-dir/\n   271/271 files uploaded                                                                                                                                                                                                                           \n\n   real\t0m2.909s\n   user\t0m0.488s\n   sys\t0m0.114s\n\nPython API\n----------\n\nThe Python API has inline documentation, which should be good. If there's\nquestions, you can open a github issue. Here's an example anyway.\n\n**Example**:\n\n.. code-block:: python\n\n    from s3peat import S3Bucket, sync_to_s3\n\n    # Create a S3Bucket instance, which is used to create connections to S3\n    bucket = S3Bucket('my-bucket', AWS_KEY, AWS_SECRET)\n\n    # Call the sync_to_s3 method\n    failures = sync_to_s3(directory='my/directory', prefix='my/key',\n        bucket=bucket, concurrency=50)\n\n    # A list of filenames will be returned if there were failures in uploading\n    if not failures:\n        print \"No failures\"\n    else:\n        print \"Failed:\", failures\n\n\nChangelog\n---------\n\n1.0.0\n-----\n\n* Drops support for Python 2.x (use version `<1` if you need to continue to use\n  Python 2.)\n* Adds support for Python 3.x (tested against 3.4, should work for any later\n  version as well).\n* Thanks go to `aboutaaron <https://github.com/aboutaaron>`_ for the Python 3\n  support.\n\n0.5.1\n-----\n\n* Use posixpath.sep for upload keys. Thanks to `kevinschaul\n  <https://github.com/kevinschaul>`_.\n\n*Released February 4th, 2015*.\n\n0.5.0\n-----\n\n* Make attaching signal handlers optional. Thanks to `kevinschaul\n  <https://github.com/kevinschaul>`_.\n\n*Released December 1st, 2014*.\n\n0.4.7\n-----\n\n* Better support for Windows. Thanks to `kevinschaul\n  <https://github.com/kevinschaul>`_.\n\n*Released November 20th, 2014*.\n\nContributors\n------------\n\n* `shakefu <http://github.com/shakefu>`_ - Creator, maintainer\n* `kevinschaul <https://github.com/kevinschaul>`_\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "http://github.com/shakefu/s3peat",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "s3peat",
        "package_url": "https://pypi.org/project/s3peat/",
        "platform": "",
        "project_url": "https://pypi.org/project/s3peat/",
        "project_urls": {
            "Homepage": "http://github.com/shakefu/s3peat"
        },
        "release_url": "https://pypi.org/project/s3peat/1.0.0/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "Fast uploader to S3",
        "version": "1.0.0"
    },
    "last_serial": 3694003,
    "releases": {
        "0.4.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "03d94fa9f533bf061d4736d9f15b0ad8",
                    "sha256": "cc1de97a4892b8f0f7dffbf068156be622139250d9a7dd0bf0003bf2c0c0b7b0"
                },
                "downloads": -1,
                "filename": "s3peat-0.4.6.tar.gz",
                "has_sig": false,
                "md5_digest": "03d94fa9f533bf061d4736d9f15b0ad8",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9430,
                "upload_time": "2014-01-22T22:45:55",
                "url": "https://files.pythonhosted.org/packages/6f/30/1579d4104ceda9ac4222d5e5a99224d8a71c00e4966d6a5bd1b18d6510a9/s3peat-0.4.6.tar.gz"
            }
        ],
        "0.4.7": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c06eadfe7d481fddb2db2e4925112a23",
                    "sha256": "650eea67044d865b8a18795ec7f24c0a74701388342d78204b267b79bf136b83"
                },
                "downloads": -1,
                "filename": "s3peat-0.4.7.tar.gz",
                "has_sig": false,
                "md5_digest": "c06eadfe7d481fddb2db2e4925112a23",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9652,
                "upload_time": "2014-11-20T18:21:51",
                "url": "https://files.pythonhosted.org/packages/b1/df/5fe051b7ba5d3f7054aaf97231317ebe7c560b14706b8b426679df94a28d/s3peat-0.4.7.tar.gz"
            }
        ],
        "0.5.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6f3a98e7ade593f301bad44d84e2a6be",
                    "sha256": "7077fa1b5538a79c54ff4d3c1a386405293a51bfeaf7317bd3392f5b664d38bb"
                },
                "downloads": -1,
                "filename": "s3peat-0.5.0.tar.gz",
                "has_sig": false,
                "md5_digest": "6f3a98e7ade593f301bad44d84e2a6be",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9745,
                "upload_time": "2014-12-01T08:30:03",
                "url": "https://files.pythonhosted.org/packages/58/e9/b0ba02f702c1f76223a6216778415238112c581557a9bf66cc0f10071a17/s3peat-0.5.0.tar.gz"
            }
        ],
        "0.5.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c72391b99225143d3409da7550790843",
                    "sha256": "c59814eb8498497a20cf0820608c1a6b5c4c2772e4d3f25568e229ef448e11cf"
                },
                "downloads": -1,
                "filename": "s3peat-0.5.1.tar.gz",
                "has_sig": false,
                "md5_digest": "c72391b99225143d3409da7550790843",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9814,
                "upload_time": "2015-02-04T22:26:23",
                "url": "https://files.pythonhosted.org/packages/69/62/63e1327a54d027d3285b944c8f213c918ff37814e9d86a1f0aab1528358e/s3peat-0.5.1.tar.gz"
            }
        ],
        "1.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "723d3dbba7ab3acb6f371d42c94ed4f3",
                    "sha256": "b743f27a545853b9430554cdd31fdcf3642128efb92623f4688ecdf32395c974"
                },
                "downloads": -1,
                "filename": "s3peat-1.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "723d3dbba7ab3acb6f371d42c94ed4f3",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 10234,
                "upload_time": "2018-03-22T02:16:53",
                "url": "https://files.pythonhosted.org/packages/19/ef/65a505408d426cfd923bf24ef65d3d9ea64c89271a1a3aa31ab5e11ec7a0/s3peat-1.0.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "723d3dbba7ab3acb6f371d42c94ed4f3",
                "sha256": "b743f27a545853b9430554cdd31fdcf3642128efb92623f4688ecdf32395c974"
            },
            "downloads": -1,
            "filename": "s3peat-1.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "723d3dbba7ab3acb6f371d42c94ed4f3",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 10234,
            "upload_time": "2018-03-22T02:16:53",
            "url": "https://files.pythonhosted.org/packages/19/ef/65a505408d426cfd923bf24ef65d3d9ea64c89271a1a3aa31ab5e11ec7a0/s3peat-1.0.0.tar.gz"
        }
    ]
}