{
    "info": {
        "author": "Sanhe Hu",
        "author_email": "husanhe@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Operating System :: MacOS",
            "Operating System :: Microsoft :: Windows",
            "Operating System :: Unix",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6"
        ],
        "description": ".. image:: https://readthedocs.org/projects/s3iotools/badge/?version=latest\n    :target: https://s3iotools.readthedocs.io/index.html\n    :alt: Documentation Status\n\n.. image:: https://travis-ci.org/MacHu-GWU/s3iotools-project.svg?branch=master\n    :target: https://travis-ci.org/MacHu-GWU/s3iotools-project?branch=master\n\n.. image:: https://codecov.io/gh/MacHu-GWU/s3iotools-project/branch/master/graph/badge.svg\n  :target: https://codecov.io/gh/MacHu-GWU/s3iotools-project\n\n.. image:: https://img.shields.io/pypi/v/s3iotools.svg\n    :target: https://pypi.python.org/pypi/s3iotools\n\n.. image:: https://img.shields.io/pypi/l/s3iotools.svg\n    :target: https://pypi.python.org/pypi/s3iotools\n\n.. image:: https://img.shields.io/pypi/pyversions/s3iotools.svg\n    :target: https://pypi.python.org/pypi/s3iotools\n\n.. image:: https://img.shields.io/badge/STAR_Me_on_GitHub!--None.svg?style=social\n    :target: https://github.com/MacHu-GWU/s3iotools-project\n\n------\n\n\n.. image:: https://img.shields.io/badge/Link-Document-blue.svg\n      :target: https://s3iotools.readthedocs.io/index.html\n\n.. image:: https://img.shields.io/badge/Link-API-blue.svg\n      :target: https://s3iotools.readthedocs.io/py-modindex.html\n\n.. image:: https://img.shields.io/badge/Link-Source_Code-blue.svg\n      :target: https://s3iotools.readthedocs.io/py-modindex.html\n\n.. image:: https://img.shields.io/badge/Link-Install-blue.svg\n      :target: `install`_\n\n.. image:: https://img.shields.io/badge/Link-GitHub-blue.svg\n      :target: https://github.com/MacHu-GWU/s3iotools-project\n\n.. image:: https://img.shields.io/badge/Link-Submit_Issue-blue.svg\n      :target: https://github.com/MacHu-GWU/s3iotools-project/issues\n\n.. image:: https://img.shields.io/badge/Link-Request_Feature-blue.svg\n      :target: https://github.com/MacHu-GWU/s3iotools-project/issues\n\n.. image:: https://img.shields.io/badge/Link-Download-blue.svg\n      :target: https://pypi.org/pypi/s3iotools#files\n\n\nWelcome to ``s3iotools`` Documentation\n==============================================================================\n\n\nUsage\n------------------------------------------------------------------------------\n\nCopy local file to s3 and download file object from s3 to local is easy:\n\n.. code-block:: python\n\n    from s3iotools import S3FileObject\n\n    s3obj = S3FileObject(bucket=\"my-bucket\", key=\"hello.txt\", path=\"hello.txt\")\n\n    # get started, now we don't have file either on local or on s3\n    if s3obj.path_obj.exists():\n        s3obj.path_obj.remove()\n    assert s3obj.exists_on_local() is False\n    assert s3obj.exists_on_s3() is False\n\n    s3obj.path_obj.write_text(\"hello world\", encoding=\"utf-8)\n    assert s3obj.exists_on_local() is True\n\n    s3obj.copy_to_s3()\n    assert s3obj.exists_on_s3() is True\n\n    s3obj.path_obj.remove()\n    assert s3obj.exists_on_local() is False\n\n    s3obj.copy_to_local()\n    assert s3obj.exists_on_local() is True\n\n\nYou can manipulate s3 backed ``pandas.DataFrame`` easily:\n\n.. code-block:: python\n\n    import boto3\n    import pandas as pd\n    from s3iotools import S3Dataframe\n\n    session = boto3.Session(profile_name=\"xxx\")\n    s3 = session.resource(\"s3\")\n    bucket_name = \"my-bucket\"\n    s3df = S3Dataframe(s3_resource=s3, bucket_name=bucket_name)\n    s3df.df = pd.DataFrame(...)\n\n    s3df.to_csv(key=\"data.csv\")\n    s3df.to_csv(key=\"data.csv.gz\", gzip_compressed=True)\n\n    s3df_new = S3Dataframe(s3_resource=s3, bucket_name=bucket_name, key=\"data.csv\")\n    s3df_new.read_csv()\n    s3df_new.df # access data\n\n    s3df_new = S3Dataframe(s3_resource=s3, bucket_name=bucket_name, key=\"data.csv.gz\")\n    s3df_new.read_csv(gzip_compressed=True)\n    s3df_new.df # access data\n\n\njson IO is similar.\n\n.. code-block:: python\n\n    s3df = S3Dataframe(s3_resource=s3, bucket_name=bucket_name)\n    s3df.df = pd.DataFrame(...)\n    s3df.to_json(key=\"data.json.gz\", gzip_compressed=True)\n    s3df_new = S3Dataframe(s3_resource=s3, bucket_name=bucket_name, key=\"data.json.gz\")\n    s3df_new.read_json(gzip_compressed=True)\n    s3df_new.df # access data\n\n\nparquet is a columnar storage format, which is very efficient for OLAP query. You can just put data on S3, then use AWS Athena to query parquet files. parquet IO in s3iotools is easy:\n\n.. code-block:: python\n\n    s3df = S3Dataframe(s3_resource=s3, bucket_name=bucket_name)\n    s3df.df = pd.DataFrame(...)\n    s3df.to_parquet(key=\"data.parquet\", compression=\"gzip\")\n    s3df_new = S3Dataframe(s3_resource=s3, bucket_name=bucket_name, key=\"data.parquet\")\n    s3df_new.read_parquet()\n    s3df_new.df # access data\n\n\ns3iotools doesn't automatically install ``pyarrow``, you can install it with ``pip install pyarrow``.\n\n\n.. _install:\n\nInstall\n------------------------------------------------------------------------------\n\n``s3iotools`` is released on PyPI, so all you need is:\n\n.. code-block:: console\n\n    $ pip install s3iotools\n\nTo upgrade to latest version:\n\n.. code-block:: console\n\n    $ pip install --upgrade s3iotools\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "https://pypi.python.org/pypi/s3iotools/0.0.3#downloads",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/MacHu-GWU/",
        "keywords": "",
        "license": "MIT",
        "maintainer": "Unknown",
        "maintainer_email": "",
        "name": "s3iotools",
        "package_url": "https://pypi.org/project/s3iotools/",
        "platform": "Windows",
        "project_url": "https://pypi.org/project/s3iotools/",
        "project_urls": {
            "Download": "https://pypi.python.org/pypi/s3iotools/0.0.3#downloads",
            "Homepage": "https://github.com/MacHu-GWU/"
        },
        "release_url": "https://pypi.org/project/s3iotools/0.0.3/",
        "requires_dist": [
            "attrs",
            "attrs-mate",
            "pathlib-mate",
            "boto3",
            "sphinx (==1.8.1) ; extra == 'docs'",
            "sphinx-rtd-theme ; extra == 'docs'",
            "sphinx-jinja ; extra == 'docs'",
            "sphinx-copybutton ; extra == 'docs'",
            "docfly (>=0.0.17) ; extra == 'docs'",
            "rstobj (>=0.0.5) ; extra == 'docs'",
            "pygments ; extra == 'docs'",
            "pytest (==3.2.3) ; extra == 'tests'",
            "pytest-cov (==2.5.1) ; extra == 'tests'",
            "boto (==2.36.0) ; extra == 'tests'",
            "boto3 (==1.9.86) ; extra == 'tests'",
            "moto ; extra == 'tests'",
            "pandas ; extra == 'tests'",
            "pyarrow ; extra == 'tests'"
        ],
        "requires_python": "",
        "summary": "Make S3 file object read/write easier, support raw file, csv, parquet, pandas.DataFrame.",
        "version": "0.0.3"
    },
    "last_serial": 5292200,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "68d6f8ab16ed5123db153fe83797c0e2",
                    "sha256": "0af60c09d406fbe9dd881226ddc4323939ec3ab64d9b1dd863ea3f0105a35894"
                },
                "downloads": -1,
                "filename": "s3iotools-0.0.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "68d6f8ab16ed5123db153fe83797c0e2",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 38194,
                "upload_time": "2019-02-27T23:41:06",
                "url": "https://files.pythonhosted.org/packages/d0/c5/b4bc25e48e7c73deeaff7f48e114b8c5a31605c56722bba013e2f0fe3d6e/s3iotools-0.0.1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "eb7130da64736d351d161649e6e5fe83",
                    "sha256": "ce75afdd00819df040efe6b3c9be9c5595f337512bcdf0870abb5b0909a521d1"
                },
                "downloads": -1,
                "filename": "s3iotools-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "eb7130da64736d351d161649e6e5fe83",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27589,
                "upload_time": "2019-02-27T23:41:08",
                "url": "https://files.pythonhosted.org/packages/46/ca/0de3d18f166e94300ac0305f5361fc3f30e7749808a894184e3ff0f66e8c/s3iotools-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "78cb6b05cef4dd48b2716907582a9aea",
                    "sha256": "ce0bb74c73b60cabe915689bd5d54542042310547aaa8f1e91aff21d3cca3c31"
                },
                "downloads": -1,
                "filename": "s3iotools-0.0.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "78cb6b05cef4dd48b2716907582a9aea",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 53959,
                "upload_time": "2019-03-15T17:23:56",
                "url": "https://files.pythonhosted.org/packages/27/32/ea8d190524052c15b3021b2f6f9efd082d59289e4b0a164412c5f209ac5c/s3iotools-0.0.2-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "6669da6dad29744ba50acee224b7144b",
                    "sha256": "17e63bb606062e22f17aef30dfc34a08aa7a267eb35fbc96c7ca7df117412cf9"
                },
                "downloads": -1,
                "filename": "s3iotools-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "6669da6dad29744ba50acee224b7144b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 32478,
                "upload_time": "2019-03-15T17:23:58",
                "url": "https://files.pythonhosted.org/packages/da/ad/a2cc6d3040f2812170b3472d9e2f4557d8ea5ebcf098e2e933c94dc134d6/s3iotools-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4f1d86fb3ed351e3ded2adad133e2ad8",
                    "sha256": "35f606a3284835dc1f13353d582d287be6a1fe6f322dc869cd074a0d4236e62d"
                },
                "downloads": -1,
                "filename": "s3iotools-0.0.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "4f1d86fb3ed351e3ded2adad133e2ad8",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 34544,
                "upload_time": "2019-05-20T13:00:21",
                "url": "https://files.pythonhosted.org/packages/e2/47/56a8722cfb95009ff6b33cf10eb064005bedf898c49a32edd983ea1ebdb9/s3iotools-0.0.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "b61c9dfebc1d4716f5e1dc775023a2aa",
                    "sha256": "689ba157b39fadc44e1884344474a57662d80e005238274d184f10b56af6019b"
                },
                "downloads": -1,
                "filename": "s3iotools-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "b61c9dfebc1d4716f5e1dc775023a2aa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27035,
                "upload_time": "2019-05-20T13:00:24",
                "url": "https://files.pythonhosted.org/packages/f4/14/f954cae8a80ce3708e53ffce76af826f2651aba55ef3c48e7947055152a6/s3iotools-0.0.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "4f1d86fb3ed351e3ded2adad133e2ad8",
                "sha256": "35f606a3284835dc1f13353d582d287be6a1fe6f322dc869cd074a0d4236e62d"
            },
            "downloads": -1,
            "filename": "s3iotools-0.0.3-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "4f1d86fb3ed351e3ded2adad133e2ad8",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 34544,
            "upload_time": "2019-05-20T13:00:21",
            "url": "https://files.pythonhosted.org/packages/e2/47/56a8722cfb95009ff6b33cf10eb064005bedf898c49a32edd983ea1ebdb9/s3iotools-0.0.3-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "b61c9dfebc1d4716f5e1dc775023a2aa",
                "sha256": "689ba157b39fadc44e1884344474a57662d80e005238274d184f10b56af6019b"
            },
            "downloads": -1,
            "filename": "s3iotools-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "b61c9dfebc1d4716f5e1dc775023a2aa",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 27035,
            "upload_time": "2019-05-20T13:00:24",
            "url": "https://files.pythonhosted.org/packages/f4/14/f954cae8a80ce3708e53ffce76af826f2651aba55ef3c48e7947055152a6/s3iotools-0.0.3.tar.gz"
        }
    ]
}