{
    "info": {
        "author": "Rob Howley",
        "author_email": "howley.robert@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 3"
        ],
        "description": "[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n# s3-streaming: handling (big) S3 files like regular files\nStoring, retrieving and using files in S3 is a regular activity so it should be easy. It should also ...\n* stream the data\n* have an api that is python file-io like\n* handle some of the desearization and compression stuff because why not\n\n## Install\n\n```bash\npip install s3-streaming\n```\n\n## Streaming S3 objects like regular files\n\n### The basics\nOpening and reading S3 objects is similar to regular python io. The only difference is that you need to provide a \n`boto3.session.Session` instance to handle the bucket access. \n\n```python\nimport boto3\nfrom s3streaming import s3_open\n\n\nwith s3_open('s3://bucket/key', boto_session=boto3.session.Session()) as f:\n    for next_line in f:\n        print(next_line)\n```\n\n### Injecting deserialization and compression handling in stream\nConsider a file that is `gzip` compressed and contains lines of `json`. There's some boilerplate in dealing with that,\nbut why bother? Just handle that in stream.\n\n```python\nfrom s3streaming import s3_open, deserialize, compression\n\n\nreader_settings = dict(\n  boto_session=boto3.session.Session(),\n  deserializer=deserialize.json_lines, \n  compression=compression.gzip\n)\n\nwith s3_open('s3://bucket/key.gzip', **reader_settings) as f:\n    for next_line in f:\n        print(next_line.keys())    # because the file was decompressed ...\n        print(next_line.values())  #   ... and the json is now a loaded dict!\n\n```\n\nOther `deserialize` options include \n* `csv`\n* `csv_as_dict`\n* `tsv`\n* `tsv_as_dict`\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/robhowley/s3-streaming",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "s3-streaming",
        "package_url": "https://pypi.org/project/s3-streaming/",
        "platform": "",
        "project_url": "https://pypi.org/project/s3-streaming/",
        "project_urls": {
            "Homepage": "https://github.com/robhowley/s3-streaming"
        },
        "release_url": "https://pypi.org/project/s3-streaming/0.0.1/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "stream and (de)serialize s3 objects with no local footprint",
        "version": "0.0.1"
    },
    "last_serial": 4975045,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f51e615626fce9a558e114fc5d2fa487",
                    "sha256": "ea7b8a4f6fc8f756b1c76452c6cf86e496fb8e9ab879b0379b0e1f22c04ea5e8"
                },
                "downloads": -1,
                "filename": "s3_streaming-0.0.1-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "f51e615626fce9a558e114fc5d2fa487",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 3890,
                "upload_time": "2019-03-23T04:14:29",
                "url": "https://files.pythonhosted.org/packages/a9/8b/a0d22ddb20f9c765ec53378ce36fdb4af9dbd985883fa2da53ed1fc33d63/s3_streaming-0.0.1-py2-none-any.whl"
            }
        ],
        "0.0.1a0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "249700c1ded558bf6ffeaf7a10660fc3",
                    "sha256": "bb2da5c8f6afd938a1d2c6020d320413976b217aba0294ad99a02a207dba8f26"
                },
                "downloads": -1,
                "filename": "s3_streaming-0.0.1a0-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "249700c1ded558bf6ffeaf7a10660fc3",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 3911,
                "upload_time": "2019-03-23T04:14:31",
                "url": "https://files.pythonhosted.org/packages/0c/38/4cc6b8b0681d12ba74d904a11acc6ded6800ac7ae9ca46955f6769657d2e/s3_streaming-0.0.1a0-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "470963d85ae67ab7d6da33293a6b4774",
                    "sha256": "7fdfaaa654e275d65d034bd17aa8a958a84d84fcbc8ff21abc70a59253d3f780"
                },
                "downloads": -1,
                "filename": "s3-streaming-0.0.1a0.tar.gz",
                "has_sig": false,
                "md5_digest": "470963d85ae67ab7d6da33293a6b4774",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 3160,
                "upload_time": "2019-03-23T04:14:32",
                "url": "https://files.pythonhosted.org/packages/19/56/ac7da839610d08b17ca08f5f81b56c1ed6543def343c5dc6bcbb06d57e5c/s3-streaming-0.0.1a0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "f51e615626fce9a558e114fc5d2fa487",
                "sha256": "ea7b8a4f6fc8f756b1c76452c6cf86e496fb8e9ab879b0379b0e1f22c04ea5e8"
            },
            "downloads": -1,
            "filename": "s3_streaming-0.0.1-py2-none-any.whl",
            "has_sig": false,
            "md5_digest": "f51e615626fce9a558e114fc5d2fa487",
            "packagetype": "bdist_wheel",
            "python_version": "py2",
            "requires_python": null,
            "size": 3890,
            "upload_time": "2019-03-23T04:14:29",
            "url": "https://files.pythonhosted.org/packages/a9/8b/a0d22ddb20f9c765ec53378ce36fdb4af9dbd985883fa2da53ed1fc33d63/s3_streaming-0.0.1-py2-none-any.whl"
        }
    ]
}