{
    "info": {
        "author": "Tetsuya Hirata",
        "author_email": "tetsuya.hirata@classi.jp",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Programming Language :: Python :: 3.7",
            "Topic :: Utilities"
        ],
        "description": "This software is released under the MIT License, see LICENSE.txt.\n\n\n`gcp-accessor` is a wrapper of [Google Cloud Storage API](https://cloud.google.com/storage/) and [Google BigQuery API](https://cloud.google.com/bigquery/what-is-bigquery) for more simply accesseing both functions.  \n\nWhile there are [google-cloud-storage](https://github.com/googleapis/google-cloud-python/tree/master/storage/) and [google-cloud-bigquery](https://github.com/googleapis/google-cloud-python) library provided by Google which have several sophisticated features, frequently used features can be limited in python application development.  \n\n`gcp-accessor` focuses on the simplicity of usage pattern and includes the selected features as described below.  \n\n***\n- bq_accessor\n  - get_dataset\n  - get_table_name\n  - check_if_dataset_exists\n  - check_if_table_exists\n  - create_table_from_json\n  - load_data_from_gcs\n  - execute_query\n- gcs_accessor\n  - get_blob\n  - get_blob_list\n  - get_uris_list\n  - upload_csv_gzip\n  - download_csv_gzip\n\n***\nSetup\n-\n#### Installation\n```bash\npip install gcp-accessor\n```\n\n#### Set GOOGLE_APPLICATION_CREDENTIALS\n```bash\nexport GOOGLE_APPLICATION_CREDENTIALS='full path to credential key json file'\n```\n\nUsage of Google Cloud Storage Accessor\n-\n#### Import gcp_accessor and create google cloud storage client\n```python\nimport gcp_accessor\ngcs = gcp_accessor.GoogleCloudStorageAccessor()\n```\n\n#### Get Binary Large Object (blob) from google cloud storage (gcs)\n```python\ngcs.get_blob('bucket_name', 'full path to file on gcs')\n```\n\n#### Get Binary Large Object (blob) lists as HTTPIterator object from gcs\n```python\ngcs.get_blob_list('bucket_name', prefix=None, delimiter=None)\n```\n\n#### Get uris lists\n```python\ngcs.get_uris_list('bucket_name', prefix=None, delimiter=None)\n```\n\n#### Upload gzip object on gcs\n```python\ngcs.upload_csv_gzip('bucket_name', 'full path to file on gcs', 'texts')\n```\n\n#### Download gzip file from gcs and load it through in-memory\n```python\ngcs.download_csv_gzip('bucket_name', 'full path to file on gcs')\n```\n\nUsage of Big Query Accessor\n-\n#### Import gcp_accessor and create big query client\n```python\nimport gcp_accessor\nbq = gcp_accessor.BigQueryAccessor()\n```\n#### Get datasets if datasets do not exist, then return empty list\n\n```python\nbq.get_dataset()\n```\n\n#### Get table names if table names do not exist, then return exception error message\n```python\nbq.get_table_name('dataset_name')\n```\n#### Check if dataset exsists in bigquery and then return True or False\n```python\nbq.check_if_dataset_exists('dataset_name')\n```\n\n#### Check if table exsists in bigquery and then return True or False\n```python\nbq.check_if_table_exists('dataset_name', 'table_name')\n```\n\n\n#### Create table on bigquery based on schema json file\n```python\nbq.create_table_from_json('path_schema_file', 'dataset', 'table_name')\n```\n\n#### Load data from gcs (You have to upload file on gcs.).\n```python\nbq.load_data_from_gcs(\n        'dataset_name',\n        'uris',\n        'table_name',\n        location=\"US\",\n        skip_leading_rows=0,\n        source_format=\"CSV\",\n        create_disposition=\"CREATE_NEVER\",\n        write_disposition=\"WRITE_EMPTY\",\n    )\n```\n\n#### Execute a simple query or query with the below optipons\n```python\nbq.execute_query(\n        'query',\n        location=\"US\",\n        timeout=30,\n        page_size=0,\n        project=None,\n        allow_large_results=False,\n        destination=None,\n        destination_encryption_configuration=None,\n        dry_run=False,\n        labels=None,\n        priority=None,\n        query_parameters=None,\n        schema_update_options=None,\n        table_definitions=None,\n        time_partitioning=None,\n        udf_resources=None,\n        use_legacy_sql=False,\n        use_query_cache=False,\n        write_disposition=None\n    )\n```\n\n\n\nNote\n-\nSome argument names and descriptions about each argument are cited and referred from the documents of ['Google Cloud Client Libraries for Python'](https://googleapis.github.io/google-cloud-python/latest/index.html) The explanations about each argument are written in the code.\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/tetsuya0617/gcputils",
        "keywords": "",
        "license": "MIT",
        "maintainer": "Tetsuya Hirata",
        "maintainer_email": "tetsuya.hirata@classi.jp",
        "name": "gcp-accessor",
        "package_url": "https://pypi.org/project/gcp-accessor/",
        "platform": "",
        "project_url": "https://pypi.org/project/gcp-accessor/",
        "project_urls": {
            "Homepage": "https://github.com/tetsuya0617/gcputils"
        },
        "release_url": "https://pypi.org/project/gcp-accessor/0.0.3/",
        "requires_dist": [
            "google-cloud-storage",
            "google-cloud-bigquery"
        ],
        "requires_python": "",
        "summary": "Accessor to Google Cloud Storage and Big Query",
        "version": "0.0.3"
    },
    "last_serial": 5958384,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "2f85420c472e7ee139d247d303ea1437",
                    "sha256": "36f91eeccf504baf732efd0e2df05759bc8b44f1e0d26a76614b3f3035426d48"
                },
                "downloads": -1,
                "filename": "gcp_accessor-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "2f85420c472e7ee139d247d303ea1437",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 8771,
                "upload_time": "2019-09-12T09:40:59",
                "url": "https://files.pythonhosted.org/packages/43/16/9f2b0152fe4bf6527a7f3b84eb18f266e492b4f13d987be7352d43268101/gcp_accessor-0.0.1-py3-none-any.whl"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "6980665f24b151879246cf701be3c150",
                    "sha256": "017d31d83da2dda76b79262fd6a90514f206b7d3612fb8f3fba134f6c0b494e5"
                },
                "downloads": -1,
                "filename": "gcp_accessor-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "6980665f24b151879246cf701be3c150",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 8771,
                "upload_time": "2019-10-11T03:18:40",
                "url": "https://files.pythonhosted.org/packages/fa/e6/91ed6f23216a8389a3e5265e0a5a95012fa905d61a7ee4fb79a80d5e3d38/gcp_accessor-0.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "f1abfd6c6e82eb104fbcdb69798caf09",
                    "sha256": "e5cb3c8d136785a8ec3253bb820fc1ac7b8a783ffa1826a95c4d92499f9e74e1"
                },
                "downloads": -1,
                "filename": "gcp-accessor-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "f1abfd6c6e82eb104fbcdb69798caf09",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6683,
                "upload_time": "2019-10-11T03:18:44",
                "url": "https://files.pythonhosted.org/packages/f7/b6/bc9542c7c93eaeda7d99b21bae2e1a838c06af3de2d61a5e793c0b39cb87/gcp-accessor-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "05125ce2b72c489e2c4dddc4203ae467",
                    "sha256": "2f505de0a927d5d1426309de4dbbf36a03bb4d297831bb92d69dd30c1d162d4c"
                },
                "downloads": -1,
                "filename": "gcp_accessor-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "05125ce2b72c489e2c4dddc4203ae467",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 8777,
                "upload_time": "2019-10-11T04:53:53",
                "url": "https://files.pythonhosted.org/packages/00/fe/e86e9468a0a495e214f605b5791a951cd8be068128a7353e7e7f6ed70034/gcp_accessor-0.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3eff1449a70351e8d83bab41925f615f",
                    "sha256": "8e86701874b4b658d8a119b6d5b01f138c6cc95865611f4fb8731d9c1914d6bc"
                },
                "downloads": -1,
                "filename": "gcp-accessor-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "3eff1449a70351e8d83bab41925f615f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 6686,
                "upload_time": "2019-10-11T04:53:55",
                "url": "https://files.pythonhosted.org/packages/69/29/2b3ec8e2889e7e2e513722b1b672e2b0f39ab5bd7ddd5d95b2ec8dab74be/gcp-accessor-0.0.3.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "05125ce2b72c489e2c4dddc4203ae467",
                "sha256": "2f505de0a927d5d1426309de4dbbf36a03bb4d297831bb92d69dd30c1d162d4c"
            },
            "downloads": -1,
            "filename": "gcp_accessor-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "05125ce2b72c489e2c4dddc4203ae467",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 8777,
            "upload_time": "2019-10-11T04:53:53",
            "url": "https://files.pythonhosted.org/packages/00/fe/e86e9468a0a495e214f605b5791a951cd8be068128a7353e7e7f6ed70034/gcp_accessor-0.0.3-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "3eff1449a70351e8d83bab41925f615f",
                "sha256": "8e86701874b4b658d8a119b6d5b01f138c6cc95865611f4fb8731d9c1914d6bc"
            },
            "downloads": -1,
            "filename": "gcp-accessor-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "3eff1449a70351e8d83bab41925f615f",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 6686,
            "upload_time": "2019-10-11T04:53:55",
            "url": "https://files.pythonhosted.org/packages/69/29/2b3ec8e2889e7e2e513722b1b672e2b0f39ab5bd7ddd5d95b2ec8dab74be/gcp-accessor-0.0.3.tar.gz"
        }
    ]
}