{
    "info": {
        "author": "Manojkumar vadivel",
        "author_email": "manojkumar.vsj@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: 3.5",
            "Programming Language :: Python :: 3.6",
            "Topic :: Software Development :: Build Tools"
        ],
        "description": "# GCP Utility tool\n\nWe can backfill any kind of bigquery sql to specific destination table with powerful python jinja templating feature. By default it can support date templates in your query. It will automatically convert to date template to date value in your query while executing.\n\nFollowing are the available date templates you can use it in your query.\n\n```bash\n{{ds}} - YYYY-DD-MM date string \n{{ds_nodash}} - YYYYDDMM date string \n{{ds_yesterday}} - YYYY-DD-MM yesterday date string \n{{ds_yesterday_nodash}} - YYYYDDMM yesterday date string \n{{ds_tomorrow}} - YYYY-DD-MM tomorrow date string \n{{ds_tomorrow_nodash}} - YYYYDDMM tomorrow date string \n\n``` \n\n## Getting Started\n\n\n### Prerequisites\n\nWhat things you need to install the software and how to install them\n\n```\npython 2.7\ngoogle-cloud==0.32.0\nJinja2==2.10\n```\n\n### Installing\n\nUse virtualenv to Install the dependancy packages \n\n```\ngit clone https://gitlab.com/manojnuk50/GCPUtility\ncd GCPUtility\nvirtualenv venv\nactivate venv/bin/activate\npip install -e .\n```\n\n-h command will return the help content which contains all the available  options\n```\nbq_client -h\n```\n\nEnd with an example of getting some data out of the system or using it for a little demo\n\n## Available option\n\n```bash\n-h, --help            show this help message and exit\n  -v, --version         show program's version number and exit\n  -q QUERY, --query QUERY\n                        provide valid bigquery sql\n  -f, --no-flattern     Flattern results\n  -p PROJECT, --project_id PROJECT\n                        provide valid project id\n  -ssql SSQL, --standard-sql SSQL\n                        Mention if using Standard sql\n  -d DESTINATION_TABLE, --destination-table DESTINATION_TABLE\n                        <projectname>:<datasetid>.<tableid> provide valid\n                        destination project-id\n  -w WRITE_DESPOSITION, --write-desposition WRITE_DESPOSITION\n                        Write disposition value\n  -qf QUERY_FILE, --query-file QUERY_FILE\n                        provide bigquery sql filepath\n  -t TEMPLATE, --template TEMPLATE\n                        provide template values\n  -tf TEMPLATE_FILE, --template-file TEMPLATE_FILE\n                        provide template file path\n  -s START_DATE, --start-date START_DATE\n                        Provide valid startdate (YYYY-MM-DD)\n  -e END_DATE, --end-date END_DATE\n                        Provide valid end date (YYYY-MM-DD)\n  -sf SERVICE_ACCOUNT_PATH, --service-account-file-path SERVICE_ACCOUNT_PATH\n                        provide valid path of service account json file\n```\n\n\n### passing raw query\n\nyou can pass the raw query to script that need to be execute\n\n```\nbq_client -q \"select * from [sample_proj:datasetid.tableid]\" -p destination_projectid -d dest_datasetid.tableid\n```\n\n### Using sql file\n\nif your query is quite long, you can save it to text file and pass the path of it\n\n```\nbq_client --query-file \"/path/sample.sql\" -p destination_projectid -d dest_datasetid.tableid```\n\n```\n\n### Passing date parameter\n\nIn default it will substitute current date into template fields. but if you want to change the date value you can pass manually using start date parameter\n\n```\nbq_client --query-file sample.sql -p projectid -d dest_datasetid.tableid -s 2018-02-02\n\n```\n\n### Backfill\n\nPass start-date and end-date to script, it will backfill the data.\n\n```\nbq_client --query-file sample.sql -p projectid -d dest_datasetid.tableid -s 2018-02-02 -e 2018-02-10\n\n```\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/manojvsj/gcp-utility.git",
        "keywords": "bigquery backfill,gcp",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "gcp-utility",
        "package_url": "https://pypi.org/project/gcp-utility/",
        "platform": "",
        "project_url": "https://pypi.org/project/gcp-utility/",
        "project_urls": {
            "Bug Reports": "https://github.com/manojvsj/gcp-utility/issues",
            "Funding": "https://donate.pypi.org",
            "Homepage": "https://github.com/manojvsj/gcp-utility.git",
            "Say Thanks!": "http://saythanks.io/to/example",
            "Source": "https://github.com/manojvsj/gcp-utility.git"
        },
        "release_url": "https://pypi.org/project/gcp-utility/1.1.3/",
        "requires_dist": [
            "google-api-core (==1.5.2)",
            "google-cloud-bigquery (==1.7.0)",
            "Jinja2 (==2.10)",
            "check-manifest ; extra == 'dev'",
            "coverage ; extra == 'test'"
        ],
        "requires_python": "",
        "summary": "GCP utility will give additional custom features over gcs",
        "version": "1.1.3"
    },
    "last_serial": 4615542,
    "releases": {
        "1.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0112f6e31171df6349bf0776c317369f",
                    "sha256": "bf10623121745750ba9c9dbf7deac151763ecfa260bca6f4480ebe6b1a098330"
                },
                "downloads": -1,
                "filename": "gcp_utility-1.1.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0112f6e31171df6349bf0776c317369f",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 8423,
                "upload_time": "2018-10-27T16:09:16",
                "url": "https://files.pythonhosted.org/packages/b8/7f/398ed48ea0a764f7189ffd3fe3853c4859c424057d458faa724f2d68e02f/gcp_utility-1.1.0-py2.py3-none-any.whl"
            }
        ],
        "1.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0852cbb682154228560a02e0646b79c4",
                    "sha256": "aa0d52b82aa498cadd5abe6cea4d832fd11c8cdcca8e951cc15fc706a7e52e58"
                },
                "downloads": -1,
                "filename": "gcp_utility-1.1.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0852cbb682154228560a02e0646b79c4",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 8421,
                "upload_time": "2018-11-16T18:21:43",
                "url": "https://files.pythonhosted.org/packages/bd/c3/d556fdd04be2b05570c4eab66695ddc539cae453e35978c61ca9e09d9d07/gcp_utility-1.1.1-py2.py3-none-any.whl"
            }
        ],
        "1.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "789aaa213a49536ec959d536f042f1f1",
                    "sha256": "19b632f64bf174bb46c1570c90a2203c6edef1676a5d02ce43a3fcd2c289d4bc"
                },
                "downloads": -1,
                "filename": "gcp_utility-1.1.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "789aaa213a49536ec959d536f042f1f1",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 8716,
                "upload_time": "2018-11-22T12:35:14",
                "url": "https://files.pythonhosted.org/packages/22/0d/701592613e9bee8d56a164a70fc9660ca3cf81aabd2042a5d4d1496483f8/gcp_utility-1.1.2-py2.py3-none-any.whl"
            }
        ],
        "1.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "c24c0c206ad0270b1385392a9ff05fd9",
                    "sha256": "c32a2b8171feb943cc162932e13c588e493f2408fd3661fbacf884eadf5883ca"
                },
                "downloads": -1,
                "filename": "gcp_utility-1.1.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "c24c0c206ad0270b1385392a9ff05fd9",
                "packagetype": "bdist_wheel",
                "python_version": "py2.py3",
                "requires_python": null,
                "size": 8793,
                "upload_time": "2018-12-19T08:16:07",
                "url": "https://files.pythonhosted.org/packages/48/84/30a0aea81d94470dc26ea4e2a8b7066210a510356a3d49cf9548ef7caeca/gcp_utility-1.1.3-py2.py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "c24c0c206ad0270b1385392a9ff05fd9",
                "sha256": "c32a2b8171feb943cc162932e13c588e493f2408fd3661fbacf884eadf5883ca"
            },
            "downloads": -1,
            "filename": "gcp_utility-1.1.3-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c24c0c206ad0270b1385392a9ff05fd9",
            "packagetype": "bdist_wheel",
            "python_version": "py2.py3",
            "requires_python": null,
            "size": 8793,
            "upload_time": "2018-12-19T08:16:07",
            "url": "https://files.pythonhosted.org/packages/48/84/30a0aea81d94470dc26ea4e2a8b7066210a510356a3d49cf9548ef7caeca/gcp_utility-1.1.3-py2.py3-none-any.whl"
        }
    ]
}