{
    "info": {
        "author": "Thinking Machines Data Science",
        "author_email": "engineering@thinkingmachin.es",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "# bqup\n\nBigQuery backup scripts\n\n## Dependencies\n\n1. [Python BigQuery Client Library](https://cloud.google.com/bigquery/docs/reference/libraries#client-libraries-usage-python)\n\n## Usage\n\n```\nUsage:\n  bqup [-p PROJECT_ID] [-d TARGET_DIR] [-fvx]\n\nOptions:\n  -p PROJECT_ID, --project PROJECT_ID  Project ID to load. If unspecified, defaults to current project in configuration.\n  -d TARGET_DIR, --dir TARGET_DIR      The target directory where the project will be written. Defaults to current timestamp.\n  -f --force                           Overwrite target directory if it exists.\n  -v --verbose                         Print a summary of the loaded project.\n  -x --schema                          Export table schemata as json.\n```\n\n## Development\n\n1. Set up `gcloud` to run with your personal account (aka run with scissors)\n1. Set up [application-default](https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login): `gcloud auth application-default login`\n1. Install wheel: `pip3 install wheel`\n1. Install `bqup` with `pip3 install -e .` (or `python3 setup.py develop`)\n1. Run `bqup` (see [Usage](#usage))\n\n## Production\n\n**Note: When deploying for a new GCP project, consider using [Cloud Scheduler](https://cloud.google.com/scheduler/)**\n\n1. Turn off your host Google Instance.\n1. Enable BigQuery on the instance's Cloud API Access Scopes\n1. Start the instance.\n1. `ssh` into the Google Instance you want to run `bqup` from.\n1. Authorize your Compute Instance Account to read from the target BigQuery project.\n1. Install `bqup` by cloning this repo and running `pip3 install --user -e .` inside the repo.\n1. Run `bqup`\n    - If it still doesn't work, check in IAM that the service account you are using has BigQuery read access.\n\n### Setting up regular backups\n\n1. On the machine that will run your backups, set up your git config (username, email, the usual).\n1. Make a directory to use as the Git repository. For this example, let's use `repo`:\n\n    ```\n    mkdir repo\n    cd repo\n    git init\n    ```\n\n1. Add the remote to the git repository (ideally a GCP repository). For this example, let's use `google`:\n\n    ```\n    git remote add google <url-to-remote-repository>\n    ```\n\n1. Create a script called `bqup.sh` that follows the following template. For our example, our repository is dedicated to backups, so we just assume that our `HEAD` is the latest and just push gently to `master`.\n\n    ```\n    #!/bin/bash\n    <path-to-bqup> -p <project-id> -d <path-to-repo>/projects -fv >> <path-to-log-file>\n    cd <path-to-repo>\n    date > last-updated.log\n    git add .\n    git commit -m \"Automated bqup\"\n    git push <remote> <branch>\n    ```\n\n1. Add this script to your [crontab](https://awc.com.my/uploadnew/5ffbd639c5e6eccea359cb1453a02bed_Setting%20Up%20Cron%20Job%20Using%20crontab.pdf) to run as frequently as your heart desires.\n\n## Distribution\n\n- Rebuild the `dist` directory\n\n    ```sh\n    rm -rf dist\n    python3 setup.py bdist_wheel\n    ```\n\n- Test upload (optional, needs account)\n\n    ```sh\n    twine upload -u thinkdatasci --repository-url https://test.pypi.org/legacy/ dist/*\n    ```\n\n- Final upload\n\n    ```sh\n    twine upload -u thinkdatasci dist/*\n    ```\n\n## Policies\n\n### Maintenance\n\n#### Disclaimers\n\nbqup is maintained on a **best effort** basis:\n\n- No amount of official time is currently being dedicated to the regular\nmaintenance of this project.\n- Thinking Machines does not make any guarantees about the quality of the\nsoftware.\n\n#### Maintainers\n\nThe official maintainers in charge of responding to issues and merging pull\nrequests are:\n\n- Primary:\n    - [Pepe Bawagan](https://github.com/syk0saje)\n- Secondary:\n    - [Mark Steve Samson](https://github.com/marksteve)\n    - [Carlson Cheng](https://github.com/crcheng)\n\n### Contribution\n\nA list is maintained for all external contributors who have submitted pull\nrequests that were subsequently approved. Users are allowed and encouraged to\nfork the project and submit pull requests and issues. All contributions must\nadhere to these guidelines:\n\n- [Commit messages](https://chris.beams.io/posts/git-commit/)\n- [Github flow](https://guides.github.com/introduction/flow/)\n\nThinking Machines reserves the rights to:\n\n- refuse to resolve issues\n- close issues without resolution\n- request changes to pull requests\n- reject pull requests outright\n\n\n",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/thinkingmachines/bqup",
        "keywords": "google bigquery,bigquery,bq,backup,bqup",
        "license": "MIT License",
        "maintainer": "",
        "maintainer_email": "",
        "name": "bqup",
        "package_url": "https://pypi.org/project/bqup/",
        "platform": "",
        "project_url": "https://pypi.org/project/bqup/",
        "project_urls": {
            "Homepage": "https://github.com/thinkingmachines/bqup"
        },
        "release_url": "https://pypi.org/project/bqup/0.0.5/",
        "requires_dist": [
            "docopt",
            "google-cloud-bigquery (==1.1.0)"
        ],
        "requires_python": "",
        "summary": "BigQuery backup scripts",
        "version": "0.0.5"
    },
    "last_serial": 4883759,
    "releases": {
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0a2247de4e03c84bc2e10832939b46d5",
                    "sha256": "760dcb00e2bdb51ecca04a76445b04146913a091af5552dc4663f35fabd83978"
                },
                "downloads": -1,
                "filename": "bqup-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "0a2247de4e03c84bc2e10832939b46d5",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 1941,
                "upload_time": "2019-03-01T09:52:20",
                "url": "https://files.pythonhosted.org/packages/a5/31/78c6a2c1651e51958479f145b26ce99978eda09aee3eb0de39313527b304/bqup-0.0.2-py3-none-any.whl"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "232619e37ea780d789d49342f188dd4f",
                    "sha256": "4b51726813d1a77810022607511488a537bfbbc6ca0a1af827f17de028cc95f9"
                },
                "downloads": -1,
                "filename": "bqup-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "232619e37ea780d789d49342f188dd4f",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 6160,
                "upload_time": "2019-03-01T10:36:47",
                "url": "https://files.pythonhosted.org/packages/09/ab/bc28c054fef2c6faafd1154f43016a579eec11874f4d6c1b9adde70e7954/bqup-0.0.3-py3-none-any.whl"
            }
        ],
        "0.0.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "88bb4d24cadf6b66a4ca24276bbb40d6",
                    "sha256": "d8c8e34d056fefa06b86ef08feff0daba19768f6dd9f190999c4cae3de5161ff"
                },
                "downloads": -1,
                "filename": "bqup-0.0.4-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "88bb4d24cadf6b66a4ca24276bbb40d6",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9179,
                "upload_time": "2019-03-01T11:08:24",
                "url": "https://files.pythonhosted.org/packages/b6/ff/a058119af15c64022cede5edd6fb851d8f3b838482c57f2f24da0f5df809/bqup-0.0.4-py3-none-any.whl"
            }
        ],
        "0.0.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "befb31a9040d7a73826d4550596e2060",
                    "sha256": "4dea5301fc2b3aeb6af0c55498885f39d7ad9aabf89db79f08de0fc38b9a404e"
                },
                "downloads": -1,
                "filename": "bqup-0.0.5-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "befb31a9040d7a73826d4550596e2060",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 9177,
                "upload_time": "2019-03-01T11:17:34",
                "url": "https://files.pythonhosted.org/packages/a3/5b/0dd3c74bd8ecb6733bba434a51d515b661513263407ce53ec8afc1dd1bab/bqup-0.0.5-py3-none-any.whl"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "befb31a9040d7a73826d4550596e2060",
                "sha256": "4dea5301fc2b3aeb6af0c55498885f39d7ad9aabf89db79f08de0fc38b9a404e"
            },
            "downloads": -1,
            "filename": "bqup-0.0.5-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "befb31a9040d7a73826d4550596e2060",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 9177,
            "upload_time": "2019-03-01T11:17:34",
            "url": "https://files.pythonhosted.org/packages/a3/5b/0dd3c74bd8ecb6733bba434a51d515b661513263407ce53ec8afc1dd1bab/bqup-0.0.5-py3-none-any.whl"
        }
    ]
}