{
    "info": {
        "author": "Adam Thornton",
        "author_email": "athornton@lsst.org",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 4 - Beta",
            "License :: OSI Approved :: MIT License",
            "Programming Language :: Python :: 2.7"
        ],
        "description": "# ![Travis](https://img.shields.io/travis/lsst-sqre/sqre-codekit.svg)\n\n# sqre-git-snapshot\n\nLSST DM SQuaRE github snapshot management tools\n\n## Components\n\n### github-snapshot\n\nMakes mirror clones of all public repositories in the specified\norganizations.\n\n### snapshot-purger\n\nRemoves old snapshots.  Retention criteria are:\n\n1. Everything is retained for a week.\n2. Saturday backups are retained for at least a month.\n3. First-of-the-month backups are retained for at least 3 months.\n4. First-of-the-quarter backups are never automatically deleted.\n\n## Installation\n\n### On an AWS host\n\n* In the AWS console:\n    * Create a new `t2.micro` (or whatever, but micro's plenty big) in\n      `us-west-2` using `ami-d2c924b2`.\n    * It needs IAM role `github-snapshot-s3-access`.\n    * Give it 30GB of SSD as root (you only really need enough for one\n      repository at a time, so more-free-space-than-the-biggest-repository\n      is good enough, but you're not charged extra (I think) for 30GB or\n      less).\n    * SSH is the only port it needs open.\n    * Launch it.\n    * Associate an EIP with it.\n    * (optional) Add a DNS record for that EIP\n* Once it comes up, log in as `centos`, and then:\n   `sudo -i`  \n   `hostnamectl set-hostname ghsnap.lsst.codes # Or whatever you called it`  \n   `yum update -y`  \n   `yum install -y epel-release && yum repolist`  \n   `yum install -y git python-pip python-virtualenvwrapper jq`  \n   `yum install -y emacs-nox # If you're not a barbarian`  \n   `curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh -o install-git-lfs-repo.sh`  \n   Examine `install-git-lfs-repo.sh` until you're sure it's not nefarious.  \n   `bash ./install-git-lfs-repo.sh`  \n   `yum -y install git-lfs-1.5.2-1.el7.x86_64`  \n   Reboot.\n* Once it is up again, log in as `centos`, and then:  \n   `mkdir Venvs gh-snap git`  \n   `cd git`  \n   `git clone https://github.com/lsst-sqre/shelltools.git`  \n   `cd ../gh-snap`  \n   `ln -s ../git/shelltools/lsst-shellfuncs.bash`    \n   `cd`  \n```bash\ncat << 'EOF' >> .bashrc\nif [ -f /usr/bin/virtualenvwrapper.sh ] && [ -z \"${VIRTUAL_ENV}\" ]; then\n    WORKON_HOME=${HOME}/Venvs\n    export WORKON_HOME\n    mkdir -p ${WORKON_HOME}\n    source /usr/bin/virtualenvwrapper.sh\nfi\n\nif [ -f \"${HOME}/gh-snap/lsst-shellfuncs.bash\" ]; then\n    source \"${HOME}/gh-snap/lsst-shellfuncs.bash\"\nfi\nEOF\n```\n* Log out and back in (as `centos` still).  Then:  \n  `mkvirtualenv github-snapshot`  \n  `pip install sqre-github-snapshot`  \n  `cd gh-snap`  \n```bash\ncat << 'EOF' > run_as_cronjob\n#!/bin/bash\naction=$1\ncase $action in\n\"purge\"|\"snap\")\n\t;;\n*)\n\techo 1>&2 \"Action must be 'purge' or 'snap'\"\n\texit 1\nesac\n\ndeclare -F | grep -q '^declare -f workon$'\nrc=$?\nif [ ${rc} -ne 0 ]; then\n    . ${HOME}/.bashrc\nelse\n    declare -F | grep -q '^declare -f deactivate$'\n    rc=$?\n    if [ ${rc} -ne 0 ]; then\n\t. ${HOME}/.bashrc\n    fi\nfi\n\nif [ -n \"${VIRTUAL_ENV}\" ]; then\n    vname=$(basename \"${VIRTUAL_ENV}\")\nfi\nif [ \"${vname}\" != \"github-snapshot\" ]; then\n    workon github-snapshot\nfi\n\ncheck_github_lfs\nset_aws_variables\n\nif [ \"${action}\" == \"purge\" ]; then\n    snapshot-purger\nelse\n    github-snapshot\nfi\nEOF\n```\n* Install cron job:    \n   `chmod 0755 run_as_cronjob`  \n   set `$EDITOR` if you don't like `vi`  \n   `crontab -e`  \n   Add the following:  \n```\n# Take backup snapshots every night at 12:23 AM\n# Purge old backups every night at 4:46 AM\n\n23 0 * * * /home/centos/gh-snap/run_as_cronjob snap\n46 4 * * * /home/centos/gh-snap/run_as_cronjob purge\n```\n\n### Docker\n\nBuild a container from the included `Dockerfile`.\n\nYou may want to specify some or all of the following environment\nvariables with -e when you do a `docker run`:\n\n`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`, \n`AWS_DEFAULT_REGION`, `GITHUB_ORGS`, `GITHUB_TOKEN`, `S3_BACKUP_BUCKET`\n\nThe default command is to take a snapshot; use `snapshot-purger` as an\nappended command on the docker run command line to run a purge instead.\n\nYou can also just pull the image with `docker pull\nlsstsqre/github-snapshot`.\n\n\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/lsst-sqre/sqre-github-snapshots/",
        "keywords": "lsst",
        "license": "MIT",
        "maintainer": "",
        "maintainer_email": "",
        "name": "sqre-github-snapshot",
        "package_url": "https://pypi.org/project/sqre-github-snapshot/",
        "platform": "",
        "project_url": "https://pypi.org/project/sqre-github-snapshot/",
        "project_urls": {
            "Homepage": "https://github.com/lsst-sqre/sqre-github-snapshots/"
        },
        "release_url": "https://pypi.org/project/sqre-github-snapshot/0.2.1/",
        "requires_dist": [
            "MapGitConfig (==1.1)",
            "awscli (==1.11.21)",
            "github3.py (==1.0.0a4)",
            "progressbar2 (==3.11.0)",
            "requests (==2.8.1)",
            "sqre-codekit (==1.0.1)",
            "uritemplate.py (==2.0.0)"
        ],
        "requires_python": "",
        "summary": "LSST Data Management SQuaRE Github Organization Snapshotter",
        "version": "0.2.1"
    },
    "last_serial": 2496565,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ba1f9b33e251aeedf15a522a9a8109ef",
                    "sha256": "6672750aad1e6ae3c0b728c061f86ce1607ed3a59548aff9c72baee4dd4fbaa0"
                },
                "downloads": -1,
                "filename": "sqre_github_snapshot-0.1.0-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "ba1f9b33e251aeedf15a522a9a8109ef",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 11632,
                "upload_time": "2016-12-01T23:03:14",
                "url": "https://files.pythonhosted.org/packages/10/39/57c7244bc450db9ee0b0b3272e6c3dc623e7177488e36c67442a4d272807/sqre_github_snapshot-0.1.0-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "b2151cd410910ce97978d38a3868bdc1",
                    "sha256": "f53db56042485d48e131eceb174fabf81008774d6687b9d3f14833ff36182dd8"
                },
                "downloads": -1,
                "filename": "sqre-github-snapshot-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "b2151cd410910ce97978d38a3868bdc1",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 8246,
                "upload_time": "2016-12-01T23:03:16",
                "url": "https://files.pythonhosted.org/packages/ef/ee/e3c76d7df8907ce2ea26f67343eb7db418c5ac547266a682dfe37c6fde82/sqre-github-snapshot-0.1.0.tar.gz"
            }
        ],
        "0.2.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "4d2c18a088e0aeaaea368cb238e655cb",
                    "sha256": "798a53ea803dc3cc0f28f0df6ff70b0f0ffe10198013e31ce3cfe0ce4d6135d3"
                },
                "downloads": -1,
                "filename": "sqre_github_snapshot-0.2.1-py2-none-any.whl",
                "has_sig": false,
                "md5_digest": "4d2c18a088e0aeaaea368cb238e655cb",
                "packagetype": "bdist_wheel",
                "python_version": "py2",
                "requires_python": null,
                "size": 12223,
                "upload_time": "2016-12-02T21:40:54",
                "url": "https://files.pythonhosted.org/packages/65/94/d00f7557e6949e15acbb8154f469af5257beec4eb61a5025961ad5be59f1/sqre_github_snapshot-0.2.1-py2-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "6d4d31c33cbc06fa534425a25cc0e81e",
                    "sha256": "f8f73b60054405f7cc206d9facf3f911233159e54c5eb008d22a4a97609fe8d1"
                },
                "downloads": -1,
                "filename": "sqre-github-snapshot-0.2.1.tar.gz",
                "has_sig": false,
                "md5_digest": "6d4d31c33cbc06fa534425a25cc0e81e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 9190,
                "upload_time": "2016-12-02T21:40:57",
                "url": "https://files.pythonhosted.org/packages/50/81/30b98d8cf5201b2fbdb2a2391863ee6e9196e74984ee54c986e320d47640/sqre-github-snapshot-0.2.1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "4d2c18a088e0aeaaea368cb238e655cb",
                "sha256": "798a53ea803dc3cc0f28f0df6ff70b0f0ffe10198013e31ce3cfe0ce4d6135d3"
            },
            "downloads": -1,
            "filename": "sqre_github_snapshot-0.2.1-py2-none-any.whl",
            "has_sig": false,
            "md5_digest": "4d2c18a088e0aeaaea368cb238e655cb",
            "packagetype": "bdist_wheel",
            "python_version": "py2",
            "requires_python": null,
            "size": 12223,
            "upload_time": "2016-12-02T21:40:54",
            "url": "https://files.pythonhosted.org/packages/65/94/d00f7557e6949e15acbb8154f469af5257beec4eb61a5025961ad5be59f1/sqre_github_snapshot-0.2.1-py2-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "6d4d31c33cbc06fa534425a25cc0e81e",
                "sha256": "f8f73b60054405f7cc206d9facf3f911233159e54c5eb008d22a4a97609fe8d1"
            },
            "downloads": -1,
            "filename": "sqre-github-snapshot-0.2.1.tar.gz",
            "has_sig": false,
            "md5_digest": "6d4d31c33cbc06fa534425a25cc0e81e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 9190,
            "upload_time": "2016-12-02T21:40:57",
            "url": "https://files.pythonhosted.org/packages/50/81/30b98d8cf5201b2fbdb2a2391863ee6e9196e74984ee54c986e320d47640/sqre-github-snapshot-0.2.1.tar.gz"
        }
    ]
}