{
    "info": {
        "author": "Stuart Bishop",
        "author_email": "stuart.bishop@canonical.com",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Environment :: Console",
            "Environment :: OpenStack",
            "Intended Audience :: System Administrators",
            "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
            "Natural Language :: English",
            "Operating System :: OS Independent",
            "Programming Language :: Python",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Topic :: Database",
            "Topic :: System :: Archiving :: Backup",
            "Topic :: System :: Recovery Tools",
            "Topic :: System :: Systems Administration"
        ],
        "description": "########\nSwiftWAL\n########\n\nAbout\n=====\n\nSwiftWAL is a tool for `PostgreSQL <http://postgresql.org/>`_ adminstrators,\nallowing filesystem level backups, WAL shipping and point-in-time recovery\n(PITR) to be made to and from OpenStack Swift storage. It is distrubuted\nunder the :doc:`GPLv3 license <license>`.\n\n\nRequirements\n------------\n\n* PostgreSQL 9.1 or later.\n\n* `pigz <http://zlib.net/pigz/>`_ for compression and decompression.\n  :program:`pigz` is a multithreaded, drop-in replacement to\n  :program:`gzip`.\n\n* The `python-swiftclient <https://pypi.python.org/pypi/python-swiftclient>`_\n  Python library.\n\n* :command:`pg_basebackup`, a standard tool shipped with PostgreSQL.\n\n\nLimitations\n-----------\n\n* PostgreSQL tablespaces are not supported due to limitations in\n  :command:`pg_basebackup`'s tar output format.\n\n\nUsage\n=====\n\nEverything is done through the command line tool :program:`swiftwal`.\n\nA standard setup allowing full |PITR| would be:\n\n    * :command:`swiftwal backup` scheduled to be run regularly, followed\n      by :command:`swiftwal prune` if the backup succeeded.\n\n    * :command:`swiftwal archive-wal` configured as the primary\n      PostgreSQL server's ``archive_command`` setting in\n      :file:`postgresql.conf`.\n\n    * :command:`swiftwal restore-wal` configured as the hot\n      standbys` ``restore_command`` setting in the :file:`recovery.conf`\n      files.\n\n\nCommon Options\n--------------\n\n.. TIP::\n    If you have the standard OpenStack environment variables\n    set, or a configuration file containing the settings,  you will not\n    need to provide them explicitly on the command line. For hooking\n    into PostgreSQL's ``archive_command`` and ``restore_command`` using\n    the :option:`--config` option is usually the best approach.\n\n:program:`swiftwal`\n\n.. program:: swiftwal\n\n.. option:: --container CONTAINER, -C CONTAINER\n\n    The Swift container name used to store everything. It is required\n    for all operations. You will generally use a unique container name\n    for each PostgreSQL server. You certainly don't want two or more\n    servers archiving WAL files into the same container as you will\n    experience conflicts.\n\n.. option:: --config FILE, -c FILE\n\n    Specify OpenStack authentication credentials and the container using\n    a config file::\n\n        OS_AUTH_URL=https://keystone.example.com/v2.0/\n        OS_USERNAME=alan_parsons\n        OS_PASSWORD=Pyramid\n        OS_TENANT_NAME=alan_parsons_project\n        CONTAINER=pgprod\n\n.. option:: --verbose, -v\n\n    Extra output and longer reports.\n\n.. option:: --os-auth-url URL\n\n    OpenStack authentication URL. Defaults to the\n    :envvar:`OS_AUTH_URL` environment variable.\n\n.. option:: --os-username USER\n\n    OpenStack username. Defaults to the :envvar:`OS_USERNAME`\n    environment variable.\n\n.. option:: --os-password KEY\n\n    OpenStack key. Defaults to the :envvar:`OS_PASSWORD` environment\n    variable.\n\n.. option:: --os-tenant-name TENANT\n\n    OpenStack tenant name. Defaults to the :envvar:`OS_TENANT_NAME`\n    environment variable.\n\n\nBackup\n------\n\nThe :command:`swiftwal backup` command wraps the standard PostgreSQL\nutility :program:`pg_basebackup`, compressing its tarball output and\nstreaming it into Swift storage. Further information about the backup\ncommand line options can be found in the\n:manpage:`pg_basebackup(1)` `documentation\n<http://www.postgresql.org/docs/9.1/static/app-pgbasebackup.html>`_.\n\n.. IMPORTANT::\n    Always use the :option:`--xlog` option if you do not have reliable WAL\n    archiving configured for |PITR|. This option ensures the required WAL\n    information is included in the backup. Without the required WAL\n    information your backup cannot be recovered.\n\n\nTo make a backup of your database into the ``pgprod`` container::\n\n    % swiftwal -C pgprod -v backup -c fast --xlog -H grind.example.com\n    Creating backup 20130828T1810Z in container pgprod\n    WARNING:  skipping special file \"./server.crt\"\n    WARNING:  skipping special file \"./server.key\"\n    START WAL LOCATION: 0/5C000020 (file 00000001000000000000005C)\n    CHECKPOINT LOCATION: 0/5C000058\n    BACKUP METHOD: streamed\n    START TIME: 2013-08-29 01:10:19 ICT\n    LABEL: pg_basebackup (via swiftwal)\n\n.. NOTE::\n    The WARNING messages here are emitted from :command:`pg_basebackup`,\n    and are always emitted on Debian and Ubuntu PostgreSQL default\n    installations as the SSL certificate files are installed as\n    symlinks.\n\n\nThe ISO-8601 timestamp ``20130828T1810Z`` is the key used to identify\nthis backup in SwiftWAL.\n\nThe :command:`swiftwal list-backups` command will show you the backups\nstored in Swift::\n\n    % swiftwal -C pgprod list-backups\n    20130829T1125Z\n    20130829T1131Z\n    20130902T1138Z\n    20130902T1311Z\n\nThe :option:`--verbose` option will show a more informative report::\n\n    % swiftwal -C pgprod --verbose list-backups\n    Size   \tTimestamp     \tSwift Object Name\n    =======\t==============\t================================\n    2.72 MB\t20130829T1125Z\tpg_basebackup_20130829T1125Z.tgz\n    2.72 MB\t20130829T1131Z\tpg_basebackup_20130829T1131Z.tgz\n    2.72 MB\t20130902T1138Z\tpg_basebackup_20130902T1138Z.tgz\n    2.72 MB\t20130902T1311Z\tpg_basebackup_20130902T1311Z.tgz\n\n    Total Size: 10.9 MB\n\n\n:program:`swiftwal backup`\n\n.. option:: --xlog, -x\n\n    Passed through to :program:`pg_basebackup`, instructing it to\n    include all WAL files required to restore the database. Without this\n    option, backups are useless unless the necessary WAL information\n    has been archived (eg. WAL archiving using\n    :command:`swiftwal archive-wal`.)\n\n.. option:: --checkpoint {fast, spread}, -c {fast, spread}\n\n    Start the backup as soon as possible, or wait until the next\n    checkpoint has completed normally. :option:`--checkpoint fast`\n    may cause unwanted load on a particularly busy server.\n\n.. option:: --progress, -P\n\n    Display progress indicators.\n\n.. option:: --label LABEL, -l LABEL\n\n    Set the backup label. This is for your own use, as SwiftWAL uses the\n    backup start time to reference backups.\n\n.. option:: --username NAME, -U NAME\n\n    Connect to PostgreSQL as the specified user. This user needs to have\n    the ``REPLICATION`` attribute granted with ``ALTER ROLE``, which is\n    normally the case for the default ``postgres`` user. It also needs\n    to have been granted permissions to connect to the hidden\n    ``replication`` database in :file:`pg_hba.conf`.\n\n.. option:: --host HOSTNAME, -H HOSTNAME\n\n    Connect to PostgreSQL on the specified host.\n\n.. option:: --port PORT, -p PORT\n\n    Connect to PostgreSQL on the specified port.\n\n\nRestore\n-------\n\nRestoring backups is done with the :command:`swiftwal restore` command.\nThis command streams the backup from Swift through decompression and\nuntar, unpacking the backup to the directory of your choosing. As a\nprecaution, the directory must be empty or non-existant::\n\n    % swiftwal --container=pgprod restore 20130828T1952Z ./\n    Total bytes read: 22159360 (22MiB, 993KiB/s)\n\n.. TIP::\n    With a default Debian or Ubuntu PostgreSQL setup, you will also need\n    to recreate the SSL certificate symlinks or update your\n    :file:`postgresql.conf` file to specify the path directly::\n\n        ln -s /etc/ssl/certs/ssl-cert-snakeoil.pem .\n        ln -s /etc/ssl/private/ssl-cert-snakeoil.key .\n\n\nReports\n-------\n\nThe :command:`swiftwal list-backups` command lists the backups stored\nin Swift.\n\nThe :command:`swiftwal list-wal` command lists the WAL files stored in\nSwift.\n\n\nWAL Archiving\n-------------\n\nThe :command:`swiftwal archive-wal` command can be used as an\n``archive_command`` in your PostgreSQL's :file:`postgresql.conf` file\nto archive WAL logs directly into Swift. This allows you to configure\nlog-shipped replication and, combined with a backup made with the\n:command:`swiftwal backup` command, lets you do |PITR|::\n\n    archive_mode = on\n    wal_level = hot_standby\n    archive_command = 'swiftwal --config=/etc/swiftwal.conf archive-wal %p'\n\nprogram:`swiftwal archive-wal`\n\n.. option:: --force, -f\n\n    Overwrite an existing WAL file. In normal operation, WAL archiving\n    commands should refuse to overwrite a WAL file if it already exists\n    at the target destination. This is a safety measure, and will\n    normally never happen unless you incorrectly configure two servers\n    to archive to the same Swift container. This option allows you to\n    override this behavior, helping you repair the problem.\n\n\nWAL Shipping\n------------\n\nThe :command:`swiftwal restore-wal` command can be used as a ``replay_command`` in a PostgreSQL :file:`recovery.conf` file. This lets you perform |PITR|,\nreplaying WAL information directly from Swift. It also lets you setup\nWAL log-shipping replication and create a warm or hot standby server.\nLog-shipping replication is often configured in addition to streaming\nreplication as a fall back, allowing a standby server to recover if it\nhas fallen behind for some reason without needing to keep vast amounts\nof WAL files available on the primary server using the\n``wal_keep_segments`` configuration option::\n\n    standby_mode = on\n    restore_command = 'swiftwal --config=/etc/swiftwal.conf restore-wal %f %p'\n    recovery_target_timeline = latest\n\n\nRemoving Backups & WAL Files\n----------------------------\n\nOld backups and WAL files can be removed using the :command:`swiftwal prune`\ncommand, reclaiming the disk space in Swift::\n\n  % swiftwal -C pgprod prune --keep-backups 3 --keep-wal 0\n  Removing    1 backups,   20130828T1524Z -> 20130828T1524Z\n  Keeping     3 backups,   20130828T1528Z -> 20130828T1904Z\n\n  Removing    2 WAL files, 000000010000000000000056 -> 000000010000000000000057\n  Keeping    13 WAL files, 000000010000000000000058 -> 000000010000000000000064\n\nUse the common :option:`--verbose` option to generate a more verbose report.\n\n.. IMPORTANT::\n    Do not clean out your old backups and WAL files unless you are\n    confident your newer ones are actually recoverable. If you are\n    not using the :option:`--xlog` option when making backups, it\n    is worth checking your servers $DATADIR/pg_xlog/archive_status\n    directory for files with a .ready extension; these correspond\n    WAL files that have not been archived and may mean WAL files you\n    need to restore your backup are not yet in Swift. A script like the\n    following will let you perform a regular backup and cleanup from\n    the PostgreSQL with confidence::\n\n        #!/bin/bash\n        PGDATA=/var/lib/postgresql/9.1/main\n        swiftwal -c /etc/swiftwal.conf -C pgprod backup\n        sleep 600\n        if [ `find ${PGDATA}/pg_xlog/archive_status -name \\*.ready -cmin +9` ]\n        then\n            echo ERROR: WAL archiving is failing. Leaving old backups.\n        else\n            swiftwal -c /etc/swiftwal.conf -C pgprod cleanup --keep-backups=1\n        fi\n\n    Alternatively, use the :option:`--xlog` option to make self-contained \n    backups.\n\n:program:`swiftwal prune`\n\n.. option:: --keep-backups N\n\n    How many backups to keep. The most recent backups are kept, and\n    older ones removed.\n\n.. option:: --keep-wals N\n\n    How many WAL files to keep. More will be kept if they are needed for\n    |PITR| of a backup (all WAL information younger than the oldest backup\n    is kept). Each WAL file is 16MB in size. If set to 0, all WAL files\n    are removed except for those needed for |PITR|.\n\n.. option:: --dry-run, -n\n\n    Don't actually delete anything.\n\n\nSimple Single Standby Cleanup\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nFor a simple setup involving a single standby server without |PITR|, the :command:`swiftwal archive-cleanup` command provides the same functionality as the standard pg_archivecleanup(1) tool shipped with PostgreSQL. If it is installed in your standby server's recovery.conf file, WAL files will be automatically removed once they have been replayed and are no longer needed::\n\n    standby_mode = on\n    restore_command = 'swiftwal -c /etc/swiftwal.conf restore-wal %f %p'\n    archive_cleanup_command='swiftwal -c /etc/swiftwal.conf archive-cleanup %r'\n    \n\n.. |PITR| replace:: :abbr:`PITR (point-in-time recovery)`",
        "description_content_type": null,
        "docs_url": "https://pythonhosted.org/swiftwal/",
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://launchpad.net/swiftwal",
        "keywords": "postgresql postgres wal pitr backup",
        "license": "UNKNOWN",
        "maintainer": null,
        "maintainer_email": null,
        "name": "swiftwal",
        "package_url": "https://pypi.org/project/swiftwal/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/swiftwal/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://launchpad.net/swiftwal"
        },
        "release_url": "https://pypi.org/project/swiftwal/0.2.2/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "PostgreSQL backups, WAL archiving & PITR to OpenStack Swift",
        "version": "0.2.2"
    },
    "last_serial": 983859,
    "releases": {
        "0.2.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "53d55b30c5496c108336a32ae020c3cc",
                    "sha256": "c59643d9b19f1787a0661faadf9ee2f08d93a4754476698b85c857cc61fb631a"
                },
                "downloads": -1,
                "filename": "swiftwal-0.2.2.tar.bz2",
                "has_sig": true,
                "md5_digest": "53d55b30c5496c108336a32ae020c3cc",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 27709,
                "upload_time": "2014-01-28T08:35:53",
                "url": "https://files.pythonhosted.org/packages/8b/19/9378af52fd05f105fd303d254a743065549c4e2fb0f5963c586ed3367df2/swiftwal-0.2.2.tar.bz2"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "53d55b30c5496c108336a32ae020c3cc",
                "sha256": "c59643d9b19f1787a0661faadf9ee2f08d93a4754476698b85c857cc61fb631a"
            },
            "downloads": -1,
            "filename": "swiftwal-0.2.2.tar.bz2",
            "has_sig": true,
            "md5_digest": "53d55b30c5496c108336a32ae020c3cc",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 27709,
            "upload_time": "2014-01-28T08:35:53",
            "url": "https://files.pythonhosted.org/packages/8b/19/9378af52fd05f105fd303d254a743065549c4e2fb0f5963c586ed3367df2/swiftwal-0.2.2.tar.bz2"
        }
    ]
}