{ "info": { "author": "Hannu Valtonen", "author_email": "hannu.valtonen@ohmu.fi", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "Intended Audience :: System Administrators", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Topic :: Database :: Database Engines/Servers", "Topic :: Software Development :: Libraries" ], "description": "PGHoard |BuildStatus|_\n======================\n\n.. |BuildStatus| image:: https://travis-ci.org/aiven/pghoard.png?branch=master\n.. _BuildStatus: https://travis-ci.org/aiven/pghoard\n\n``pghoard`` is a PostgreSQL backup daemon and restore tooling that stores backup data in cloud object stores.\n\nFeatures:\n\n* Automatic periodic basebackups\n* Automatic transaction log (WAL/xlog) backups (using either ``pg_receivexlog``,\n ``archive_command`` or experimental PG native replication protocol support with ``walreceiver``)\n* Optional Standalone Hot Backup support\n* Cloud object storage support (AWS S3, Google Cloud, OpenStack Swift, Azure, Ceph)\n* Backup restoration directly from object storage, compressed and encrypted\n* Point-in-time-recovery (PITR)\n* Initialize a new standby from object storage backups, automatically configured as\n a replicating hot-standby\n\nFault-resilience and monitoring:\n\n* Persists over temporary object storage connectivity issues by retrying transfers\n* Verifies WAL file headers before upload (backup) and after download (restore),\n so that e.g. files recycled by PostgreSQL are ignored\n* Automatic history cleanup (backups and related WAL files older than N days)\n* \"Archive sync\" tool for detecting holes in WAL backup streams and fixing them\n* \"Archive cleanup\" tool for deleting obsolete WAL files from the archive\n* Keeps statistics updated in a file on disk (for monitoring tools)\n* Creates alert files on disk on problems (for monitoring tools)\n\nPerformance:\n\n* Parallel compression and encryption\n* WAL pre-fetching on restore\n\n\nOverview\n========\n\nPostgreSQL Point In Time Replication (PITR) consists of a having a database\nbasebackup and changes after that point go into WAL log files that can be\nreplayed to get to the desired replication point.\n\nPGHoard supports multiple operating models. The basic mode where you have a\nseparate backup machine, ``pghoard`` can simply connect with\n``pg_receivexlog`` to receive WAL files from the database as they're\nwritten. Another model is to use ``pghoard_postgres_command`` as a\nPostgreSQL ``archive_command``. There is also experimental support for PGHoard to\nuse PostgreSQL's native replication protocol with the experimental\n``walreceiver`` mode.\n\nWith both modes of operations PGHoard creates periodic basebackups using\n``pg_basebackup`` that is run against the database in question.\n\nThe PostgreSQL write-ahead log (WAL) and basebackups are compressed with\nSnappy (default), Zstandard (configurable, level 3 by default) or LZMA (configurable,\nlevel 0 by default) in order to ensure good compression speed and relatively small backup size.\nFor performance critical applications it is recommended to test compression\nalgorithms to find the most suitable trade-off for the particular use-case.\nE.g. Snappy is fast but yields larger compressed files, Zstandard (zstd) on the other hand\noffers a very wide range of compression/speed trade-off.\n\nOptionally, PGHoard can encrypt backed up data at rest. Each individual\nfile is encrypted and authenticated with file specific keys. The file\nspecific keys are included in the backup in turn encrypted with a master\nRSA private/public key pair.\n\nPGHoard supports backing up and restoring from either a local filesystem or\nfrom various object stores (AWS S3, Azure, Ceph, Google Cloud and OpenStack\nSwift.)\n\nIn case you just have a single database machine, it is heavily recommended\nto utilize one of the object storage services to allow backup recovery even\nif the host running PGHoard is incapacitated.\n\n\nRequirements\n============\n\nPGHoard can backup and restore PostgreSQL versions 9.3 and above. The\ndaemon is implemented in Python and works with CPython version 3.3 or newer.\nThe following Python modules are required:\n\n* psycopg2_ to look up transaction log metadata\n* requests_ for the internal client-server architecture\n\n.. _`psycopg2`: http://initd.org/psycopg/\n.. _`requests`: http://www.python-requests.org/en/latest/\n\nOptional requirements include:\n\n* azure_ for Microsoft Azure object storage\n* botocore_ for AWS S3 (or Ceph-S3) object storage\n* google-api-client_ for Google Cloud object storage\n* cryptography_ for backup encryption and decryption (version 0.8 or newer required)\n* snappy_ for Snappy compression and decompression\n* zstandard_ for Zstandard (zstd) compression and decompression\n* systemd_ for systemd integration\n* swiftclient_ for OpenStack Swift object storage\n* paramiko_ for sftp object storage\n\n.. _`azure`: https://github.com/Azure/azure-sdk-for-python\n.. _`botocore`: https://github.com/boto/botocore\n.. _`google-api-client`: https://github.com/google/google-api-python-client\n.. _`cryptography`: https://cryptography.io/\n.. _`snappy`: https://github.com/andrix/python-snappy\n.. _`zstandard`: https://github.com/indygreg/python-zstandard\n.. _`systemd`: https://github.com/systemd/python-systemd\n.. _`swiftclient`: https://github.com/openstack/python-swiftclient\n.. _`paramiko`: https://github.com/paramiko/paramiko\n\nDeveloping and testing PGHoard also requires the following utilities:\nflake8_, pylint_ and pytest_.\n\n.. _`flake8`: https://flake8.readthedocs.io/\n.. _`pylint`: https://www.pylint.org/\n.. _`pytest`: http://pytest.org/\n\nPGHoard has been developed and tested on modern Linux x86-64 systems, but\nshould work on other platforms that provide the required modules.\n\nVagrant\n=======\n\nThe Vagrantfile can be used to setup a vagrant development environment, consisting of two\nvagrant virtual machines.\n\n1) Postgresql 9.4, python 3.5 and 3.6::\n\n vagrant up\n vagrant ssh postgres9\n cd /vagrant\n source ~/venv3/bin/activate\n make test\n source ~/venv3.6/bin/activate\n make test\n\n2) Postgresql 10 and python 3.7::\n\n vagrant ssh postgres10\n cd /vagrant\n make test\n\nNote: make deb does not work from vagrant at the moment, hopefully this will be resolved soon\n\nBuilding\n========\n\nTo build an installation package for your distribution, go to the root\ndirectory of a PGHoard Git checkout and run:\n\nDebian::\n\n make deb\n\nThis will produce a ``.deb`` package into the parent directory of the Git\ncheckout.\n\nFedora::\n\n make rpm\n\nThis will produce a ``.rpm`` package usually into ``rpm/RPMS/noarch/``.\n\nPython/Other::\n\n python setup.py bdist_egg\n\nThis will produce an egg file into a dist directory within the same folder.\n\n\nInstallation\n============\n\nTo install it run as root:\n\nDebian::\n\n dpkg -i ../pghoard*.deb\n\nFedora::\n\n dnf install rpm/RPMS/noarch/*\n\nOn Linux systems it is recommended to simply run ``pghoard`` under\n``systemd``::\n\n systemctl enable pghoard.service\n\nand eventually after the setup section, you can just run::\n\n systemctl start pghoard.service\n\nPython/Other::\n\n easy_install dist/pghoard-1.7.0-py3.6.egg\n\nOn systems without ``systemd`` it is recommended that you run ``pghoard``\nunder Supervisor_ or other similar process control system.\n\n.. _`Supervisor`: http://supervisord.org\n\n\nSetup\n=====\n\nAfter this you need to create a suitable JSON configuration file for your\ninstallation.\n\n0. Make sure PostgreSQL is configured to allow WAL archival and retrieval.\n ``postgresql.conf`` should have ``wal_level`` set to ``archive`` or\n higher and ``max_wal_senders`` set to at least ``1`` (``archive_command`` mode)\n or at least ``2`` (``pg_receivexlog`` and ``walreceiver`` modes), for example::\n\n wal_level = archive\n max_wal_senders = 4\n\n Note that changing ``wal_level`` or ``max_wal_senders`` settings requires\n restarting PostgreSQL.\n\n1. Create a suitable PostgreSQL user account for ``pghoard``::\n\n CREATE USER pghoard PASSWORD 'putyourpasswordhere' REPLICATION;\n\n2. Edit the local ``pg_hba.conf`` to allow access for the newly created\n account to the ``replication`` database from the master and standby\n nodes. For example::\n\n # TYPE DATABASE USER ADDRESS METHOD\n host replication pghoard 127.0.0.1/32 md5\n\n After editing, please reload the configuration with either::\n\n SELECT pg_reload_conf();\n\n or by sending directly a ``SIGHUP`` to the PostgreSQL ``postmaster`` process.\n\n3. Fill in the created user account and master/standby addresses into the\n configuration file ``pghoard.json`` to the section ``backup_sites``.\n\n4. Fill in the possible object storage user credentials into the\n configuration file ``pghoard.json`` under section ``object_storage``\n in case you wish ``pghoard`` to back up into the cloud.\n\n5. Now copy the same ``pghoard.json`` configuration to the standby\n node if there are any.\n\nOther possible configuration settings are covered in more detail under the\n`Configuration keys`_ section of this README.\n\n6. If all has been set up correctly up to this point, ``pghoard`` should now be\n ready to be started.\n\n\nBacking up your database\n========================\n\nPostgreSQL backups consist of full database backups, *basebackups*, plus\nwrite ahead logs and related metadata, *WAL*. Both *basebackups* and *WAL*\nare required to create and restore a consistent database (does not apply\nfor standalone hot backups).\n\nTo enable backups with PGHoard the ``pghoard`` daemon must be running\nlocally. The daemon will periodically take full basebackups of the database\nfiles to the object store. Additionally, PGHoard and PostgreSQL must be set\nup correctly to archive the WAL. There are two ways to do this:\n\nThe default option is to use PostgreSQL's own WAL-archive mechanism with\n``pghoard`` by running the ``pghoard`` daemon locally and entering the\nfollowing configuration keys in ``postgresql.conf``::\n\n archive_mode = on\n archive_command = pghoard_postgres_command --mode archive --site default --xlog %f\n\nThis instructs PostgreSQL to call the ``pghoard_postgres_command`` whenever\na new WAL segment is ready. The command instructs PGHoard to store the\nsegment in its object store.\n\nThe other option is to set up PGHoard to read the WAL stream directly from\nPostgreSQL. To do this ``archive_mode`` must be disabled in\n``postgresql.conf`` and ``pghoard.json`` must set ``active_backup_mode`` to\n``pg_receivexlog`` in the relevant site, for example::\n\n {\n \"backup_sites\": {\n \"default\": {\n \"active_backup_mode\": \"pg_receivexlog\",\n ...\n },\n },\n ...\n }\n\nNote that as explained in the `Setup`_ section, ``postgresql.conf`` setting\n``wal_level`` must always be set to ``archive``, ``hot_standby`` or\n``logical`` and ``max_wal_senders`` must allow 2 connections from PGHoard,\ni.e. it should be set to 2 plus the number of streaming replicas, if any.\n\nWhile ``pghoard`` is running it may be useful to read the JSON state file\n``pghoard_state.json`` that exists where ``json_state_file_path`` points.\nThe JSON state file is human readable and is meant to describe the current\nstate of ``pghoard`` 's backup activities.\n\n\nStandalone Hot Backup Support\n=============================\n\nPghoard has the option to enable standalone hot backups.\n\nTo do this ``archive_mode`` must be disabled in ``postgresql.conf`` and\n``pghoard.json`` must set ``active_backup_mode`` to ``standalone_hot_backup``\nin the relevant site, for example::\n\n\n {\n \"backup_sites\": {\n \"default\": {\n \"active_backup_mode\": \"standalone_hot_backup\",\n ...\n },\n },\n ...\n }\n\n\nFor more information refer to the postgresql documentation\nhttps://www.postgresql.org/docs/9.5/continuous-archiving.html#BACKUP-STANDALONE\n\n\nRestoring databases\n===================\n\nYou can list your database basebackups by running::\n\n pghoard_restore list-basebackups --config /var/lib/pghoard/pghoard.json\n\n Basebackup Size Start time Metadata\n ------------------------------- ---- -------------------- ------------\n default/basebackup/2016-04-12_0 8 MB 2016-04-12T07:31:27Z {'original-file-size': '48060928',\n 'start-wal-segment': '000000010000000000000012',\n 'compression-algorithm': 'snappy'}\n\nIf we'd want to restore to the latest point in time we could fetch the\nrequired basebackup by running::\n\n pghoard_restore get-basebackup --config /var/lib/pghoard/pghoard.json \\\n --target-dir /var/lib/pgsql/9.5/data --restore-to-master\n\n Basebackup complete.\n You can start PostgreSQL by running pg_ctl -D foo start\n On systemd based systems you can run systemctl start postgresql\n On SYSV Init based systems you can run /etc/init.d/postgresql start\n\nNote that the ``target-dir`` needs to be either an empty or non-existent\ndirectory in which case PGHoard will automatically create it.\n\nAfter this we'd proceed to start both the PGHoard server process and the\nPostgreSQL server normally by running (on systemd based systems, assuming\nPostgreSQL 9.5 is used)::\n\n systemctl start pghoard\n systemctl start postgresql-9.5\n\nWhich will make PostgreSQL start recovery process to the latest point\nin time. PGHoard must be running before you start up the\nPostgreSQL server. To see other possible restoration options please run::\n\n pghoard_restore --help\n\n\nCommands\n========\n\nOnce correctly installed, there are six commands available:\n\n``pghoard`` is the main daemon process that should be run under a service\nmanager, such as ``systemd`` or ``supervisord``. It handles the backup of\nthe configured sites.\n\n``pghoard_restore`` is a command line tool that can be used to restore a\nprevious database backup from either ``pghoard`` itself or from one of the\nsupported object stores. ``pghoard_restore`` can also configure\n``recovery.conf`` to use ``pghoard_postgres_command`` as the WAL\n``restore_command`` in ``recovery.conf``.\n\n``pghoard_archive_cleanup`` can be used to clean up any orphan WAL files\nfrom the object store. After the configured number of basebackups has been\nexceeded (configuration key ``basebackup_count``), ``pghoard`` deletes the\noldest basebackup and all WAL associated with it. Transient object storage\nfailures and other interruptions can cause the WAL deletion process to leave\norphan WAL files behind, they can be deleted with this tool.\n\n``pghoard_archive_sync`` can be used to see if any local files should\nbe archived but haven't been or if any of the archived files have unexpected\ncontent and need to be archived again. The other usecase it has is to determine\nif there are any gaps in the required files in the WAL archive\nfrom the current WAL file on to to the latest basebackup's first WAL file.\n\n``pghoard_create_keys`` can be used to generate and output encryption keys\nin the ``pghoard`` configuration format.\n\n``pghoard_postgres_command`` is a command line tool that can be used as\nPostgreSQL's ``archive_command`` or ``recovery_command``. It communicates with\n``pghoard`` 's locally running webserver to let it know there's a new file that\nneeds to be compressed, encrypted and stored in an object store (in archive\nmode) or it's inverse (in restore mode.)\n\n\nConfiguration keys\n==================\n\n``active`` (default ``true``)\n\nCan be set on a per ``backup_site`` level to ``false`` to disable the taking\nof new backups and to stop the deletion of old ones.\n\n``active_backup_mode`` (default ``pg_receivexlog``)\n\nCan be either ``pg_receivexlog`` or ``archive_command``. If set to\n``pg_receivexlog``, ``pghoard`` will start up a ``pg_receivexlog`` process to be\nrun against the database server. If ``archive_command`` is set, we rely on the\nuser setting the correct ``archive_command`` in\n``postgresql.conf``. You can also set this to the experimental ``walreceiver`` mode\nwhereby pghoard will start communicating directly with PostgreSQL\nthrough the replication protocol. (Note requires an unreleased version\nof psycopg2 library)\n\n``alert_file_dir`` (default ``backup_location`` if set else ``os.getcwd()``)\n\nDirectory in which alert files for replication warning and failover are\ncreated.\n\n``backup_location`` (no default)\n\nPlace where ``pghoard`` will create its internal data structures for local state\ndata and the actual backups. (if no object storage is used)\n\n``backup_sites`` (default ``{}``)\n\nThis object contains names and configurations for the different PostgreSQL\nclusters (here called ``sites``) from which to take backups. The\nconfiguration keys for sites are listed below.\n\n* ``compression`` WAL/basebackup compression parameters\n\n * ``algorithm`` default ``\"snappy\"`` if available, otherwise ``\"lzma\"`` or ``\"zstd\"``\n * ``level`` default ``\"0\"`` compression level for ``\"lzma\"`` or ``\"zstd\"`` compression\n * ``thread_count`` (default max(cpu_count, ``5``)) number of parallel compression threads\n\n``hash_algorithm`` (default ``\"sha1\"``)\n\nThe hash algorithm used for calculating checksums for WAL or other files. Must\nbe one of the algorithms supported by Python's hashlib.\n\n``http_address`` (default ``\"127.0.0.1\"``)\n\nAddress to bind the PGHoard HTTP server to. Set to an empty string to\nlisten to all available addresses.\n\n``http_port`` (default ``16000``)\n\nHTTP webserver port. Used for the archive command and for fetching of\nbasebackups/WAL's when restoring if not using an object store.\n\n``json_state_file_path`` (default ``\"/var/lib/pghoard/pghoard_state.json\"``)\n\nLocation of a JSON state file which describes the state of the ``pghoard``\nprocess.\n\n``log_level`` (default ``\"INFO\"``)\n\nDetermines log level of ``pghoard``.\n\n``maintenance_mode_file`` (default ``\"/var/lib/pghoard/maintenance_mode_file\"``)\n\nIf a file exists in this location, no new backup actions will be started.\n\n``restore_prefetch`` (default ``transfer.thread_count``)\n\nNumber of files to prefetch when performing archive recovery. The default\nis the number of Transfer Agent threads to try to utilize them all.\n\n``statsd`` (default: disabled)\n\nEnables metrics sending to a statsd daemon that supports Telegraf\nor DataDog syntax with tags.\n\nThe value is a JSON object::\n\n {\n \"host\": \"\",\n \"port\": ,\n \"format\": \"\",\n \"tags\": {\n \"\": \"\"\n }\n }\n\n``format`` (default: ``\"telegraf\"``)\n\nDetermines statsd message format. Following formats are supported:\n\n* ``telegraf`` `Telegraf spec`_\n\n.. _`Telegraf spec`: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/statsd\n\n* ``datadog`` `DataDog spec`_\n\n.. _`DataDog spec`: http://docs.datadoghq.com/guides/dogstatsd/#datagram-format\n\nThe ``tags`` setting can be used to enter optional tag values for the metrics.\n\n``pushgateway`` (default: disabled)\n\nEnables metrics sending to a Prometheus Pushgateway with tags.\n\nThe value is a JSON object::\n\n {\n \"endpoint\": \"\",\n \"tags\": {\n \"\": \"\"\n }\n }\n\nThe ``tags`` setting can be used to enter optional tag values for the metrics.\n\n``prometheus`` (default: disabled)\n\nExpose metrics through a Prometheus endpoint.\n\nThe value is a JSON object::\n\n {\n \"tags\": {\n \"\": \"\"\n }\n }\n\nThe ``tags`` setting can be used to enter optional tag values for the metrics.\n\n``syslog`` (default ``false``)\n\nDetermines whether syslog logging should be turned on or not.\n\n``syslog_address`` (default ``\"/dev/log\"``)\n\nDetermines syslog address to use in logging (requires syslog to be true as\nwell)\n\n``syslog_facility`` (default ``\"local2\"``)\n\nDetermines syslog log facility. (requires syslog to be true as well)\n\n* ``transfer`` WAL/basebackup transfer parameters\n\n * ``thread_count`` (default max(cpu_count, ``5``)) number of parallel uploads/downloads\n\n``upload_retries_warning_limit`` (default ``3``)\n\nAfter this many failed upload attempts for a single file, create an\nalert file.\n\n``tar_executable`` (default ``\"pghoard_gnutaremu\"``)\n\nThe tar command to use for restoring basebackups. This must be GNU tar because some\nadvanced switches like ``--transform`` are needed. If this value is not defined (or\nis explicitly set to ``\"pghoard_gnutaremu\"``), Python's internal tarfile\nimplementation is used. The Python implementation is somewhat slower than the\nactual tar command and in environments with fast disk IO (compared to available CPU\ncapacity) it is recommended to set this to ``\"tar\"``.\n\nBackup site configuration\n=========================\n\nThe following options control the behavior of each backup site. A backup\nsite means an individual PostgreSQL installation (\"cluster\" in PostgreSQL\nterminology) from which to take backups.\n\n``basebackup_chunks_in_progress`` (default ``5``)\n\nHow many basebackup chunks can there be simultaneously on disk while\nit is being taken. For chunk size configuration see ``basebackup_chunk_size``.\n\n``basebackup_chunk_size`` (default ``2147483648``)\n\nIn how large backup chunks to take a ``local-tar`` basebackup. Disk\nspace needed for a successful backup is this variable multiplied by\n``basebackup_chunks_in_progress``.\n\n``basebackup_count`` (default ``2``)\n\nHow many basebackups should be kept around for restoration purposes. The\nmore there are the more diskspace will be used.\n\n``basebackup_interval_hours`` (default ``24``)\n\nHow often to take a new basebackup of a cluster. The shorter the interval,\nthe faster your recovery will be, but the more CPU/IO usage is required from\nthe servers it takes the basebackup from. If set to a null value basebackups\nare not automatically taken at all.\n\n``basebackup_mode`` (default ``\"basic\"``)\n\nThe way basebackups should be created. The default mode, ``basic`` runs\n``pg_basebackup`` and waits for it to write an uncompressed tar file on the\ndisk before compressing and optionally encrypting it. The alternative mode\n``pipe`` pipes the data directly from ``pg_basebackup`` to PGHoard's\ncompression and encryption processing reducing the amount of temporary disk\nspace that's required.\n\nNeither ``basic`` nor ``pipe`` modes support multiple tablespaces.\n\nSetting ``basebackup_mode`` to ``local-tar`` avoids using ``pg_basebackup``\nentirely when ``pghoard`` is running on the same host as the database.\nPGHoard reads the files directly from ``$PGDATA`` in this mode and\ncompresses and optionally encrypts them. This mode allows backing up user\ntablespaces.\n\nNote that the ``local-tar`` backup mode can not be used on replica servers\nprior to PostgreSQL 9.6 unless the pgespresso extension is installed.\n\n``encryption_key_id`` (no default)\n\nSpecifies the encryption key used when storing encrypted backups. If this\nconfiguration directive is specified, you must also define the public key\nfor storing as well as private key for retrieving stored backups. These\nkeys are specified with ``encryption_keys`` dictionary.\n\n``encryption_keys`` (no default)\n\nThis key is a mapping from key id to keys. Keys in turn are mapping from\n``public`` and ``private`` to PEM encoded RSA public and private keys\nrespectively. Public key needs to be specified for storing backups. Private\nkey needs to be in place for restoring encrypted backups.\n\nYou can use ``pghoard_create_keys`` to generate and output encryption keys\nin the ``pghoard`` configuration format.\n\n``object_storage`` (no default)\n\nConfigured in ``backup_sites`` under a specific site. If set, it must be an\nobject describing a remote object storage. The object must contain a key\n``storage_type`` describing the type of the store, other keys and values are\nspecific to the storage type.\n\nThe following object storage types are supported:\n\n* ``local`` makes backups to a local directory, see ``pghoard-local-minimal.json``\n for example. Required keys:\n\n * ``directory`` for the path to the backup target (local) storage directory\n\n* ``sftp`` makes backups to a sftp server, required keys:\n\n * ``server``\n * ``port``\n * ``username``\n * ``password`` or ``private_key``\n\n* ``google`` for Google Cloud Storage, required configuration keys:\n\n * ``project_id`` containing the Google Storage project identifier\n * ``bucket_name`` bucket where you want to store the files\n * ``credential_file`` for the path to the Google JSON credential file\n\n* ``s3`` for Amazon Web Services S3, required configuration keys:\n\n * ``aws_access_key_id`` for the AWS access key id\n * ``aws_secret_access_key`` for the AWS secret access key\n * ``region`` S3 region of the bucket\n * ``bucket_name`` name of the S3 bucket\n\nOptional keys for Amazon Web Services S3:\n\n * ``encrypted`` if True, use server-side encryption. Default is False.\n\n* ``s3`` for other S3 compatible services such as Ceph, required\n configuration keys:\n\n * ``aws_access_key_id`` for the AWS access key id\n * ``aws_secret_access_key`` for the AWS secret access key\n * ``bucket_name`` name of the S3 bucket\n * ``host`` for overriding host for non AWS-S3 implementations\n * ``port`` for overriding port for non AWS-S3 implementations\n * ``is_secure`` for overriding the requirement for https for non AWS-S3\n * ``is_verify_tls`` for configuring tls verify for non AWS-S3\n implementations\n\n* ``azure`` for Microsoft Azure Storage, required configuration keys:\n\n * ``account_name`` for the name of the Azure Storage account\n * ``account_key`` for the secret key of the Azure Storage account\n * ``bucket_name`` for the name of Azure Storage container used to store\n objects\n * ``azure_cloud`` Azure cloud selector, ``\"public\"`` (default) or ``\"germany\"``\n\n* ``swift`` for OpenStack Swift, required configuration keys:\n\n * ``user`` for the Swift user ('subuser' in Ceph RadosGW)\n * ``key`` for the Swift secret_key\n * ``auth_url`` for Swift authentication URL\n * ``container_name`` name of the data container\n\n * Optional configuration keys for Swift:\n\n * ``auth_version`` - ``2.0`` (default) or ``3.0`` for keystone, use ``1.0`` with\n Ceph Rados GW.\n * ``segment_size`` - defaults to ``1024**3`` (1 gigabyte). Objects larger\n than this will be split into multiple segments on upload. Many Swift\n installations require large files (usually 5 gigabytes) to be segmented.\n * ``tenant_name``\n * ``region_name``\n * ``user_id`` - for auth_version 3.0\n * ``user_domain_id`` - for auth_version 3.0\n * ``user_domain_name`` - for auth_version 3.0\n * ``tenant_id`` - for auth_version 3.0\n * ``project_id`` - for auth_version 3.0\n * ``project_name`` - for auth_version 3.0\n * ``project_domain_id`` - for auth_version 3.0\n * ``project_domain_name`` - for auth_version 3.0\n * ``service_type`` - for auth_version 3.0\n * ``endpoint_type`` - for auth_version 3.0\n\n``nodes`` (no default)\n\nArray of one or more nodes from which the backups are taken. A node can be\ndescribed as an object of libpq key: value connection info pairs or libpq\nconnection string or a ``postgres://`` connection uri. If for example you'd\nlike to use a streaming replication slot use the syntax {... \"slot\": \"slotname\"}.\n\n``pg_bin_directory`` (default: find binaries from well-known directories)\n\nSite-specific option for finding ``pg_basebackup`` and ``pg_receivexlog``\ncommands matching the given backup site's PostgreSQL version. If a value is\nnot supplied PGHoard will attempt to find matching binaries from various\nwell-known locations. In case ``pg_data_directory`` is set and points to a\nvalid data directory the lookup is restricted to the version contained in\nthe given data directory.\n\n``pg_data_directory`` (no default)\n\nThis is used when the ``local-tar`` ``basebackup_mode`` is used. The data\ndirectory must point to PostgreSQL's ``$PGDATA`` and must be readable by the\n``pghoard`` daemon.\n\n``prefix`` (default: site name)\n\nPath prefix to use for all backups related to this site. Defaults to the\nname of the site.\n\n\nAlert files\n===========\n\nAlert files are created whenever an error condition that requires human\nintervention to solve. You're recommended to add checks for the existence\nof these files to your alerting system.\n\n``authentication_error``\n\nThere has been a problem in the authentication of at least one of the\nPostgreSQL connections. This usually denotes a wrong username and/or\npassword.\n\n``configuration_error``\n\nThere has been a problem in the authentication of at least one of the\nPostgreSQL connections. This usually denotes a missing ``pg_hba.conf`` entry or\nincompatible settings in postgresql.conf.\n\n``upload_retries_warning``\n\nUpload of a file has failed more times than\n``upload_retries_warning_limit``. Needs human intervention to figure\nout why and to delete the alert once the situation has been fixed.\n\n``version_mismatch_error``\n\nYour local PostgreSQL client versions of ``pg_basebackup`` or\n``pg_receivexlog`` do not match with the servers PostgreSQL version. You\nneed to update them to be on the same version level.\n\n``version_unsupported_error``\n\nServer PostgreSQL version is not supported.\n\n\nLicense\n=======\n\nPGHoard is licensed under the Apache License, Version 2.0. Full license text\nis available in the ``LICENSE`` file and at\nhttp://www.apache.org/licenses/LICENSE-2.0.txt\n\n\nCredits\n=======\n\nPGHoard was created by Hannu Valtonen for\n`Aiven Cloud Database`_ and is now maintained by `Ohmu Ltd`_ hackers and\nAiven developers .\n\n.. _`Ohmu Ltd`: https://ohmu.fi/\n.. _`Aiven Cloud Database`: https://aiven.io/\n\nRecent contributors are listed on the GitHub project page,\nhttps://github.com/aiven/pghoard/graphs/contributors\n\n\nContact\n=======\n\nBug reports and patches are very welcome, please post them as GitHub issues\nand pull requests at https://github.com/aiven/pghoard . Any possible\nvulnerabilities or other serious issues should be reported directly to the\nmaintainers .\n\n\nCopyright\n=========\n\nCopyright (C) 2015 Ohmu Ltd\n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/aiven/pghoard/", "keywords": "", "license": "Apache 2.0", "maintainer": "", "maintainer_email": "", "name": "pghoard", "package_url": "https://pypi.org/project/pghoard/", "platform": "POSIX", "project_url": "https://pypi.org/project/pghoard/", "project_urls": { "Homepage": "https://github.com/aiven/pghoard/" }, "release_url": "https://pypi.org/project/pghoard/2.1.0/", "requires_dist": null, "requires_python": "", "summary": "PostgreSQL automatic backup/restore service daemon", "version": "2.1.0" }, "last_serial": 5321898, "releases": { "1.0.0": [ { "comment_text": "", "digests": { "md5": "036da5333010e5a9c8e74f2d5a8c1cb2", "sha256": "06b715d88f0cdfb88df57ee0e0203da54b357676f25f2765256fd71c6576fa0d" }, "downloads": -1, "filename": "pghoard-1.0.0.tar.gz", "has_sig": false, "md5_digest": "036da5333010e5a9c8e74f2d5a8c1cb2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 67227, "upload_time": "2016-03-16T16:49:29", "url": "https://files.pythonhosted.org/packages/28/3d/3730bf258be4dd972f82cd33865fea39b4aaf8f829ab5345b1a07c9c07dc/pghoard-1.0.0.tar.gz" } ], "1.1.0": [ { "comment_text": "", "digests": { "md5": "d9cf72f7eae0803066dbec38188cd308", "sha256": "096a6329982b3236dfcd41ae3c500e8d022425cdeaad546084d8746feafc4904" }, "downloads": -1, "filename": "pghoard-1.1.0.tar.gz", "has_sig": false, "md5_digest": "d9cf72f7eae0803066dbec38188cd308", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 74131, "upload_time": "2016-04-05T08:09:03", "url": "https://files.pythonhosted.org/packages/e6/85/0da0a9b8737f7f080dc8fc8aaa2822655b5659cb3bf75bd058e296954304/pghoard-1.1.0.tar.gz" } ], "1.2.0": [ { "comment_text": "", "digests": { "md5": "64650790ef80298ef74165d48b7707f2", "sha256": "2cb8308064e8b6a5fa17342b2ee60944e99dff3f42017ad70754e275fb9e49d9" }, "downloads": -1, "filename": "pghoard-1.2.0.tar.gz", "has_sig": false, "md5_digest": "64650790ef80298ef74165d48b7707f2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 87543, "upload_time": "2016-04-28T10:49:15", "url": "https://files.pythonhosted.org/packages/53/b4/c747efc46717803e9984589dbbca4e0fd9b1f6e0d6d74b08e1a7d6d223a0/pghoard-1.2.0.tar.gz" } ], "1.3.0": [ { "comment_text": "", "digests": { "md5": "00cfe1059de186a2480498e11272f0ae", "sha256": "c1bc694610965ca7aef60ec7c62be3a2532c4bc0a3b70ad3acc9c2f3a34e9551" }, "downloads": -1, "filename": "pghoard-1.3.0.tar.gz", "has_sig": false, "md5_digest": "00cfe1059de186a2480498e11272f0ae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 90341, "upload_time": "2016-05-31T18:55:07", "url": "https://files.pythonhosted.org/packages/62/b2/9a9ec9ca1365299fc66ba39a7777f4dbd90ba7bb42b02ab34cdd8d275c0a/pghoard-1.3.0.tar.gz" } ], "1.4.0": [ { "comment_text": "", "digests": { "md5": "844b316ffa0365419328769c97cedc51", "sha256": "0ec08aba90ca7ed1a614df4e9301e9e2294a68d2294b2527f5f00af333e9475e" }, "downloads": -1, "filename": "pghoard-1.4.0.tar.gz", "has_sig": false, "md5_digest": "844b316ffa0365419328769c97cedc51", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 109117, "upload_time": "2016-09-28T21:11:32", "url": "https://files.pythonhosted.org/packages/09/40/2790398f43f5e03256797e429ddd80ab1dc5cac45e18d3bca942b5dcfea1/pghoard-1.4.0.tar.gz" } ], "1.5.0": [ { "comment_text": "", "digests": { "md5": "cade727d1bb73064c614de91bb9c4937", "sha256": "6a3f52626cc5dc94bf70cb71757f692e0aff957c15f53925dd2865427a754298" }, "downloads": -1, "filename": "pghoard-1.5.0.tar.gz", "has_sig": false, "md5_digest": "cade727d1bb73064c614de91bb9c4937", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 124904, "upload_time": "2017-06-07T13:00:47", "url": "https://files.pythonhosted.org/packages/9e/c3/3621b33f21da1828ffd16c49034b7b008d9cb4ef10de86d5d0d66c93be85/pghoard-1.5.0.tar.gz" } ], "1.7.0": [ { "comment_text": "", "digests": { "md5": "024cd4a68941966a1747ad9d368e6852", "sha256": "dfa3a2eecd9457abcd19e979eaf0386ae3e86906b11384f38687854a8d75e670" }, "downloads": -1, "filename": "pghoard-1.7.0.tar.gz", "has_sig": false, "md5_digest": "024cd4a68941966a1747ad9d368e6852", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 129950, "upload_time": "2018-02-09T12:28:48", "url": "https://files.pythonhosted.org/packages/ec/ae/095d13183d9fbbac7a8fcfe72a322c51527dd7a17306e39f926dde8564cd/pghoard-1.7.0.tar.gz" } ], "2.0.0": [ { "comment_text": "", "digests": { "md5": "d62d8390827085f8950db9c2629ab9c2", "sha256": "9fcddbf50b0c31d4287655e122dcea7d304c66ab6cbf089520d34884b7e76cd0" }, "downloads": -1, "filename": "pghoard-2.0.0.tar.gz", "has_sig": false, "md5_digest": "d62d8390827085f8950db9c2629ab9c2", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 143058, "upload_time": "2018-12-17T09:52:16", "url": "https://files.pythonhosted.org/packages/07/23/0c0b3919fc60003bd0f18770819de0c494af4091f8111b2ec1967931a165/pghoard-2.0.0.tar.gz" } ], "2.1.0": [ { "comment_text": "", "digests": { "md5": "2c8912484e96e5d7942092ffb8185fae", "sha256": "516eea50d385fccc9ebfdf71dc1ded9af4c23451e5478791919a187514578da3" }, "downloads": -1, "filename": "pghoard-2.1.0.tar.gz", "has_sig": false, "md5_digest": "2c8912484e96e5d7942092ffb8185fae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 160708, "upload_time": "2019-05-27T11:02:41", "url": "https://files.pythonhosted.org/packages/0a/41/d59cb2fb09ef0ff2ae4eb8ca05a87a31c39741e8bd9e27e657cb73d9ffd1/pghoard-2.1.0.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "2c8912484e96e5d7942092ffb8185fae", "sha256": "516eea50d385fccc9ebfdf71dc1ded9af4c23451e5478791919a187514578da3" }, "downloads": -1, "filename": "pghoard-2.1.0.tar.gz", "has_sig": false, "md5_digest": "2c8912484e96e5d7942092ffb8185fae", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 160708, "upload_time": "2019-05-27T11:02:41", "url": "https://files.pythonhosted.org/packages/0a/41/d59cb2fb09ef0ff2ae4eb8ca05a87a31c39741e8bd9e27e657cb73d9ffd1/pghoard-2.1.0.tar.gz" } ] }