{ "info": { "author": "Yanis Guenane", "author_email": "yguenane@redhat.com", "bugtrack_url": null, "classifiers": [ "Environment :: Console", "Intended Audience :: Developers", "Intended Audience :: Information Technology", "License :: OSI Approved :: Apache Software License", "Operating System :: POSIX :: Linux", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.4" ], "description": "# swiftbackmeup\n\nAn utility that allows one to create items backups and upload them to\nOpenStack Swift (Object Store).\n\n## TODO\n\n - [ ] Add the possibility to encrypt data\n - [ ] Add support for Amazon S3\n\n## Goal\n\nThe goal of `swiftbackmeup` is to be able to backup items and upload\nthose backups to Swift (OpenStack Object Store).\n\n`swiftbackmeup` is configuration driven. Every item that needs to be\nbacked up are described in the configuration file.\n\n\n## How to run it\n\n`swiftbackmeup` has various operation modes:\n\n * `backup`: Allow one to backup items\n * `restore`: Allow one to restore backups from object store\n * `purge`: Allow one to purge backups from object store\n\n\n`swiftbackmeup` is configuration driven. The configuration file\nis (by order of priority):\n\n 1. The one specified on the command line (`swiftbackmeup --conf /path/to/conf.yml`)\n 2. The one specified in the environment variable `$SWIFTBACKMEUP_CONFIGURATION`\n 3. The `/etc/swiftbackmeup.conf`\n\n\n### backup\n\nThis mode allows a user to backup items listed in the backups array in the configuration\nfile.\n\n#### `--items`\n\nThis option allows one to limit for which backups item the script will be run.\nIf the backups array has serveral items; ie. `mydb_prod`, `mydb_preprod`, `mydb_test`\n\n`swiftbackmeup backup` would backup all three of them\n\n`swiftbackmeup backup --items mydb_prod` would only backup `mydb_prod`\n\n**Note**: this command is valid in combination with any other command such as `--list` and `--list-items\n\n\n#### `--list`\n\nThis option allows one to list the remote backups for the items listed in the configuration\nfile. It will by default list all the backups of every items. Items can be limited by using\nthe `--items` parameter.\n\n```\n#> swiftbackmeup backup --list\n+---------------+-----------------------------------------------------+----------------------------+\n| Item | Backup file | Last Modified |\n+---------------+-----------------------------------------------------+----------------------------+\n| db1 | db1/db1_20160624054028.dump.sql | 2016-06-24T09:40:29.719150 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624054028.dump.sql | 2016-06-24T09:40:30.377840 |\n+---------------+-----------------------------------------------------+----------------------------+\n```\n\n```\n#> swiftbackmeup backup --list --items db1\n+---------------+-----------------------------------------------------+----------------------------+\n| Item | Backup file | Last Modified |\n+---------------+-----------------------------------------------------+----------------------------+\n| db1 | db1/db1_20160624054028.dump.sql | 2016-06-24T09:40:29.719150 |\n+---------------+-----------------------------------------------------+----------------------------+\n```\n\n\n#### `--list-items`\n\nThis option allows one to list the items in the backups array listed in the configuration file.\nItems can be limited by using the `--items` parameter.\n\n```\n#> swiftbackmeup backup --list-items\n+---------------+----------------------+-----------+-----------------+---------------------+-----------------------------+\n| Item | Type | Host | Swift Container | Swift Pseudo-Folder | Subscriptions |\n+---------------+----------------------+-----------+-----------------+---------------------+-----------------------------+\n| db1 | databases/mariadb | 127.0.0.1 | backup | db1 | daily, now, monthly, weekly |\n| swiftbackmeup | databases/postgresql | local | backup | swiftbackmeup | daily, now, monthly, weekly |\n+---------------+----------------------+-----------+-----------------+---------------------+-----------------------------+\n```\n\n```\n#> swiftbackmeup backup --list-items --items db1\n+---------------+----------------------+-----------+-----------------+---------------------+-----------------------------+\n| Item | Type | Host | Swift Container | Swift Pseudo-Folder | Subscriptions |\n+---------------+----------------------+-----------+-----------------+---------------------+-----------------------------+\n| db1 | databases/mariadb | 127.0.0.1 | backup | db1 | daily, now, monthly, weekly |\n+---------------+----------------------+-----------+-----------------+---------------------+-----------------------------+\n```\n\n\n### purge\n\nThis mode allows a user to purge items on the remote object store.\n\nThe purge logic is based on the mode `retention` and `unit` parameters.\n\n```\nmodes:\n daily:\n retention: 7\n unit: day\n now:\n retention: 1\n unit: item\n\nbackups:\n - name: db1\n database: db1\n subscriptions:\n - daily\n - now\n```\n\nIf a user executes `swiftbackmeup purge` only the last 1 item of the db1 backups\non the remote store will be kept, the other will be purged (ie. --mode now is the\ndefault)\n\n\nIf a user executs `swiftbackmeup purge --mode daily` the backups of the item older\nthan 7 days will be purged.\n\n#### `--noop`\n\nThis option allows one to list the items that would be purged if run without the `--noop` item.\n\n```\n#> swiftbackmeup backup --list\n+---------------+-----------------------------------------------------+----------------------------+\n| Item | Backup file | Last Modified |\n+---------------+-----------------------------------------------------+----------------------------+\n| db1 | db1/db1_20160624054028.dump.sql | 2016-06-24T09:40:29.719150 |\n| db1 | db1/db1_20160624063610.dump.sql | 2016-06-24T10:36:12.295490 |\n| db1 | db1/db1_20160624063613.dump.sql | 2016-06-24T10:36:14.780210 |\n| db1 | db1/db1_20160624063615.dump.sql | 2016-06-24T10:36:17.117850 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624054028.dump.sql | 2016-06-24T09:40:30.377840 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063611.dump.sql | 2016-06-24T10:36:13.216240 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063614.dump.sql | 2016-06-24T10:36:15.799500 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063616.dump.sql | 2016-06-24T10:36:17.923470 |\n+---------------+-----------------------------------------------------+----------------------------+\n#> swiftbackmeup purge --noop --force\n+---------------+-----------------------------------------------------+----------------------------+---------------+\n| Item | Backup file | Last Modified | Status |\n+---------------+-----------------------------------------------------+----------------------------+---------------+\n| db1 | db1/db1_20160624054028.dump.sql | 2016-06-24T09:40:29.719150 | Purged (noop) |\n| db1 | db1/db1_20160624063610.dump.sql | 2016-06-24T10:36:12.295490 | Purged (noop) |\n| db1 | db1/db1_20160624063613.dump.sql | 2016-06-24T10:36:14.780210 | Purged (noop) |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624054028.dump.sql | 2016-06-24T09:40:30.377840 | Purged (noop) |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063611.dump.sql | 2016-06-24T10:36:13.216240 | Purged (noop) |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063614.dump.sql | 2016-06-24T10:36:15.799500 | Purged (noop) |\n+---------------+-----------------------------------------------------+----------------------------+---------------+\n```\n#### `--force`\n\nThis options allows one not to have to answer the security question: \"Are you sure you want to purge the backups?\"\n\n```\n#> swiftbackmeup backup --list \n+---------------+-----------------------------------------------------+----------------------------+\n| Item | Backup file | Last Modified |\n+---------------+-----------------------------------------------------+----------------------------+\n| db1 | db1/db1_20160624054028.dump.sql | 2016-06-24T09:40:29.719150 |\n| db1 | db1/db1_20160624063610.dump.sql | 2016-06-24T10:36:12.295490 |\n| db1 | db1/db1_20160624063613.dump.sql | 2016-06-24T10:36:14.780210 |\n| db1 | db1/db1_20160624063615.dump.sql | 2016-06-24T10:36:17.117850 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624054028.dump.sql | 2016-06-24T09:40:30.377840 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063611.dump.sql | 2016-06-24T10:36:13.216240 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063614.dump.sql | 2016-06-24T10:36:15.799500 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063616.dump.sql | 2016-06-24T10:36:17.923470 |\n+---------------+-----------------------------------------------------+----------------------------+\n#> swiftbackmeup purge --force\n+---------------+-----------------------------------------------------+----------------------------+--------+\n| Item | Backup file | Last Modified | Status |\n+---------------+-----------------------------------------------------+----------------------------+--------+\n| db1 | db1/db1_20160624054028.dump.sql | 2016-06-24T09:40:29.719150 | Purged |\n| db1 | db1/db1_20160624063610.dump.sql | 2016-06-24T10:36:12.295490 | Purged |\n| db1 | db1/db1_20160624063613.dump.sql | 2016-06-24T10:36:14.780210 | Purged |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624054028.dump.sql | 2016-06-24T09:40:30.377840 | Purged |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063611.dump.sql | 2016-06-24T10:36:13.216240 | Purged |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063614.dump.sql | 2016-06-24T10:36:15.799500 | Purged |\n+---------------+-----------------------------------------------------+----------------------------+--------+\n#> swiftbackmeup backup --list\n+---------------+-----------------------------------------------------+----------------------------+\n| Item | Backup file | Last Modified |\n+---------------+-----------------------------------------------------+----------------------------+\n| db1 | db1/db1_20160624063615.dump.sql | 2016-06-24T10:36:17.117850 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063616.dump.sql | 2016-06-24T10:36:17.923470 |\n+---------------+-----------------------------------------------------+----------------------------+\n```\n\n\n### restore\n\nThis mode allows a user to restore items from the remote object store.\n\nTo work, it needs the name of the item to restore and the version from which to restore it from\n\n```\n#> swiftbackmeup backup --list\n+---------------+-----------------------------------------------------+----------------------------+\n| Item | Backup file | Last Modified |\n+---------------+-----------------------------------------------------+----------------------------+\n| db1 | db1/db1_20160624063615.dump.sql | 2016-06-24T10:36:17.117850 |\n| swiftbackmeup | swiftbackmeup/swiftbackmeup_20160624063616.dump.sql | 2016-06-24T10:36:17.923470 |\n+---------------+-----------------------------------------------------+----------------------------+\n#> swiftbackmeup restore --items db1 --version db1/db1_20160624063615.dump.sql\n```\n\nThe previous example will restore the database `db1` to the dump remotely stored as `db1/db1_20160624063615.dump.sql`\n\n#### `--force`\n\nThis options allows one not to have to answer the security question: \"Are you sure you want to restore the backup?\"\n\n\n## Modes\n\nModes are equivalent to tag the backup needs to be subscribed to.\n\nSo within the configuration file, backups are 'subscribed' to tags;\n\n```\nbackups:\n - database: mydatabasenumberone\n subscriptions:\n - daily\n - monthly\n - now\n - tag1\n - mydatabasenumberone\n```\n\nThis means that this backup will run only if `swiftbackmeup` is run with one\nof the tags lists in subscriptions\n\nModes are defined in the configuration file, by default 4 modes come predefined.\n\n * `daily`: If one wants to backup the database on a daily basis, with a default of 7 day of retention\n * `weekly`: If one wants to backup the database on a weekly basis, with a default of 4 weeks of retention\n * `monthly`: If one wants to backup the database on a monthly basis, with a default of 6 months of retention\n * `now`: If one wants to backup the database at the moment t, with a default of 10 backup having the same name pattern.\n\n\nModes understand for now only two parameters:\n\n * `retention`: Number of unit a backup should be kept, else purged.\n * `unit`: The unit the retention represent. Possible value: `day`, `item`. Default `day`.\n * `pattern`: Pattern that will be used in datetime.format later.\n\n\n## Naming configuration\n\nThere are various way the filename of the backup can be specified.\n\n 1. The backup is part of a tag with a matching pattern.\n\n\nBy default the name will be the pattern. But one can specify 2 parameters\n`backup_filename_prefix` and `backup_filename_suffix` to actually have a\nmeaningfull name.\n\nSo name would be the equivalent of:\n\n```\n'%s%s%s' % (backup_filename_prefix, modes.pattern, backup_filename_suffix)\n```\n\nExample:\n\n```\nbackups:\n - name: mydatabase\n database: mydatabase\n subscriptions:\n - daily\n backup_filename_prefix: 'mydatabase_'\n backup_filename_suffix: '.dump.gz'\n```\n\n 2. The backup is part of a tag with a matching pattern (or not)\n\n\nWhether the backup is part of a tag or not, one can override the final backup\nfilename by specifying `backup_filename`\n\n```\nbackups:\n - name: mydatabase\n database: mydatabase\n subscriptions:\n - daily\n backup_filename: 'mydatabase_backup.dump.gz'\n```\n\n## Configuration\n\nThe below section aims to explain every parameter of the configuration file\n\n### Swift Parameters\n\n| Parameter | Scope | Default | Description |\n|---------------------|----------------|---------|-------------------------------------------------------------------|\n| os_username | global | None | OpenStack Username |\n| os_password | global | None | OpenStack Password |\n| os_tenant_name | global | None | OpenStack Tenant Name |\n| os_auth_url | global | None | OpenStack Authentication URL |\n| store_type | global, backup | None | The store type to upload backup to (available: swift) |\n| create_container | global, backup | True | If the container does not exist, should it be created |\n| purge_container | global, backup | False | Should the remote objects be purged |\n| swift_container | global, backup | backup | Name of the swift container on which to store the database backup |\n| swift_pseudo_folder | global, backup | None | If wanted, name of the pseudo folder |\n\n\n### Filesystem Parameters\n\n| Parameter | Scope | Default | Description |\n|------------------------|----------------|----------|---------------------------------------------------------------------------------------------|\n| ouput_directory | global, backup | /var/tmp | Directory where to store the database backup |\n| clean_local_copy | global, backup | True | Should the local copy be removed once uploaded to Swift |\n| backup_filename | backup | None | Name the backup file (will override any mode pattern) |\n| backup_filename_prefix | backup | None | Prefix of the backup file name (mode pattern and backup_filename_suffix will be appended) |\n| backup_filename_suffix | backup | None | Suffix of the backup file name (backup_filename_prefix and mode pattern will be prepended ) |\n\n\n### Database Parameters\n\n| Parameter | Scope | Default | Description |\n|-----------------|----------------|------------|-----------------------------------------------------------------|\n| type | global, backup | postgresql | Database type (available: postgres, mariadb) |\n| dump_options | global, backup | None | Parameters to pass to the dump command |\n| database | backup | None | Name of the database to backup |\n| user | global, backup | None | User to connect to the database system |\n| password | global, backup | None | Password to connect to the database system |\n| host | global, backup | None | Host to connect to the database system |\n| port | global, backup | None | Port to connect to the database system |\n| subscriptions | backup | None | Mode that this database is backed up when activated |\n\n\n#### PostgreSQL Specifics\n\nWhen backuping PostgreSQL databases there are two modes of working:\n\n * `database: all`: This will make `swiftbackmeup` rely on the `pg_dumpall` program. It hence allows access to options like (`roles_only`, `globals_only`, etc...)\n * `database: mydatabase`: This will make `swiftbackmeup` rely on the `pg_dump` program.\n\n| Parameter | Scope | Default | Description |\n|------------------|---------------------|---------|---------------------------------------------------|\n| data_only | backup (the all db) | None | Should --data-only be passed to pg_dumpall |\n| globals_only | backup (the all db) | None | Should --globals-only be passed to pg_dumpall |\n| roles_only | backup (the all db) | None | Should --roles-only be passed to pg_dumpall |\n| schema_only | backup (the all db) | None | Should --schema-only be passed to pg_dumpall |\n| tablespaces_only | backup (the all db) | None | Should --tablespaces-only be passed to pg_dumpall |\n\n\n### Configuration file example\n\n```\n---\nos_username: os_username\nos_password: os_password\nos_tenant_name: os_tenant_name\nos_auth_url: os_auth_url\n\nstore_type: swift\nswift_container: backup\nswift_pseudo_folder: example\n\ncreate_container: True\npurge_container: False\n\noutput_directory: /var/tmp\nclean_local_copy: True\n\nmodes:\n daily:\n retention: 7\n unit: day\n pattern: \"%Y%m%d\"\n weekly:\n retention: 28\n unit: day\n pattern: \"%Y%m%d-%U\"\n monthly:\n retention: 31\n unit: day\n pattern: \"%Y%m\"\n now:\n retention: 10\n unit: items\n pattern: \"%Y%m%d%H%M%S\"\n\n\nbackups:\n\n - name: mytestfile\n type: file\n path: /tmp/file\n backup_filename_prefix: 'mytestfile_'\n backup_filename_suffix: '.bk'\n subscriptions:\n - now\n - daily\n - monthly\n\n - name: mygitrepo\n type: git\n path: /srv/git/mygitrepo\n branches: all\n backup_filename_prefix: 'mygitrepo_'\n backup_filename_suffix: '.bundle'\n subscriptions:\n - now\n - daily\n - monthly\n\n - name: swiftbackmeup_mariadb\n type: mariadb\n database: swiftbackmeup\n host: 127.0.0.1\n user: root\n password: passpass\n backup_filename_prefix: 'swiftbackmeup_mariadb_'\n backup_filename_suffix: '.dump.sql'\n swift_pseudo_folder: swiftbackmeup_mariadb\n subscriptions:\n - now\n - daily\n - monthly\n\n - name: swiftbackmeup_postgresql\n type: postgresql\n database: swiftbackmeup\n backup_filename_prefix: 'swiftbackmeup_postgresql_'\n dump_options: -Z9 -Fc\n backup_filename_suffix: '.dump'\n swift_pseudo_folder: swiftbackmeup_postgresql\n subscriptions:\n - now\n - daily\n - monthly\n```", "description_content_type": null, "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/redhat-cip/swiftbackmeup", "keywords": "", "license": "Apache v2.0", "maintainer": "", "maintainer_email": "", "name": "swiftbackmeup", "package_url": "https://pypi.org/project/swiftbackmeup/", "platform": "", "project_url": "https://pypi.org/project/swiftbackmeup/", "project_urls": { "Homepage": "https://github.com/redhat-cip/swiftbackmeup" }, "release_url": "https://pypi.org/project/swiftbackmeup/0.0.5/", "requires_dist": null, "requires_python": "", "summary": "Tools to backup databases to OpenStack Object Store Swift", "version": "0.0.5" }, "last_serial": 3367962, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "662b960730c19598c160cd44c75a885d", "sha256": "9445db5421cf1d04e1242117d88c9c0db31af45cbc311fe0a365c8dcfe903889" }, "downloads": -1, "filename": "swiftbackmeup-0.0.1.tar.gz", "has_sig": false, "md5_digest": "662b960730c19598c160cd44c75a885d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 16141, "upload_time": "2016-06-27T10:24:58", "url": "https://files.pythonhosted.org/packages/aa/6f/1321aab509b0efd5bd02dfe51e6b72aaa7802d8e5d1a25c98b5ea586fbee/swiftbackmeup-0.0.1.tar.gz" } ], "0.0.2": [ { "comment_text": "", "digests": { "md5": "50c78e11c5e6859e77b3438d3f794388", "sha256": "cb1baf1a54ed2c2a631c797056d67f6fa4316635830a9b407ffaad07e9af8cf4" }, "downloads": -1, "filename": "swiftbackmeup-0.0.2.tar.gz", "has_sig": false, "md5_digest": "50c78e11c5e6859e77b3438d3f794388", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15426, "upload_time": "2016-06-27T11:01:42", "url": "https://files.pythonhosted.org/packages/86/b3/3979697858b2a795332fdfb26bb3ab09bcb0a5c5194e41c8b642f752365a/swiftbackmeup-0.0.2.tar.gz" } ], "0.0.3": [], "0.0.4": [ { "comment_text": "", "digests": { "md5": "9a44dc50200dbce456db5259a4e83c17", "sha256": "229b3ff3a8be7a35610b4e46af82936b065263fdf7c0ad1847a3233770a6cd8a" }, "downloads": -1, "filename": "swiftbackmeup-0.0.4.tar.gz", "has_sig": false, "md5_digest": "9a44dc50200dbce456db5259a4e83c17", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 15467, "upload_time": "2016-07-04T12:15:45", "url": "https://files.pythonhosted.org/packages/52/8b/c52b6f2afb8872c25743d32237cc105b48391b89a6de8c2b586ef84e6234/swiftbackmeup-0.0.4.tar.gz" } ], "0.0.5": [ { "comment_text": "", "digests": { "md5": "5749f1fc4292000142c6fa9b3f7c7aaf", "sha256": "662149bb9ddbb9094602cf0271aea285907d39e30f3f88713f4343d415e59dfb" }, "downloads": -1, "filename": "swiftbackmeup-0.0.5.tar.gz", "has_sig": false, "md5_digest": "5749f1fc4292000142c6fa9b3f7c7aaf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14902, "upload_time": "2017-11-27T13:58:09", "url": "https://files.pythonhosted.org/packages/35/7e/9947718105f03fe43db4de30d270e9970f363332485675d207c2867c49e1/swiftbackmeup-0.0.5.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "5749f1fc4292000142c6fa9b3f7c7aaf", "sha256": "662149bb9ddbb9094602cf0271aea285907d39e30f3f88713f4343d415e59dfb" }, "downloads": -1, "filename": "swiftbackmeup-0.0.5.tar.gz", "has_sig": false, "md5_digest": "5749f1fc4292000142c6fa9b3f7c7aaf", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 14902, "upload_time": "2017-11-27T13:58:09", "url": "https://files.pythonhosted.org/packages/35/7e/9947718105f03fe43db4de30d270e9970f363332485675d207c2867c49e1/swiftbackmeup-0.0.5.tar.gz" } ] }