{ "info": { "author": "Robert Schindler", "author_email": "r.schindler@efficiosoft.com", "bugtrack_url": null, "classifiers": [], "description": "btrfs-backup\n============\nThis project supports incremental backups for *btrfs* using *snapshots*\nand *send/receive* between filesystems. Think of it as a basic version\nof Time Machine.\n\nBackups can be stored locally and/or remotely (e.g. via SSH). Multi-target\nsetups are supported as well as dealing with transmission failures\n(e.g. due to network outage).\n\nIts main goals are to be **reliable** and **functional** while\nmaintaining **user-friendliness**. It should be easy to get started in\njust a few minutes without detailled knowledge on how btrfs send/receive\nworks. However, you should have a basic understanding of snapshots and\nsubvolumes.\n\nbtrfs-backup has almost no dependencies and hence is well suited for\nmany kinds of setups with only minimal maintenance effort.\n\nOriginally, it started as a fork of a project with the same name,\nwritten by Chris Lawrence. Since then, most of the code has been\nrefactored and many new features were added before this repository\nhas been transferred to me. Many thanks to Chris for his work.\nThe old code base has been tagged with ``legacy``. If, for any reason,\nyou want to continue using it and miss the new features, you can check\nthat out.\n\n:Latest release: v0.3.0\n:Downloads: http://pypi.python.org/pypi/btrfs_backup\n:Source: https://github.com/efficiosoft/btrfs-backup\n:Platforms: Linux >= 3.12, Python >= 3.3\n:Keywords: backup, btrfs, snapshot, send, receive, ssh\n\n\nFeatures\n--------\n- Initial creation of full backups\n- Incremental backups on subsequent runs\n- Different backup storage engines:\n\n - Local storage\n - Remote storage via SSH\n - Custom storage: Alternatively, the output of ``btrfs send`` may be\n piped to a custom shell command.\n\n- Multi-target support with tracking of which snapshots are missing at\n each location.\n- Retransmission on errors (e.g. due to network outage).\n- Simple and configurable retention policy for local and remote\n snapshots\n- Optionally, create snapshots without transferring them anywhere\n and vice versa.\n- Creation of backups without root privileges, if some special\n conditions are met\n- Detailled logging output with configurable log level\n\n\nInstallation\n------------\nRequirements\n~~~~~~~~~~~~\n- Python 3.3 or later\n- Appropriate btrfs-progs; typically you'll want **at least** 3.12 with\n Linux 3.12/3.13\n- (optional) OpenSSH's ``ssh`` command - needed for remote backup pulling\n and pushing via SSH\n- (optional) ``sshfs`` - only needed for pulling via SSH\n- (optional) ``pv`` command for displaying progress during backups\n\nInstall via PIP\n~~~~~~~~~~~~~~~\nThe easiest way to get up and running with the latest stable version is via PIP. If ``pip3`` is missing\non your system and you run a Debian-based distribution, simply install\nit via:\n\n::\n\n $ sudo apt-get install python3-pip python3-wheel\n\nThen, you can fetch the latest version of btrfs-backup:\n\n::\n\n $ sudo pip3 install btrfs_backup\n\nPre-built packages\n~~~~~~~~~~~~~~~~~~\nThere are pre-built packages available for the following distributions.\n\n- `Arch Linux `_ (thanks to XenGi for maintaining this)\n\nManual installation\n~~~~~~~~~~~~~~~~~~~\nAlternatively, clone this git repository\n\n::\n\n $ git clone https://github.com/efficiosoft/btrfs-backup\n $ cd btrfs-backup\n $ git checkout tags/v0.3.0 # optionally checkout a specific version\n $ sudo ./setup.py install\n\n\nSample usage\n------------\nNot every feature of btrfs-backup is explained in this README, since\nthere is a detailled and descriptive help included with the command.\n\nHowever, there are some sections about the general concepts and different\nsample usages to get started as quick as possible.\n\nFor reference, a copy of the output of ``btrfs-backup --help`` is\nattached below.\n\nAs root:\n\n::\n\n $ btrfs-backup /home /backup\n\nThis will create a read-only snapshot of ``/home`` in\n``/home/snapshot/YYMMDD-HHMMSS``, and then send it to\n``/backup/YYMMDD-HHMMSS``. On future runs, it will take a new read-only\nsnapshot and send the difference between the previous snapshot and the\nnew one.\n\n**Note: Both source and destination need to be on btrfs filesystems.\nAdditionally, the source has to be either the root or any other subvolume,\nbut not just an ordinary directory because snapshots can only be created\nof subvolumes.**\n\nFor the backup to be sensible, source and destination shouldn't be the\nsame filesystem. Otherwise you could just snapshot and save the hassle.\n\nYou can backup multiple subvolumes to multiple subfolders or subvolumes at\nthe destination. For example, you might want to backup both ``/`` and\n``/home``. The main caveat is you'll want to put the backups in separate\nfolders on the destination drive to avoid confusion.\n\n::\n\n $ btrfs-backup / /backup/root\n $ btrfs-backup /home /backup/home\n\nIf you really want to store backups of different subvolumes at the same\nlocation, you have to specify a prefix using the ``-p/--snapshot-prefix``\noption. Without that, btrfs-backup can't distinguish between your\ndifferent backup chains and will mix them up. Using the example from\nabove, it could look like the following:\n\n::\n\n $ btrfs-backup --snapshot-prefix root / /backup\n $ btrfs-backup --snapshot-prefix home /home /backup\n\nYou can specify ``-N/--num-snapshots `` to only keep the latest\n```` number of snapshots on the source filesystem. ``-n/--num-backups\n`` does the same thing for the backup location.\n\nRemote backups\n~~~~~~~~~~~~~~\nBacking up to a remote server via SSH is as easy as:\n\n::\n\n $ btrfs-backup /home ssh://server/mnt/backups\n\nbtrfs-backup doesn't need to be installed on the remote side for this\nto work. It is recommended to set up public key authentication to\neliminate the need for entering passwords. A full description of how\nto customize the ``ssh`` call can be found in the help text.\n\nPulling backups from a remote side is now supported as well! Simply use\nthe ``ssh://`` scheme as source.\n\nYou could even do something like:\n\n::\n\n $ btrfs-backup ssh://source_server/home ssh://dest_server/mnt/backups\n\nto pull backups from ``source_server`` and store them at\n``dest_server``. This might be used if you can't install btrfs-backup\non either remote host for any reason. But keep in mind that this procedure\nwill generate double traffic (from ``source_server`` to you and from\nyou to ``dest_server``).\n\nOkay, just one last example, because I really like that one:\n\n::\n\n $ btrfs-backup ssh://source_server/home \\\n /mnt/backups \\\n ssh://dest_server/mnt/backups\n\nCan you guess what it does? Well, it does the same as the command before +\nan extra sending to your local ``/mnt/backups`` folder. Please note that\nbtrfs-backup is not smart enough to prevent the same data from being\npulled from ``source_server`` twice. But that wouldn't be easy to\nimplement with the current design.\n\n\nHelp text\n---------\nThis is the output of ``btrfs-backup --help``. Taking a look at it,\nyou should get a good insight in what it can and can't do (yet).\n\n::\n\n Cooming at the release.\n\n\nConfiguration files\n-------------------\nBy default, btrfs-backup doesn't read any configuration file. However,\nyou can create one or more and specify them at the command line:\n\n::\n\n $ btrfs-backup @path/to/backup_home.conf\n\nAny argument prefixed by a ``@`` is treated as file name of a\nconfiguration file.\n\nThe format of these files is simple. On every line, there may be one flag,\noption or argument you would normally specify at the command line. Valid\nconfiguration files might look like the following.\n\n``backup_home.conf``:\n\n::\n\n # This is a comment and thus ignored, as well as blank lines.\n\n # Include another configuration file here.\n @global.conf\n\n # Indentation has no effect.\n -p home\n\n # This is the source.\n /home\n\n # Back up to both local and remote storage.\n /mnt/backups/home\n ssh://server/mnt/btrfs_storage/backups/home\n\n``global.conf``:\n\n::\n\n # This file gets included by the other one.\n --quiet\n\n --num-snapshots 1\n --num-backups 3\n\nA more detailled explanation about the format can be found in the help\ntext.\n\n\nWhat are locks?\n---------------\nbtrfs-backup uses so called \"locks\" to keep track of failed snapshot\ntransfers. There is a file called ``.outstanding_transfers`` created in\nthe snapshot folder. This file is in JSON format and thus human-readable,\nif necessary.\n\nLocking works as follows:\n\n#. When a snapshot transfer is started, an entry is created in that file,\n telling that a snapshot transfer of a specific snapshot to a specific\n destination has begun. We call this entry a lock.\n#. If the snapshot transfer used another snapshot as parent, that one\n gets an entry as well, but no lock, just the note that it's a parent\n for something that failed to transfer.\n#. When the transfer\n\n #. finishes without errors, the locks for the snapshot (and its parent)\n are removed.\n #. aborts (e.g. due to network outage or a full disk), the locks\n are kept.\n\nNow, there are multiple options for dealing with those failed transfers.\n\nWhen you run btrfs-backup the next time, it finds the corrupt snapshot\nat the destination and deletes it, together with the corresponding lock\nand parent notes. Afterwards, the way is free for a new transfer. You\nmay also use ``--no-snapshot`` to only do the transfers without creating\nnew snapshots.\n\nThere is a special flag called ``--locked-dests`` available. If supplied,\nit automatically adds all destinations which locks exist for as if they\nwere specified at the command line. You might do something like:\n\n::\n\n $ btrfs-backup --no-snapshot --locked-dests /home\n\nto retry all failed backup transfers of snapshots of ``/home``. This\ncould be executed periodically because it just does nothing if there\nare no locks.\n\nSnapshots for which locks or parent notes exist are excluded from the\nretention policy and won't be purged until the locks are removed either\nautomatically (because the partially transferred snapshots could be\ndeleted from the destination) or manually (see below).\n\nAs a last resort for removing locks for transfers you don't want to retry\nanymore, there is a flag called ``--remove-locks``. Use it with caution\nand only if you can assure that there are no corrupt snapshots at the\ndestinations you apply the flag on.\n\n::\n\n $ btrfs-backup --no-snapshot --no-transfer --remove-locks /home ssh://nas/backups\n\nwill remove all locks for the destination ``ssh://nas/backups`` from\n``/home/snapshot/.outstanding_transfers``. Of course, using\n``--locked-dests`` instead of specifying the destination explicitly is\npossible as well.\n\n\nBacking up regularly\n--------------------\nNote that there is no locking included with btrfs-backup. If you back\nup too often (i.e. more quickly than it takes the first call to finish,\nwhich can take several minutes, hours or even days on a filesystem with\nlots of files), you might end up with a new backup starting while an\nold one is still in progress.\n\nYou can workaround the lack of locking using the ``flock(1)`` command,\nas suggested at https://github.com/efficiosoft/btrfs-backup/issues/4.\n\nWith anacron on Debian, you could simply add a file\n``/etc/cron.daily/local-backup``:\n\n.. code:: sh\n\n #!/bin/sh\n flock -n /tmp/btrfs-backup-home.lock \\\n ionice -c 3 btrfs-backup --quiet --num-snapshots 1 --num-backups 3 \\\n /home /backup/home\n\nYou may omit the ``-n`` flag if you want to wait rather than fail in\ncase a backup is already running.\n\nMore or less frequent backups could be made using other ``cron.*``\nscripts.\n\n\nRestoring a snapshot\n--------------------\nIf necessary, you can restore a whole snapshot by using e.g.\n\n::\n\n $ mkdir /home/snapshot\n $ btrfs send /backup/YYMMDD-HHMMSS | btrfs receive /home/snapshot\n\nThen you need to take the read-only snapshot and turn it back into a\nroot filesystem:\n\n::\n\n $ cp -aR --reflink /home/snapshot/YYMMDD-HHMMSS /home\n\nYou might instead have some luck taking the restored snapshot and\nturning it into a read-write snapshot, and then re-pivoting your mounted\nsubvolume to the read-write snapshot.\n\n\nAlternative workflow\n--------------------\nAn alternative structure is to keep all subvolumes in the root directory\n\n::\n\n /\n /active\n /active/root\n /active/home\n /inactive\n /snapshot/root/YYMMDD-HHMMSS\n /snapshot/home/YYMMDD-HHMMSS\n\nand have corresponding entries in ``/etc/fstab`` to mount the subvolumes\nfrom ``/active/*``. One benefit of this approach is that restoring a\nsnapshot can be done entirely with btrfs tools:\n\n::\n\n $ btrfs send /backup/root/YYMMDD-HHMMSS | btrfs receive /snapshot/home\n $ btrfs send /backup/home/YYMMDD-HHMMSS | btrfs receive /snapshot/root\n $ mv /active/root /inactive\n $ mv /active/home /inactive\n $ btrfs subvolume snapshot /snapshot/root/YYMMDD-HHMMSS /active/root\n $ btrfs subvolume snapshot /snapshot/home/YYMMDD-HHMMSS /active/home\n\nThe snapshots from btrfs-backup may be placed in ``/snapshots/`` by\nusing the ``--snapshot-folder`` option.\n\n\nIssues and Contribution\n-----------------------\nAs in every piece of software, there likely are bugs. When you find one,\nplease open an issue on GitHub. If you do so, please include the output\nwith debug log level (``-v debug``) and provide steps to reproduce\nthe problem. Thank you!\n\nIf you want to contribute, that's great! You can create issues (even\nfor feature requests), send pull requests or contact me via email at\nr.schindler@efficiosoft.com.\n\n\nCopyright\n---------\n.. |copy| unicode:: U+000A9 .. COPYRIGHT SIGN\n| Copyright |copy| 2017 Robert Schindler \n| Copyright |copy| 2014 Chris Lawrence \n", "description_content_type": "", "docs_url": null, "download_url": "", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/efficiosoft/btrfs-backup", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "btrfs-backup", "package_url": "https://pypi.org/project/btrfs-backup/", "platform": "", "project_url": "https://pypi.org/project/btrfs-backup/", "project_urls": { "Homepage": "https://github.com/efficiosoft/btrfs-backup" }, "release_url": "https://pypi.org/project/btrfs-backup/0.3.1/", "requires_dist": null, "requires_python": "", "summary": "Intelligent, feature-rich backups for btrfs", "version": "0.3.1" }, "last_serial": 3973160, "releases": { "0.2": [ { "comment_text": "", "digests": { "md5": "855c7c0a597415898b709c4bb53a6dcd", "sha256": "523fb1b94ab8579275915ec9e3904ad4ab73f1aa626d0d2e82b828df5aac2996" }, "downloads": -1, "filename": "btrfs_backup-0.2.tar.gz", "has_sig": false, "md5_digest": "855c7c0a597415898b709c4bb53a6dcd", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 7910, "upload_time": "2017-05-15T15:12:07", "url": "https://files.pythonhosted.org/packages/1c/88/c6ea266acf96a9007c0ccaa524d06248d95083dbecee4715558b2a30c55b/btrfs_backup-0.2.tar.gz" } ], "0.2.1": [ { "comment_text": "", "digests": { "md5": "9fb99c21c0f8cfd2f5486cb68e0de252", "sha256": "d79c3551c59b6a503f362eb8664badab8c4bc79ddd0729b02444b1bf9ec38dc8" }, "downloads": -1, "filename": "btrfs_backup-0.2.1.tar.gz", "has_sig": false, "md5_digest": "9fb99c21c0f8cfd2f5486cb68e0de252", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 10738, "upload_time": "2017-05-19T11:32:20", "url": "https://files.pythonhosted.org/packages/93/89/8c20a164b3d4cd8fd917ce672254efdce3c9b97d2ef076d441d9e4066e42/btrfs_backup-0.2.1.tar.gz" } ], "0.3.0": [ { "comment_text": "", "digests": { "md5": "3072109640539581c1201fd7d33e4818", "sha256": "204aba5d29a0db282e918be38c785b1fbb181f1a867ec5bc3a516675bde16027" }, "downloads": -1, "filename": "btrfs_backup-0.3.0.tar.gz", "has_sig": false, "md5_digest": "3072109640539581c1201fd7d33e4818", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23957, "upload_time": "2018-01-26T10:01:20", "url": "https://files.pythonhosted.org/packages/79/a9/57e8caa9b40934fec9c5f70143d5523d5df538422b083c244121566cefa6/btrfs_backup-0.3.0.tar.gz" } ], "0.3.1": [ { "comment_text": "", "digests": { "md5": "72ad0b28990bb03ea312602ca46b42e5", "sha256": "2063d5e75166b692fae351e4ef50634b8ee0f176bac60587f6057980f6285d89" }, "downloads": -1, "filename": "btrfs_backup-0.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "72ad0b28990bb03ea312602ca46b42e5", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 30461, "upload_time": "2018-06-18T11:31:14", "url": "https://files.pythonhosted.org/packages/d0/16/da09416b6b451003422ab482b119696a9f140161376f20c389ddedd24529/btrfs_backup-0.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "30d6e1d7173ca26979abe21cc4da0ad0", "sha256": "0c4a022aa62b15b8485055fb4719a2a3b6d91c93b3b72fde3099fd5fd48142f2" }, "downloads": -1, "filename": "btrfs_backup-0.3.1.tar.gz", "has_sig": false, "md5_digest": "30d6e1d7173ca26979abe21cc4da0ad0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23992, "upload_time": "2018-04-17T15:20:54", "url": "https://files.pythonhosted.org/packages/c8/a6/2d6107fe0ddd2157bc13510817efe2185bc812893beeaead69c79fcc0ca4/btrfs_backup-0.3.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "72ad0b28990bb03ea312602ca46b42e5", "sha256": "2063d5e75166b692fae351e4ef50634b8ee0f176bac60587f6057980f6285d89" }, "downloads": -1, "filename": "btrfs_backup-0.3.1-py3-none-any.whl", "has_sig": false, "md5_digest": "72ad0b28990bb03ea312602ca46b42e5", "packagetype": "bdist_wheel", "python_version": "3.5", "requires_python": null, "size": 30461, "upload_time": "2018-06-18T11:31:14", "url": "https://files.pythonhosted.org/packages/d0/16/da09416b6b451003422ab482b119696a9f140161376f20c389ddedd24529/btrfs_backup-0.3.1-py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "30d6e1d7173ca26979abe21cc4da0ad0", "sha256": "0c4a022aa62b15b8485055fb4719a2a3b6d91c93b3b72fde3099fd5fd48142f2" }, "downloads": -1, "filename": "btrfs_backup-0.3.1.tar.gz", "has_sig": false, "md5_digest": "30d6e1d7173ca26979abe21cc4da0ad0", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 23992, "upload_time": "2018-04-17T15:20:54", "url": "https://files.pythonhosted.org/packages/c8/a6/2d6107fe0ddd2157bc13510817efe2185bc812893beeaead69c79fcc0ca4/btrfs_backup-0.3.1.tar.gz" } ] }