{ "info": { "author": "Cornelius Grotjahn", "author_email": "s1@tempaddr.uk", "bugtrack_url": null, "classifiers": [ "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3 :: Only", "Topic :: System :: Archiving :: Backup" ], "description": "ABuS is a script for backing up (and restoring) your files to a local disk.\n\nThe backups are encrypted, compressed, and deduplicated.\nIt is assumed that another program (e.g. rsync) is used to make off-site\ncopies of the backups (see below).\n\nContent of this document:\n\n- Caveats_\n- Installation_\n- Documentation_\n\n - Purging_\n - Restoring_\n - `Off-site copies`_\n - `Index Database`_\n - `Configuration file`_\n - `Command line switches`_\n- `History`_\n\n\n=======\nCaveats\n=======\n\nABuS only works on Windows.\n\nABuS only backs up file content. In particular the backups do not\ninclude permissions, symbolic links, hard links, or special files.\n\nIf you use ABuS in anger (inspite of the lack of guarantees in the licence),\nplease pay particular attention to what the documentation below says about\n\n- off-site backups\n- the ``password`` option\n\n\n============\nInstallation\n============\n\n1. Install Python 3.6 from python.org\n\n - include pip\n - it helps to add python to path\n\n2. From the command line, \"as administrator\" if python has been\n installed \"for all users\"::\n\n c:\\path\\to\\python36\\scripts\\pip install abus\n\n3. Create minimal config file, e.g.::\n\n logfile c:/my/home/abus.log\n archive e:/backups\n password password1234 just kidding!\n [include]\n c:/my/home\n\n4. Initialise the backup directory and the index database with::\n\n c:\\path\\to\\python36\\scripts\\abus.exe -f c:/my/home/abus.cfg --init\n\n5. Add to Task Scheduler::\n\n c:\\path\\to\\python36\\scripts\\abus.exe -f c:/my/home/abus.cfg --backup\n\n If there are any problems that prevent ABuS from getting as far as opening\n the log file (and Windows permissions can cause many such problems), then\n use cmd.exe to allow redirection::\n\n cmd /c\n c:\\path\\to\\python36\\scripts\\abus.exe -f c:/my/home/abus.cfg --backup\n >c:\\abus.err 2>&1\n\n\n=============\nDocumentation\n=============\n\nOverview\n++++++++\n\nABuS is a single script for handling backups. Its command line parameters\ndetermine whether the backups are to be created, listed, or restored. The\nbackups are stored in subdirectories of the *backup directory* which must be on\na local filesystem. For off-site copies another program is to be used, for\nexample rsync.\n\n**Warning:** Off-site copies must be made correctly to minimise the risk of\npropagating any local corruption (see below).\n\nA configuration file is used to point to the backup directory, define the backup\nset, and some options. ABuS finds the configuration file either via a command\nline parameter or an environment variable.\n\n\nPurging\n+++++++\n\nOld backup files are deleted after every backup.\nIn order to determine which backups are deleted, time is divided into slots and\nonly the latest version of a file in each slot is retained while the others are\nsubject to purging. As slots get old they are combined into bigger slots.\n\nThe configuration file defines the slot sizes using *freq*/*age* pairs of\nnumbers, which define that 1 version in *freq* days is to be retained for\nbackups up to *age* days old.\n\nFor example, if the retention values are 1 7, 7 30, 28 150, then\nfor each file one version a day is kept from the versions that are up\nto 7 days old, one a week is kept for versions up to 30 days old, and one every\nfour weeks is kept up to 150 days.\n\nThere is also a single slot older than the highest *age* defined, called \"slot 0\".\nIn the example above one file older than 150 days will be kept as well.\n\nPurging of deleted files\n------------------------\n\nThe time that a file deletion is detected\n(i.e. a file previously backed up no longer exists)\nmust fall into slot 0\nbefore the last backup of the file is purged.\nE.g. with default retention values,\n150 days after a file is deleted its backups will be purged.\n\n\nRestoring\n+++++++++\n\nBackup files can be restored from the backups using the `--restore` command line option.\n\nBy default the backups to be restored are the latest version of each known file.\nThe set of files can be restricted using \"glob\" positional arguments. As for\nexclusions, a `*` matches the directory separator. A backup is restored if its\npath matches any of the glob arguments.\nSlashes and backslashes can be used interchangeably.\n\nWith the `-d` option the latest version of each backup before the given time is\nrestored rather than the latest version before now.\nWith `-a` all versions (before the cut-off time) are restored and a timestamp is\nadded to the restored files' names.\n\nAfter the set of restore files has been determined, ABuS removes the common part\nof their paths and creates the remaining relative paths in the current\ndirectory. E.g. if these the files were to be restored::\n\n c:/home/project/file_a\n c:/home/project/src/file_b\n c:/home/project/src/file_c\n\nThen they would be restored as::\n\n ./file_a\n ./src/file_b\n ./src/file_c\n\nDeletions\n---------\n\nFiles that have been deleted at the cut-off time are not restored.\nNote, however, that ABuS does not track historic deletions; for example, assume\na certain file was last changed on Monday, deleted on Tuesday, and recreated on\nWednesday. A restore with an end-of-Tuesday cut-off would restore the Monday\nversion.\n\nListing\n-------\n\nThe `--listing` option lists backed up files. It takes the same options as\n`--restore` and lists exactly those backup versions that would be restored.\n\nThe `--listing` option is implied if any of the restore filters are used without a\n`--restore`.\n\n\nOff-site copies\n+++++++++++++++\n\nABuS only backs up to local filesystems. This means that the backups themselves\nare at risk of corruption, for example from ramsomware. It is important that\nanother copy of the backup is made and that it fulfills these criteria:\n\n* It must not be on a locally accessible filesystem or network share, so that\n the machine being backed up cannot corrupt it.\n* Files must never be overwritten, once created, so that any local corruption\n does not propagate.\n* As a consequence, partially transferred files must be removed at the\n destination.\n\nThe following is an example of an rsync command that would copy the local\nbackup directory to an off-site location::\n\n rsync --recursive --ignore-existing \\\n --exclude index.sl3 --exclude '*.part' \\\n /my/local/backups/ me@offsite:/backups/\n\n``index.sl3`` need not be transferred because it changes and it can be rebuilt\nfrom the static files. Files with ``.part`` extension are backup files that are\ncurrently being written and will be renamed once complete. Excluding them\nensures that incomplete backup files are not transferred.\n\nOff-site purging\n----------------\n\nSince it is not advisable to propagate changed files - and therefor deletions -\nto the off-site copy of the backup files, these must be purged independently.\n\nTo that end ABuS creates a *content file* in the backup directory which lists\nall backup files. The content file is compressed with gzip and its file name is\nthat of the last backup run with a .gz\nextension. When such a file is written, the previous one is\nremoved. Since the run names are basically ISO dates, a script on the off-site\nserver can easily pick up the latest and remove all backup files that are not\nlisted in it.\n\n**N.B.:** The following is only an outline of such a script to convey the idea.\nYou must not use it without checking it first::\n\n cd .../offsite-copy\n keep_list=$(ls *.gz | tail -n 1)\n (find -type f -printf '%P\\n'; zcat $keep_list $keep_list) | sort | uniq -u >/tmp/remove\n [[ $(wc -l /tmp/remove) -lt 50 ]] || exit # sanity check\n xargs rm