{ "info": { "author": "Jan-Philip Gehrcke", "author_email": "jgehrcke@googlemail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: End Users/Desktop", "Intended Audience :: System Administrators", "License :: OSI Approved :: MIT License", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Topic :: System :: Archiving :: Backup", "Topic :: System :: Filesystems", "Topic :: System :: Systems Administration", "Topic :: Utilities" ], "description": "timegaps\n========\n\nTimegaps is a cross-platform command line program. It sorts a set of items into\n*rejected* and *accepted* ones, based on the age of each item and user-given\ntime categorization rules.\n\nTimegaps allows for thinning out a collection of items, whereas the *time gaps*\nbetween accepted items become larger with increasing age of items. This is\nuseful for implementing *backup retention policies* with the goal to keep\nbackups \"logarithmically\" distributed in time, e.g. one for each of the last 24\nhours, one for each of the last 30 days, one for each of the last 8 weeks, and\nso on.\n\nTimegaps is built with a focus on reliability. It is backed by a considerable\nset of unit tests, including direct command line interface tests. Currently,\neach commit is `automatically tested `_\nagainst CPython 2.7/3.3/3.4 on Linux via Travis CI. Releases are tested on Linux\nas well as on Windows. Simplicity and compliance with the `Unix philosophy\n`_ are the major design goals of\ntimegaps. Version tags follow the concept of semantic versioning.\n\n\nRequirements\n------------\n\nTimegaps requires `Python `_. Releases are tested on CPython\n2.7/3.3/3.4, on Linux as well as on Windows. This is where you can expect it to\nrun properly.\n\n\nInstallation\n------------\n\nInstallation via `pip `_ is\nrecommended::\n\n $ pip install timegaps\n\nThis downloads the latest timegaps releases `from PyPI\n`_ and installs it. A previously\ninstalled version can be upgraded via::\n\n $ pip install --upgrade timegaps\n\nThis is how to install the latest development version::\n\n $ pip install git+https://github.com/jgehrcke/timegaps\n\n\nDocumentation and changelog\n---------------------------\n\n- Docs and resources: the official home of this program is\n http://gehrcke.de/timegaps. The documentation consists of this ``README``,\n ``timegaps --help``, and ``timegaps --extended-help``.\n- `Changelog `_.\n\n\nHands-on introduction\n---------------------\n\nConsider the following situation: all ``*.tar.gz`` files in the current working\ndirectory happen to be daily snapshots of something. The task is to accept one\nsnapshot for each of the last 20 days, one for each for the last 8 weeks, and\none for each of the last 12 months, and to *reject all others*. Use timegaps for\nperforming this categorization into rejected and accepted items and print the\nrejected ones::\n\n $ timegaps days20,weeks8,months12 *.tar.gz | sort\n daily-2013-09-17-133413.tar.gz\n [...]\n daily-2014-02-27-070001.tar.gz\n\nThis was a read-only, non-invasive operation. By default, timegaps prints the\nrejected items to stdout, separated by newline characters (for compatibility\nwith other Unix command line tools). Repeat the operation and count the rejected\nitems::\n\n $ timegaps days20,weeks8,months12 *.tar.gz | wc -l\n 125\n\nGiven this specific set of rules and set of items, timegaps identified 125 items\nto be rejected. Move them to the directory ``notneededanymore`` (and suppress\nstdout)::\n\n $ mkdir notneededanymore\n $ timegaps --move notneededanymore days20,weeks8,months12 *.tar.gz > /dev/null\n\nCount files in the newly created directory for validation purposes (must also be\n125)::\n\n $ /bin/ls -1 notneededanymore/* | wc -l\n 125\n\nOkay, so far the item modification time was determined from the inode via the\n``stat()`` system call. In a different mode of operation (``--time-from-\nbasename``), timegaps can read the \"modification time\" from the basename. The\nfile names of the tarred snapshots in this hands-on session carry meaningful\ntime information, in a certain format (``daily-%Y-%m-%d-%H%M%S.tar.gz``).\nProviding this format string, we can instruct timegaps to parse the time from\nthese file names::\n\n $ mv notneededanymore/* .\n $ timegaps --time-from-basename daily-%Y-%m-%d-%H%M%S.tar.gz \\\n days20,weeks8,months12 *.tar.gz | wc -l\n 125\n\nThe above can be useful in cases where the actual file modification time is\nscrewed, and the real timing information is only contained in the file name. In\nanother mode of operation (``--stdin``), timegaps can read newline-separated\nitems from stdin, instead of reading items from the command line::\n\n $ /bin/ls -1 *tar.gz | timegaps --stdin days20,weeks8,months12 | wc -l\n 125\n\nGiven ``-0/--nullsep``, timegaps can handle NUL character-separated items on\nstdin. In this mode of operation, timegaps also NUL-separates the items on\nstdout::\n\n $ find . -name \"*tar.gz\" -print0 | \\\n timegaps -0 --stdin days20,weeks8,months12 | \\\n tr '\\0' '\\n' | wc -l\n 125\n\nBy default, the reference time for determining the age of items is the time of\nprogram invocation. Use ``-t/--reference-time`` for changing the reference time\nfrom *now* to an arbitrary date (January 1st, 2020 in this case)::\n\n $ timegaps --reference-time 20200101-000000 years10 *.tar.gz | wc -l\n 153\n\nWith a different reference time and different rules the number of rejected items\nobviously changed (from 125 to 153). Instead of printing the rejected items,\ntimegaps can invert the output and print the accepted ones::\n\n $ timegaps -a -t 20200101-000000 years10 *.tar.gz\n daily-2014-02-27-070001.tar.gz\n daily-2014-01-01-070001.tar.gz\n\nThere are more features, such as deleting files, or a mode in which items are\ntreated as simple strings instead of paths. See the help message::\n\n $ timegaps --help\n usage: timegaps [-h] [--extended-help] [--version] [-s] [-0] [-a] [-t TIME]\n [--time-from-basename FMT | --time-from-string FMT]\n [-d | -m DIR] [-r] [-v]\n RULES [ITEM [ITEM ...]]\n\n Accept or reject items based on age categorization.\n\n positional arguments:\n RULES A string defining the categorization rules. Must be of\n the form [,[,\n ... ]]. Example: 'recent5,days12,months5'. Valid\n values: years, months, weeks, days, hours,\n recent. Valid values: positive integers.\n Default maxcount for unspecified categories: 0.\n ITEM Treated as path to file system entry (default) or as\n string (--time-from-string mode). Must be omitted in\n --stdin mode. Warning: duplicate items are treated\n independently.\n\n optional arguments:\n -h, --help Show help message and exit.\n --extended-help Show extended help message and exit.\n --version Show version information and exit.\n -s, --stdin Read items from stdin. The default separator is one\n newline character.\n -0, --nullsep Input and output item separator is NUL character\n instead of newline character.\n -a, --accepted Output accepted items and perform actions on accepted\n items. Overrides default, which is to output rejected\n items (and act on them).\n -t TIME, --reference-time TIME\n Parse reference time from local time string TIME.\n Required format is YYYYmmDD-HHMMSS. Overrides default\n reference time, which is the time of program\n invocation.\n --time-from-basename FMT\n Parse item modification time from the item path\n basename, according to format string FMT (cf. Python's\n strptime() docs at bit.ly/strptime). This overrides\n the default behavior, which is to extract the\n modification time from the inode.\n --time-from-string FMT\n Treat items as strings (do not validate paths). Parse\n time from item string using format string FMT (cf.\n bit.ly/strptime).\n -d, --delete Attempt to delete rejected paths.\n -m DIR, --move DIR Attempt to move rejected paths to directory DIR.\n -r, --recursive-delete\n Enable deletion of non-empty directories.\n -v, --verbose Control verbosity. Can be specified multiple times for\n increasing verbosity level. Levels: error (default),\n info, debug.\n\n Version 0.1.0\n\n\nFor a detailed specification of program behavior and the time categorization\nmethod, please confer ``timegaps --extended-help``.\n\n\nGeneral description\n-------------------\n\nTimegaps' input item set is either provided with command line arguments or read\nfrom stdin. The output is the set of rejected or accepted items, written to\nstdout.\n\nTimegaps by default treats items as paths. It retrieves the modification time\n(``st_mtime``) of the corresponding file system entries via the ``stat`` system\ncall. By default, timegaps works in a non-invasive read-only mode and simply\nlists the rejected (or accepted) items. If explicitly requested, timegaps can\nalso directly delete or move the corresponding file system entries, using well-\nestablished functions from Python's standard ``shutil`` module.\n\nIn a special mode of operation, timegaps can treat items as simple strings\nwithout path validation and extract the \"modification time\" from each string,\naccording to a given time string format. This feature can be used for filtering\nany kind of time-dependent data, but also for filtering e.g. ZFS snapshots.\n\n\nMain motivation\n---------------\n\nThe well-established backup solution `rsnapshot `_\nhas the useful concept of ``hourly / daily / weekly / ...`` snapshots already\nbuilt in and creates such a structure on the fly. Unfortunately, other backup\napproaches often lack such a fine-grained backup retention logic, and people\ntend to hack simple filters themselves. Furthermore, even rsnapshot is not able\nto post-process and thin out an existing set of snapshots. This is where\ntimegaps comes in: you can use the backup solution of your choice for\nperiodically (e.g. hourly) creating a snapshot. You can then \u2014 independently\nand at any time \u2014 process this set of snapshots with timegaps and identify\nthose snapshots that need to be eliminated (removed or displaced) in order to\nmaintain a certain \u201clogarithmic\u201d distribution of snapshots in time. This is the\nmain motivation behind timegaps, but of course you can use it for filtering any\nkind of time-dependent data.\n\n\nHow can the unit tests be run?\n------------------------------\n\nIf you run into troubles with timegaps, or if you want to verify whether it\nproperly runs on your platform, it is a good idea to run the unit test suite\nunder your conditions. Timegaps' unit tests are written for `pytest\n`_. With ``timegaps/test`` being the current working\ndirectory, run the tests like this::\n\n $ py.test -v\n\n\nAuthor & license\n----------------\n\nTimegaps is written and maintained by `Jan-Philip Gehrcke `_.\nIt is licensed under an MIT license (see LICENSE file).", "description_content_type": null, "docs_url": null, "download_url": null, "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "http://gehrcke.de/timegaps", "keywords": "time,categorization,backup,deletion", "license": null, "maintainer": null, "maintainer_email": null, "name": "timegaps", "package_url": "https://pypi.org/project/timegaps/", "platform": "POSIX,Windows", "project_url": "https://pypi.org/project/timegaps/", "project_urls": { "Homepage": "http://gehrcke.de/timegaps" }, "release_url": "https://pypi.org/project/timegaps/0.1.1/", "requires_dist": null, "requires_python": null, "summary": "Accept or reject items based on age categorization.", "version": "0.1.1" }, "last_serial": 1097295, "releases": { "0.1.0": [ { "comment_text": "", "digests": { "md5": "5af9ee3cb1e2f17a2cd8162da465541c", "sha256": "f289f83d4dd4362848f657b737dbc2a265d5b72fd07d969bf87ea16b91405c80" }, "downloads": -1, "filename": "timegaps-0.1.0.zip", "has_sig": false, "md5_digest": "5af9ee3cb1e2f17a2cd8162da465541c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 40465, "upload_time": "2014-03-16T17:50:20", "url": "https://files.pythonhosted.org/packages/ab/19/ba7fdb26c16b18ee620519b3345235897aad8e4865e9b21ee8d9c54e0e22/timegaps-0.1.0.zip" } ], "0.1.1": [ { "comment_text": "", "digests": { "md5": "f3ace6ac2687ba5f9959baba9ced698d", "sha256": "ca591b51137095a16fc8e144dd2d86b0e05525c7c23b5916d32772bf1577c9e9" }, "downloads": -1, "filename": "timegaps-0.1.1.zip", "has_sig": false, "md5_digest": "f3ace6ac2687ba5f9959baba9ced698d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45860, "upload_time": "2014-05-19T15:38:12", "url": "https://files.pythonhosted.org/packages/cd/62/f8ae76f8a3da5661d26eda6da5658c459a1eea344be31f5fc8be262ba317/timegaps-0.1.1.zip" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "f3ace6ac2687ba5f9959baba9ced698d", "sha256": "ca591b51137095a16fc8e144dd2d86b0e05525c7c23b5916d32772bf1577c9e9" }, "downloads": -1, "filename": "timegaps-0.1.1.zip", "has_sig": false, "md5_digest": "f3ace6ac2687ba5f9959baba9ced698d", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 45860, "upload_time": "2014-05-19T15:38:12", "url": "https://files.pythonhosted.org/packages/cd/62/f8ae76f8a3da5661d26eda6da5658c459a1eea344be31f5fc8be262ba317/timegaps-0.1.1.zip" } ] }