{
    "info": {
        "author": "Nathan Wilcox",
        "author_email": "nejucomo@gmail.com",
        "bugtrack_url": null,
        "classifiers": [],
        "description": "chkfs - A commandline tool for storing filesystems inside a chkstore.\n\n\nThis stores a filesystem using the chkstore library.  Use case goals include:\n\n* Backup many different old hard drives with redundant copies of\n  filesystems in a deduplicating manner.\n\n* Store in a self-describing transparent format, so that if a user finds\n  themselves with a typical fresh linux install but not network and no\n  access to this code, they can still restore backups using bzip2, cat,\n  cp, etc...\n\n* Incremental backup with atomically consistent cached progress state:\n  If a backup process dies, it can be restarted and catch up to its\n  previous run without using heavy resources.\n\n  - Atomically consistent means a backup process can die suddenly at\n    any step without corrupting the store.\n\n  - Consistency *also* anticipates multiple writing processes can\n    update the storage simultaneously without a loss of consistency.\n    The only failure in this case is to overwrite a \"snapshot pointer\".\n    Dangling snapshot pointers can be reconstructed with an expensive scan\n    of the store.\n\n  - Cached means the progress tracking state can be removed, and the\n    only effect is that the next backup run will use more disk I/O\n    and time, but will not lose information or revert any committed\n    backup state.\n\n* Support many different backup source filesystems (old dos FAT, iso9660,\n  ntfs...).  Support for reading the filesystems comes from the kernel\n  by dint of mounting, but the backup tool should save all relevant\n  filesystem metadata.\n\n  - This includes filenames in any encoding. The known encodings are\n    ASCII and utf8, but if neither encoding can represent a filename,\n    an \"unknown\" encoding stores the binary data directly.  Encodings are\n    \"sniffed\" by first validating against ASCII, then utf8, then falling\n    back to unknown.  This means the encoding is only a hint, because\n    a non-ASCII or non-UTF8 filename may be misinterpreted as either of\n    those encodings.  However, no data is lost or corrupted.\n\n* Restore portions of the stored data.\n\n  - The stored data can be inspected and restored in a fine-grained\n    manner, such as by retrieving a single file from a large snapshot,\n    or a transitive directory.\n\n* Recursive directory structures.\n\n  - OSX, tahoe-lafs, and some other filesystems allow recursive directory\n    structures.  (In OSX for example, directories may be hard-linked.)\n\n\nUnsupported Use Cases:\n\n* Deletion.  My philosophy is to buy a new hard drive and to save data\n  forever.  There is a security risk, but OTOH, it's impossible to tell\n  how valuable any datum may be in the future.\n\n* Redundancy.  The underlying filesystem or storage drivers can handle\n  this, and it's best to leave that complexity in a different layer.\n\n* High Availability.  If the storage node explodes, all data is lost.\n  To prevent this, delegate to another tool such as tahoe-lafs.\n\n* Privacy.  Delegate to the underlying filesystem.\n\n* Crossing Trust Boundaries.  This is intended for a case where anyone\n  with read access to the store can read everything.  If a user needs\n  privacy within a backup, they could encrypt files before backing up\n  and manage that complexity\n\n* Keeping chkfs storage on \"unusual\" or old filesystems: The design\n  is intended to *store* old filesystem contents, but not to store *on*\n  old filesystems.  In particular, chkstore and chkfs assume directories\n  can hold many, many entries, with names at least around 80 ascii\n  bytes long.  (They also currently assume the storage filesystem\n  supports hardlinks for efficient commits, and O_CREAT|O_EXCL for\n  avoiding multi-process collisions.)\n\n\nFuture use case:\n\n* A read-only fuse interface for convenient restore out of the chkfs.\n\n\nBonus use cases:\n\n* Integration as a backend in other networked/decentralized data stores\n  such as camlistore or tahoe-lafs.\n\n\nFAQ:\n\n* Why not cp -a or cp -r?\n\n  - This is lossy in some ways in which chkfs is not: The vfs metadata\n    about the source is not copied, the source filesystem may have\n    metadata which cannot be stored in the target filesystem (including\n    different filename encoding issues).  chkfs also suffers some of\n    these limitations by relying on the vfs layer for reading source\n    filesystems.  Also it sacrifices the convenient utility of having\n    the backup files available directly as a filesystem (without a\n    fuse interface), so chkfs lose the ability to run find | grep,\n    for instance.\n\n* Why not tar or many of the existing very mature unix backup systems?\n\n  - The \"old school\" solutions I'm aware of do not support all of the\n    use cases above without excessive headache.  The tradeoff is\n    that old-school solutions are well tested in a large variety of\n    circumstances and widely available.\n\n* Why not camlistore, tahoe-lafs, freenet, or decentralized storage\n  tech X?\n\n  - I don't need decentralization for personal backups.  There's no need\n    for networking, redundancy, or trust boundary complexity.  (See the\n    unsupported features section.)\n\n* Why not bup or another scheme which is better at dedup?\n\n  - chkfs prefers a \"fairly transparent\" store, as described above.\n    It should be possible to restore a backup without using this tool but\n    only bzip2, cp, vim, etc...\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://bitbucket.org/nejucomo/chkfs",
        "keywords": null,
        "license": "GPLv3",
        "maintainer": null,
        "maintainer_email": null,
        "name": "chkfs",
        "package_url": "https://pypi.org/project/chkfs/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/chkfs/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://bitbucket.org/nejucomo/chkfs"
        },
        "release_url": "https://pypi.org/project/chkfs/0.990c613daa19960e8a900cbffc5472e44282b6f1/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "A commandline tool for storing filesystems inside a chkstore.",
        "version": "0.990c613daa19960e8a900cbffc5472e44282b6f1"
    },
    "last_serial": 811502,
    "releases": {
        "0.990c613daa19960e8a900cbffc5472e44282b6f1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "0e81b5fe4aabdf5466ff18c6c4509106",
                    "sha256": "f154f0cb92896a50e72a330973de1a306db8cd0e672539eddb568a35e413ea19"
                },
                "downloads": -1,
                "filename": "chkfs-0.990c613daa19960e8a900cbffc5472e44282b6f1.tar.gz",
                "has_sig": true,
                "md5_digest": "0e81b5fe4aabdf5466ff18c6c4509106",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 21343,
                "upload_time": "2013-07-09T01:20:00",
                "url": "https://files.pythonhosted.org/packages/9d/5d/2d05f3250bcb1411918c9825ce713f63a32a52a13de2996aeb40e2e79d57/chkfs-0.990c613daa19960e8a900cbffc5472e44282b6f1.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "0e81b5fe4aabdf5466ff18c6c4509106",
                "sha256": "f154f0cb92896a50e72a330973de1a306db8cd0e672539eddb568a35e413ea19"
            },
            "downloads": -1,
            "filename": "chkfs-0.990c613daa19960e8a900cbffc5472e44282b6f1.tar.gz",
            "has_sig": true,
            "md5_digest": "0e81b5fe4aabdf5466ff18c6c4509106",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 21343,
            "upload_time": "2013-07-09T01:20:00",
            "url": "https://files.pythonhosted.org/packages/9d/5d/2d05f3250bcb1411918c9825ce713f63a32a52a13de2996aeb40e2e79d57/chkfs-0.990c613daa19960e8a900cbffc5472e44282b6f1.tar.gz"
        }
    ]
}