{ "info": { "author": "Andrey Kislyuk", "author_email": "kislyuk@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Operating System :: MacOS :: MacOS X", "Operating System :: POSIX", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Topic :: Software Development :: Libraries :: Python Modules" ], "description": "**WARNING: Alpha code under development**\n\n# XBCF: eXtensible Block Compression Format\n\nXBCF is a container format designed for indexed block compression on Amazon S3 or similar distributed storage systems with high seek latencies. It supports the following features:\n* Configurable compression codec\n* Configurable block encryption codec\n* Seeking support, with granularity equal to the compression block size (1MB by default)\n* Recursive encapsulation of XBCF containers\n * Multiple worker nodes can independently upload parts of the same multi-part upload, each formatted as XBCF\n * At close time, a higher-level XBCF is concatenated from the parts\n\n## Dependencies\nThis proof-of-concept implementation works in Python.\n```\npip install snappy lz4 numpy boto\n```\n\n## Format specification\n* Magic header (8 bytes) == \"XBCF\" (0-padded)\n* Level (8 bytes) (unsigned int; if level is 0, blocks are raw data; otherwise, blocks are themselves XBCF-formatted)\n* Compression method (8 bytes) == \"LZ4\" (0-padded)\n* Reserved (8 bytes) (padding to 32 bytes)\n* Cipher name (16 bytes) == \"AES-CBC\" or \"null\" (0-padded)\n* IV (16 bytes)\n(If cipher != \"null\", all subsequent data is encrypted.)\n* Block size (uncompressed) (unsigned int; 8 bytes); if size is 0, block size is variable and block size table is present\n* Number of blocks (unsigned int; 8 bytes)\n* Reserved (16 bytes) (padding to 32 bytes)\n* Block start offset table (8 bytes * (Number of blocks))\n* If block size is 0:\n * Block size table (uncompressed sizes) (8 bytes * (Number of blocks))\n* Blocks", "description_content_type": null, "docs_url": null, "download_url": "UNKNOWN", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/kislyuk/xbcf", "keywords": null, "license": "Apache Software License", "maintainer": null, "maintainer_email": null, "name": "xbcf", "package_url": "https://pypi.org/project/xbcf/", "platform": "MacOS X,Posix", "project_url": "https://pypi.org/project/xbcf/", "project_urls": { "Download": "UNKNOWN", "Homepage": "https://github.com/kislyuk/xbcf" }, "release_url": "https://pypi.org/project/xbcf/0.0.1/", "requires_dist": null, "requires_python": null, "summary": "eXtensible Block Compression Format", "version": "0.0.1" }, "last_serial": 1075711, "releases": { "0.0.1": [ { "comment_text": "", "digests": { "md5": "aa01c55ea9c785761e4411923d367e38", "sha256": "bcf5f197d9d55b18ab99bfbcbb9b8599e1e5485b9f1c0a77ab533de6e6f90f9b" }, "downloads": -1, "filename": "xbcf-0.0.1.tar.gz", "has_sig": true, "md5_digest": "aa01c55ea9c785761e4411923d367e38", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5931, "upload_time": "2014-04-29T16:51:25", "url": "https://files.pythonhosted.org/packages/38/aa/7c485ca205dfe0b2bbbb4f88ca3f204c2568b28c0325465b7977ce3028bb/xbcf-0.0.1.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "aa01c55ea9c785761e4411923d367e38", "sha256": "bcf5f197d9d55b18ab99bfbcbb9b8599e1e5485b9f1c0a77ab533de6e6f90f9b" }, "downloads": -1, "filename": "xbcf-0.0.1.tar.gz", "has_sig": true, "md5_digest": "aa01c55ea9c785761e4411923d367e38", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 5931, "upload_time": "2014-04-29T16:51:25", "url": "https://files.pythonhosted.org/packages/38/aa/7c485ca205dfe0b2bbbb4f88ca3f204c2568b28c0325465b7977ce3028bb/xbcf-0.0.1.tar.gz" } ] }