{
    "info": {
        "author": "Humphrey Drummond",
        "author_email": "hum3@drummond.info",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Developers",
            "License :: OSI Approved :: MIT License",
            "Natural Language :: English",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.6"
        ],
        "description": "# odarchive\n\nOptical disc archiving - converting file systems to sets of ISO files and back.  It is meant for archiving\ninformation to optical disc for long term data storage.   \n\n\nThis readme is broken into sections:\n\n- Command line usage\n- Technical description\n- Software description\n- Future Enhancements\n- License (MIT)\n\n# Command line usage\n\nSummary of commands, see below for commandline options.    \n\nCommands  | Comment\n----------|---------- \ninit      | Creates a new catalogue \nwrite_iso | write out an iso  \narchive   |\nsegment   | split archive into segments\n\n## odarchive create_db drive_path\n\nCreates a database catalogue.json in current working directory from\nfiles in drive_path\n\n## odarchive plan_iso size\nEnriches the database to plan building ISO with a maximum size of n\nbytes or using\n\n- 'cd'     = 700MB\n- 'dvd'    = 4.7GB\n- 'bd' = 25GB\n- 'bluray' = 25GB\n- 'bd-dl' = 50GB\n- 'bd-xl' = 100GB\n- 'bd-xx' = 128GB\n\n## odarchive num_isos\n*Planned* Return number of isos required\n\n## odarchive create_iso n\nDefault for n is 0 (numbering from Zero)\n\n\n# Technical Description\n\n## Segmentation\nWhen a catalogue is created for the first time it is not segmented.\nSegmenting refers to writing out a single disc which has smaller\ncapacity than the total archive.\n\n## File name\nThis services is planned to work from supplied USB drives.\nThe internal file name is a UDF Abolute Path. It is an absolute path\n  so that if the name of the Data directory changes this won\u00e2\u20ac\u2122t make a\n  difference when combining discs with different naming conventions.\n\nUser case history: I want to backup some critical files first eg my own\nPhotgraphs and video and then other files The deeper the file system on\nthe USB the better the result will be\n\n## disc directory layout\n\n```\n/Data + All data files catalogue.json #\nREADME.MD\n```\n\n## Json file format\n\nJsonFile = HeadingSection \\*FileDefinition\nHeadingSection = Version\nFileDefinition = Hash FileNameList Mtime Size\n\nMax. filename length 255 bytes (path 1023 bytes)\n\nExample::\n```\n    {\n        \"version\": 1,\n        \"files\" :  {\n            \"Hash1\": {\n                filenames : {\n                    \"data/first.html\" : null,\n                    \"data/first_copy.html\" : null,\n                }\n                size : 21,\n                mtime : ??\n            },\n            \"Hash2\": {\n                filenames : {\n                    \"data/second.txt\" : null,\n                }\n            size : 22,\n            mtime : ?? },\n        }\n    }\n```\n# Software Structure\n\n### Object Structure\n\nThe main objects is an Archive object.\n\nThis has two main sub objects:\n\n- a File_db\n- a hash_file_db\n\nThe file_db is a temporary structure which is derived from walking the file system.\nThe hash_file_db is then derived from that by taking the hashes of all those files and then building \na database of each unique file.  This then shows the multiple location of where the file is stored.\n\n### File Structure\nThis is one module: odarchive.\n\n\nThis has three parts:\n\nThis has three top level modules: \n - ``archive.py`` which handles the\narchive \n- ``odarchive_cli.py`` which puts a command line wrapper around\nthe archive code. \n- a ``_version.py`` which holds the version number\nconstants for both pypy and the code\n\nThe archive code is the main code and 4 sub files: - ``consts.py``\nGeneral constants - ``file_db.py`` handles a file database - ``hash_db``\nextends the file database to include a database of hashentries -\n``hash_file_entry.py`` creating hashes for single files\n\nThere is a distinct order in which things must happen:\n\n-Scan the file and build a file database.\n-Create a hash tables\n\n## Unique ID's\n\nThere is a job_id which is created at the start of a job.  This should be unique and last the life\ntime of the job.  There is also a guid with the catalogue, this changes every time it is saved/changed and \nis like the version of the archive.\n\n## Limitations\n\n### Saving an archive\nWhen an archive is saved it is put into a catalogue.json file.  This is only possible once all the hashes\nfor the files have been created as the format is indexed the file hash.\n\nTemporary work can be saved via the save_as_dill method.\n\n## Inspiration\n\nThanks to M Ruffalo of\nhttps://github.com/mruffalo/hash-db/blob/master/hash_db.py for a lot of\ninspiration.\n\n# Future enhancements\n\n\n### convert a json catalogue into a pickle\nCurrently when working on a backup plan the data is pickled from command\nto command. In order to work with existing cataloges this is what you\nneed.\n\n### Make a .exe\nCreate a single exe that will do this job or make an exe that can be\ninstalled via pip.\n\n### Segemented isos into specific size\n\n### Convert catalogue from dictionary to database\nAt the moment the catalogue database is an in memory dictionary. This\nwill limit it to about 2 million files per GB of available memory.\n\n### Make a service with feedback on status\nEg archiving Z drive has taken at least an hour and I don\u00e2\u20ac\u2122t if working\nor crashed.\n\n### Make calculating Hash multi threaded\nMay work faster\n\n### Add size of catalogue to length of reserved space\n\n### Incremental\n- With an old catalogue and drive path create a additional catalog. Can\n  be current point in time, eg so not reference files that were deleted.\n- This also allows rescanning without making any changes. This should\n  also cover the case of multiple single backups being coalesced into a\n  bigger backup. You might in large file format decide to backup a\n  number of individual files first.\n\n### Split large files\n\n### Make a service as a Glacier replacement.\n\nEg rerun and post changes via web\n\n### Make read only cached drive\n\nrobot plus discs 600 discs = 15TB\n\n### Calculate directory size when segmenting\n\nAt the moment a fixed directory size is added to each entry\n\n### get_info\n\nAdd JsON alternative to returned data\n\n### Make Platinum discs\n\nGBP16 a g SG 19 1u 12cm area = 0.012 a 1u coat costs about GBP4 a disc\nHowever if these had a glass substrate they might have a very long life.\nPotentially you could write them naked and then cover them with another\nglass slip. Could even write with an electron beam although might be\nexpensive\n\n### UDF Bridge format\nTo make the discs more compatible add a ISO 9660 directory structure as\nwell as the UDF\n\nIn order to make the discs more compatible both a UDF and a ISO 9660\ndirectory structure are output. There are different levels of ISO 9660\nstructure but I have chosen level 3 as the standard.\n\nThe ISO 9660 leads to complications as it is restricted to shorter\ndirectory names and a maximum directory depth of 7. This means the the\nconversion from UDF directories to ISO 9660 directories is lossy. From\nthis it means that you have to implement a lossless scheme (store the\ndirectory mapping as a dictionary) so that you can do multiple backups\non multiple discs and still have a coherent directory structure.\n\n\n### Adding error correction\nThe main aim of this is to measure the degradation of the storage media and to know when\nthe data needs restoring.  In a DRAM this is done all the time - it should also be done on raid drives\nto scrub the errors.\n\n# Licensing\nUsing an MIT license see LICENSE.\n\n\nV0.0.0 2018-10-04\n=================\n\nFirst release\n\n\n",
        "description_content_type": "text/markdown",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/drummonds/odarchive",
        "keywords": "cdrom,dvd,bdrom,archive,odarchive",
        "license": "MIT license",
        "maintainer": "",
        "maintainer_email": "",
        "name": "odarchive",
        "package_url": "https://pypi.org/project/odarchive/",
        "platform": "",
        "project_url": "https://pypi.org/project/odarchive/",
        "project_urls": {
            "Homepage": "https://github.com/drummonds/odarchive"
        },
        "release_url": "https://pypi.org/project/odarchive/0.1.0/",
        "requires_dist": [
            "python-dateutil"
        ],
        "requires_python": "",
        "summary": "Convert file systems to archiveable ISO files",
        "version": "0.1.0"
    },
    "last_serial": 4525846,
    "releases": {
        "0.0.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "7d9c3584a677f8b8a25d0103be316528",
                    "sha256": "bfed8482607e6ebc3f2dd4cc5987a2a6282518c6426026817dac576e8685aa98"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "7d9c3584a677f8b8a25d0103be316528",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 33988,
                "upload_time": "2018-10-04T10:32:35",
                "url": "https://files.pythonhosted.org/packages/e4/80/c3a7e989f6cc2973151aca0496f7523f9aa93cf01a1ad70ad0abd4280e4a/odarchive-0.0.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8a33358cdb2ae0651bfcd8187043b826",
                    "sha256": "adb4711f98a1918386d32c5ddd82d4551e545f53743de7c31b058d2fda31c97d"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.0.tar.gz",
                "has_sig": false,
                "md5_digest": "8a33358cdb2ae0651bfcd8187043b826",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29206,
                "upload_time": "2018-10-04T10:32:36",
                "url": "https://files.pythonhosted.org/packages/4b/59/3dd50cc8307edd3298234221965dbfb799e6080f9716bfe7196ae4cae8ef/odarchive-0.0.0.tar.gz"
            }
        ],
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1d0cd1a13f44b94c000b37be7269db74",
                    "sha256": "a056ae002cb705bcebd01826895f20c762625bb99eaa3b44e8d72829fe6182a6"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1d0cd1a13f44b94c000b37be7269db74",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 33979,
                "upload_time": "2018-10-04T14:38:56",
                "url": "https://files.pythonhosted.org/packages/6d/e6/5ec7b79e002b8ab12d0cee50901048471eeb94dd41b4e5bab0ad57345c25/odarchive-0.0.1-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "275755723a2128df0fbe390e7f98ecf0",
                    "sha256": "9aaa9a047efbb0520bd814d87c36694346d063f6d54d54d206afd7e04fd04cd4"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "275755723a2128df0fbe390e7f98ecf0",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29205,
                "upload_time": "2018-10-04T14:38:58",
                "url": "https://files.pythonhosted.org/packages/dd/6e/d5655614670c87efedf7355ce971a7791232a1ee9395a06ce3edfa74e6b9/odarchive-0.0.1.tar.gz"
            }
        ],
        "0.0.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "a3939e49cc96cc78b8669e17a1dc9334",
                    "sha256": "1b9ebc0dbbc996ea5a279ab7146df53dad55ac238a68df8e520801f7e7b1ffdc"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.2-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "a3939e49cc96cc78b8669e17a1dc9334",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 34045,
                "upload_time": "2018-10-17T16:07:18",
                "url": "https://files.pythonhosted.org/packages/d7/86/a86a5843a56a708edcc739ec905260ecd8a4a655cfe6bf2e5c93da045395/odarchive-0.0.2-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "8a8cc718137e416eb204598562325021",
                    "sha256": "3f125a7d782383db15d4d6c409c14d0accaf84694c1b588139bc8aa69fc096f1"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.2.tar.gz",
                "has_sig": false,
                "md5_digest": "8a8cc718137e416eb204598562325021",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 29336,
                "upload_time": "2018-10-17T16:07:19",
                "url": "https://files.pythonhosted.org/packages/ac/dc/8ca1e9d7c21cfe579d1da275cca231c331546d444da8818f707b3e8c4ca4/odarchive-0.0.2.tar.gz"
            }
        ],
        "0.0.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "ea49ea56266f9264bdbf2bc372d012b4",
                    "sha256": "268772d0a84c264633fb4441d0ac62e672413ff5ae873e01a7952a346391768f"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.3-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "ea49ea56266f9264bdbf2bc372d012b4",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 35053,
                "upload_time": "2018-11-01T22:19:18",
                "url": "https://files.pythonhosted.org/packages/ac/5f/4109eaa434f631fabef845e88f0e5e54fb38c1be182e9b107b2b931cc30b/odarchive-0.0.3-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "e66781f53b4191d52e51137d87c430f4",
                    "sha256": "799dda1d81c96c4bf46b36917fba685a8dd9e2581f15311174f188072d00c29e"
                },
                "downloads": -1,
                "filename": "odarchive-0.0.3.tar.gz",
                "has_sig": false,
                "md5_digest": "e66781f53b4191d52e51137d87c430f4",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 30208,
                "upload_time": "2018-11-01T22:19:20",
                "url": "https://files.pythonhosted.org/packages/b8/cf/ba67b6d62ec23101505ca102b59fd778eb77411321022f8ed7e0b9940161/odarchive-0.0.3.tar.gz"
            }
        ],
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "fa7c873cc3cb8b6625fd0c5b8cf0fd99",
                    "sha256": "56ad44f04aba6b2e8ebc45f3b3052e9bf16b8e5e54f0ec99d409316c67f6fc9f"
                },
                "downloads": -1,
                "filename": "odarchive-0.1.0-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "fa7c873cc3cb8b6625fd0c5b8cf0fd99",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 36918,
                "upload_time": "2018-11-25T11:28:54",
                "url": "https://files.pythonhosted.org/packages/3d/56/5d5a5456f71cee6c2b1d38331f029f06f4ac075e199b9a9cdca610baa402/odarchive-0.1.0-py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "56f50494b096826b073823152273db7c",
                    "sha256": "87a59c0d30e539f33d78c60a8a5d9b889d24e94d38b30fff8918c3eee0eed60d"
                },
                "downloads": -1,
                "filename": "odarchive-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "56f50494b096826b073823152273db7c",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 31054,
                "upload_time": "2018-11-25T11:28:56",
                "url": "https://files.pythonhosted.org/packages/01/f8/5a332f15287cbb7f3ecc6cfe645f7ac57864035fe100328ebdaf5ff43067/odarchive-0.1.0.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "fa7c873cc3cb8b6625fd0c5b8cf0fd99",
                "sha256": "56ad44f04aba6b2e8ebc45f3b3052e9bf16b8e5e54f0ec99d409316c67f6fc9f"
            },
            "downloads": -1,
            "filename": "odarchive-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "fa7c873cc3cb8b6625fd0c5b8cf0fd99",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 36918,
            "upload_time": "2018-11-25T11:28:54",
            "url": "https://files.pythonhosted.org/packages/3d/56/5d5a5456f71cee6c2b1d38331f029f06f4ac075e199b9a9cdca610baa402/odarchive-0.1.0-py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "56f50494b096826b073823152273db7c",
                "sha256": "87a59c0d30e539f33d78c60a8a5d9b889d24e94d38b30fff8918c3eee0eed60d"
            },
            "downloads": -1,
            "filename": "odarchive-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "56f50494b096826b073823152273db7c",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 31054,
            "upload_time": "2018-11-25T11:28:56",
            "url": "https://files.pythonhosted.org/packages/01/f8/5a332f15287cbb7f3ecc6cfe645f7ac57864035fe100328ebdaf5ff43067/odarchive-0.1.0.tar.gz"
        }
    ]
}