{
    "info": {
        "author": "Aidan Heerdegen",
        "author_email": "aidan.heerdegen@anu.edu.au",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 3 - Alpha",
            "Environment :: Console",
            "Intended Audience :: Science/Research",
            "License :: OSI Approved :: Apache Software License",
            "Operating System :: POSIX :: Linux",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3.5"
        ],
        "description": "=============================\nnchash\n=============================\n\nA fast checksum function for netCDF files\n\n.. image:: https://travis-ci.org/aidanheerdegen/nchash.svg?branch=master\n  :target: https://travis-ci.org/aidanheerdegen/nchash\n.. image:: https://circleci.com/gh/aidanheerdegen/nchash.svg?style=shield\n  :target: https://circleci.com/gh/aidanheerdegen/nchash\n.. image:: http://codecov.io/github/aidanheerdegen/nchash/coverage.svg?branch=master\n  :target: http://codecov.io/github/aidanheerdegen/nchash?branch=master\n.. image:: https://landscape.io/github/aidanheerdegen/nchash/master/landscape.svg?style=flat\n  :target: https://landscape.io/github/aidanheerdegen/nchash/master\n.. image:: https://codeclimate.com/github/aidanheerdegen/nchash/badges/gpa.svg\n  :target: https://codeclimate.com/github/aidanheerdegen/nchash\n.. image:: https://badge.fury.io/py/nchash.svg\n  :target: https://pypi.python.org/pypi/nchash\n\n.. content-marker-for-sphinx\n\nA fast hashing function for netCDF files.\n\nTrue hashing or checksum functions take longer the larger the file size, and hence don't scale\nwell for very large files.\n\nThe intention of this code was to create something that would be able to detect changes\nin files as a \"first pass\" in a hierarchy of checksums. Full md5 or sha style checksums would \nstill be required to robustly identify files.\n\nThe specific use case is verifying input files in ocean simulation codes, which are run, checkpointed\nand then re-run in a continuous cycle. This allows for a quick integrity check on resubmission, which \nif failed can trigger a slower but more rigorous full checksum comparison.\n\nThe code hashes a dump of the netCDF header, the size and the modification time of the file\n(internal hdf5 modification time in the case of netCDF4 formatted files).\n\nThis hash is unlikely to be robust over long time periods, as there may well be changes to the \nway ncdump outputs the header. But in the use-case envisaged this would trigger a more exhaustive\ncheck, which if successful, could then regenerate a new nchash for fast checking.\n\nOver short to medium timescales, and certainly within the time taken to re-run the simulation this\nwould be robust to changes. Particularly so for netCDF4 files which have an internal hdf5 modfication\ntime stamp. In addition, netCDF4 files that are compressed will likely change size with even small\nchanges to variables in the file.\n\nTesting on an HPC system with fast disk access hashing is independent of size and takes 0.5-0.8s. Testing\nhas included file sizes up to 100GB.\n\n-------\nInstall\n-------\n\nConda install::\n\n    conda install -c coecms nchash\n\nPip install (into a virtual environment)::\n\n    pip install nchash\n\n---\nUse\n---\n\n-------\nDevelop\n-------\n\nDevelopment install::\n\n    git checkout https://github.com/aidanheerdegen/nchash\n    cd nchash\n    conda env create -f conda/dev-environment.yml\n    source activate nchash-dev\n    pip install -e '.[dev]'\n\nThe `dev-environment.yml` file is for speeding up installs and installing\npackages unavailable on pypi, `requirements.txt` is the source of truth for\ndependencies.\n\nRun tests::\n\n    py.test",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "",
        "keywords": "",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "nchash",
        "package_url": "https://pypi.org/project/nchash/",
        "platform": "",
        "project_url": "https://pypi.org/project/nchash/",
        "project_urls": null,
        "release_url": "https://pypi.org/project/nchash/0.1.5/",
        "requires_dist": null,
        "requires_python": "",
        "summary": "A fast checksum function for netCDF files",
        "version": "0.1.5"
    },
    "last_serial": 3510084,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "892bd3875f66e6748e690f9e1b097967",
                    "sha256": "52a2a9216bf025910d20bce859526868975a8a9ec8267d92514d80e7a0c73071"
                },
                "downloads": -1,
                "filename": "nchash-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "892bd3875f66e6748e690f9e1b097967",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17068,
                "upload_time": "2018-01-12T05:05:58",
                "url": "https://files.pythonhosted.org/packages/b6/32/3efab41f0a6e40483485780099159c7791b03510c6355a51d672a48abdde/nchash-0.1.0.tar.gz"
            }
        ],
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1e69fa39c6ea513fc27f3de2acb548b7",
                    "sha256": "10c8d8416cbb299628a0deafa30c4559ce059f44f9fed88d30a5d29ca1e9c258"
                },
                "downloads": -1,
                "filename": "nchash-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "1e69fa39c6ea513fc27f3de2acb548b7",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 17987,
                "upload_time": "2018-01-19T03:28:09",
                "url": "https://files.pythonhosted.org/packages/e9/71/4b1e35ce1738c4dc73c4fa237ef9a488e8b00fa5ee813328b5a5f9f4e84d/nchash-0.1.3.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d0570629b45e9a51dcc5da69a5726d0f",
                    "sha256": "8005b596e7997f859ee2af10daa91f4f10f045bba36b7a9cd855506f9504a975"
                },
                "downloads": -1,
                "filename": "nchash-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "d0570629b45e9a51dcc5da69a5726d0f",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 18093,
                "upload_time": "2018-01-22T04:36:13",
                "url": "https://files.pythonhosted.org/packages/fb/70/71894bb96e02214c93d7b22a3f56be95eaed3bd6ee64f977c3b22e2e425c/nchash-0.1.5.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d0570629b45e9a51dcc5da69a5726d0f",
                "sha256": "8005b596e7997f859ee2af10daa91f4f10f045bba36b7a9cd855506f9504a975"
            },
            "downloads": -1,
            "filename": "nchash-0.1.5.tar.gz",
            "has_sig": false,
            "md5_digest": "d0570629b45e9a51dcc5da69a5726d0f",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 18093,
            "upload_time": "2018-01-22T04:36:13",
            "url": "https://files.pythonhosted.org/packages/fb/70/71894bb96e02214c93d7b22a3f56be95eaed3bd6ee64f977c3b22e2e425c/nchash-0.1.5.tar.gz"
        }
    ]
}