{
    "info": {
        "author": "Martin Matusiak",
        "author_email": "numerodix@gmail.com",
        "bugtrack_url": null,
        "classifiers": [
            "License :: OSI Approved :: GNU General Public License v2 (GPLv2)",
            "Programming Language :: Python :: 2.6",
            "Programming Language :: Python :: 2.7",
            "Programming Language :: Python :: 3",
            "Programming Language :: Python :: 3.2",
            "Programming Language :: Python :: 3.3",
            "Programming Language :: Python :: 3.4",
            "Programming Language :: Python :: Implementation :: CPython",
            "Programming Language :: Python :: Implementation :: PyPy"
        ],
        "description": "killdupes\n=========\n\n.. image:: https://badge.fury.io/py/killdupes.png\n        :target: https://badge.fury.io/py/killdupes\n\n.. image:: https://travis-ci.org/numerodix/killdupes.png?branch=master\n    :target: https://travis-ci.org/numerodix/killdupes\n\n.. image:: https://pypip.in/wheel/killdupes/badge.png\n    :target: https://pypi.python.org/pypi/killdupes/\n\n.. image:: https://pypip.in/license/killdupes/badge.png\n        :target: https://pypi.python.org/pypi/killdupes/\n\n\nPython version support: CPython 2.6, 2.7, 3.2, 3.3, 3.4 and PyPy.\n\n\nHow it works\n------------\n\n``killdupes`` scans your filesystem to find duplicate files, partial files\nand empty files.\n\nIt performs n:n comparison of files through md5 hashing and heavy use of\ndictionaries.\nExecute with wildcard, or input file containing file names to check.\n\nThe method:\n\n1. Scan all files, find the smallest.\n2. Read ``read size`` amount of bytes (equal to the remaining size of the\n   smallest file, or at most ``CHUNK`` size) from all files into ``records``.\n3. Hash all records, use hashes as keys into ``offsets[current_offset]`` dict.\n4. Files in the same bucket are known to be equal up to this offset.\n5. Continue until at least two files remain that are still equal at all\n   offsets.\n6. Equal files are either a duplicate case (if they are the same size), or\n   one is partial relative to the other (if not the same size).\n\nMemory consumption should not exceed ``files_in_bucket * read_size``.\n\nThe algorithm adapts to file changes; it will read all files until eof regardless\nof the filesize as recorded at startup.\n\n\nInstallation\n------------\n\n.. code:: bash\n\n    $ pip install killdupes\n\n\nUsage\n-----\n\n.. code:: bash\n\n    $ killdupes.py 'tests/samples/*'\n    Empty files:\n    X    0.0  B  14.03.14 17:39:48  tests/samples/empty\n    Incompletes:\n    =    2.0  B  14.03.14 18:17:43  tests/samples/full\n    X    1.0  B  14.03.14 18:17:26  tests/samples/partial\n    Duplicates:\n    =    2.0  B  14.03.14 18:17:43  tests/samples/full\n    X    2.0  B  14.03.14 18:17:37  tests/samples/full2\n    Kill files? (all/empty/incompletes/duplicates) [a/e/i/d/N]\n\nIf there are many files to scan it will display a progress dashboard while\nworking:\n\n.. code:: bash\n\n    176.1 KB | Offs   0.0  B | Buck 1/1 | File 193868/600084 | Rs   1.0  B\n\nThe dashboard fields:\n\n1. Total bytes read\n2. Current offset of reading\n3. Current number of buckets\n4. File/files in this bucket\n5. Readsize at this offset\n",
        "description_content_type": null,
        "docs_url": null,
        "download_url": "UNKNOWN",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "https://github.com/numerodix/killdupes",
        "keywords": null,
        "license": "UNKNOWN",
        "maintainer": null,
        "maintainer_email": null,
        "name": "killdupes",
        "package_url": "https://pypi.org/project/killdupes/",
        "platform": "UNKNOWN",
        "project_url": "https://pypi.org/project/killdupes/",
        "project_urls": {
            "Download": "UNKNOWN",
            "Homepage": "https://github.com/numerodix/killdupes"
        },
        "release_url": "https://pypi.org/project/killdupes/0.1.6/",
        "requires_dist": null,
        "requires_python": null,
        "summary": "Kill duplicate files, finding partial files as well",
        "version": "0.1.6"
    },
    "last_serial": 1032869,
    "releases": {
        "0.1.0": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "83ee02c88f32f000bf33f3c77357a3f4",
                    "sha256": "bce50fcbbe8969f4f7e062edc4198d4cb8028cdcb6fd5501313e5f70925c77fc"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.0-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "83ee02c88f32f000bf33f3c77357a3f4",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 7376,
                "upload_time": "2014-03-13T22:37:42",
                "url": "https://files.pythonhosted.org/packages/c9/b7/5d3c6ca57ba61d3db4b8422aacb38ec8669d729c19eb0f818463b5ac29a1/killdupes-0.1.0-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "ef4f1ae87eb8752976be8c86dfb19451",
                    "sha256": "48d4ad1a8e5eb3461efd099d1a9dedfb304c298c261e3d3633aa88023333d696"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.0.tar.gz",
                "has_sig": false,
                "md5_digest": "ef4f1ae87eb8752976be8c86dfb19451",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4291,
                "upload_time": "2014-03-13T22:37:39",
                "url": "https://files.pythonhosted.org/packages/6d/f4/956a84a8cd216d3af8b6476012242c1fbf9e02dbad5b135977241cb7b000/killdupes-0.1.0.tar.gz"
            }
        ],
        "0.1.1": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "f1bb3dee318f72caa702af32614de4a1",
                    "sha256": "27d8995b8164825d747b649a1044f7163debeb6bb8318d817fce6a6ced8cb889"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.1-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "f1bb3dee318f72caa702af32614de4a1",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 8056,
                "upload_time": "2014-03-14T20:10:58",
                "url": "https://files.pythonhosted.org/packages/78/1f/8fa00729d183f055731f0f481ecf528bcdf49f6063ad841c3b7a60576c2e/killdupes-0.1.1-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "be61922f27e99f75038951b226b5c0fa",
                    "sha256": "3f3e921a3229d5a3d6832c03e0f5a60f9dfb5735c9e7bb3d6065fc46450de4c2"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.1.tar.gz",
                "has_sig": false,
                "md5_digest": "be61922f27e99f75038951b226b5c0fa",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4849,
                "upload_time": "2014-03-14T20:10:56",
                "url": "https://files.pythonhosted.org/packages/82/18/e045849efffdb82f0c5e981466b263b3c3a01e28088590a4aa2eaf94d2bb/killdupes-0.1.1.tar.gz"
            }
        ],
        "0.1.2": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1eefa68b2734ace4a404c7c83b7a0012",
                    "sha256": "d147631835f4116eacd310994d70c3b5237efa17619205ebda3949b163782235"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.2-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1eefa68b2734ace4a404c7c83b7a0012",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 8070,
                "upload_time": "2014-03-14T20:32:45",
                "url": "https://files.pythonhosted.org/packages/e9/33/1518080a14d46015191ebd987e37c8a5cb3f45df0688eeb217b8145d2b29/killdupes-0.1.2-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "155ca37fc844e4f04b55ec8e1d5dbb08",
                    "sha256": "69b79efa49c72908bb77c5f17936d00f7c96d1ece9a10361f0d32e9fdbdaeac4"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.2.tar.gz",
                "has_sig": false,
                "md5_digest": "155ca37fc844e4f04b55ec8e1d5dbb08",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4868,
                "upload_time": "2014-03-14T20:32:43",
                "url": "https://files.pythonhosted.org/packages/f1/d9/58221dd6a6aaa08a6c490f010108db6d716be441fcac611d254fc20adf97/killdupes-0.1.2.tar.gz"
            }
        ],
        "0.1.3": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "73fd935ebf6c0b0aa188789129a5ff35",
                    "sha256": "ffaf580f3cd97a8f94c2b82a82fb3610e7fe7dab3e2d4a1c030b7d0a30c6e6af"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.3-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "73fd935ebf6c0b0aa188789129a5ff35",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 8152,
                "upload_time": "2014-03-14T20:49:58",
                "url": "https://files.pythonhosted.org/packages/42/b1/bfd3d14116ea10520f79b096822108aacf00a0a02c03b98e8e075d34130a/killdupes-0.1.3-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "3c303d69f550a4a96d045cea9d42af70",
                    "sha256": "4d33da1b672afdd0b09855aa36794c1bf0ce07b6d68d167920283696a1a8c382"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.3.tar.gz",
                "has_sig": false,
                "md5_digest": "3c303d69f550a4a96d045cea9d42af70",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4910,
                "upload_time": "2014-03-14T20:49:55",
                "url": "https://files.pythonhosted.org/packages/03/16/5f2b57f028ecc4ff64d6f7f07cb301688084dc4e167d324d95d9b5fa693f/killdupes-0.1.3.tar.gz"
            }
        ],
        "0.1.4": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "1443451d1884df5ddb57b1b4dcab832a",
                    "sha256": "176480c1cbb40db9a1678a7b967ea9b369dbce56281598cc40ff604356717d21"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.4-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "1443451d1884df5ddb57b1b4dcab832a",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 8081,
                "upload_time": "2014-03-17T20:33:15",
                "url": "https://files.pythonhosted.org/packages/d8/05/348e215b705a27562da4679ee0efd08be4096e0dfca8d27e2fc699895491/killdupes-0.1.4-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "c1d98466a7c96e954cbc2be49755309b",
                    "sha256": "2ae4c387a7411b1682f94612f82632ea2e87df10dfb67c54cca48e3fca0c02e4"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.4.tar.gz",
                "has_sig": false,
                "md5_digest": "c1d98466a7c96e954cbc2be49755309b",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4894,
                "upload_time": "2014-03-17T20:33:12",
                "url": "https://files.pythonhosted.org/packages/e5/7a/4b276907257dc787f01d651b7a61547b44017a72e2a3450432dc9b6a13ed/killdupes-0.1.4.tar.gz"
            }
        ],
        "0.1.5": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "68d9e2d97662250d441e5bb4d796cd6a",
                    "sha256": "15cbfc82b9072bb9d98bf028464abb83b667ae56a61e32a9893f87f800a856a6"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.5-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "68d9e2d97662250d441e5bb4d796cd6a",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 8081,
                "upload_time": "2014-03-17T20:34:34",
                "url": "https://files.pythonhosted.org/packages/43/72/a6a2c299c2c2ad2b9ce01f853a23ccef4c02f2a5c6cc622c103d4e0c0d18/killdupes-0.1.5-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "00a5a718664f8be54e93c9606022e877",
                    "sha256": "45c844fc256fb8c117501e1ba1666549b5a4af5c408880f4836017a57b989c38"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.5.tar.gz",
                "has_sig": false,
                "md5_digest": "00a5a718664f8be54e93c9606022e877",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4895,
                "upload_time": "2014-03-17T20:34:31",
                "url": "https://files.pythonhosted.org/packages/3c/ce/a4d14a6d140c5e46b9b13bff5509803b4515133d57187e7451af45361411/killdupes-0.1.5.tar.gz"
            }
        ],
        "0.1.6": [
            {
                "comment_text": "",
                "digests": {
                    "md5": "d689e663ab1444cb8170226bafdba356",
                    "sha256": "195ce0818e0154e69a95b802f36c19c21896c546825bd2bb47d4fa03fdcd6a49"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.6-py2.py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "d689e663ab1444cb8170226bafdba356",
                "packagetype": "bdist_wheel",
                "python_version": "2.7",
                "requires_python": null,
                "size": 8091,
                "upload_time": "2014-03-17T20:36:45",
                "url": "https://files.pythonhosted.org/packages/50/3b/feb2b7a08063e1ed04251a8184701654b4acdcb2e5128d26433d835111be/killdupes-0.1.6-py2.py3-none-any.whl"
            },
            {
                "comment_text": "",
                "digests": {
                    "md5": "7803d4d074f72b782d54f08e7332384e",
                    "sha256": "0fb441443fa23cd3087478e4b492afe2fb987a1d437a24613c6f4875a0ad0684"
                },
                "downloads": -1,
                "filename": "killdupes-0.1.6.tar.gz",
                "has_sig": false,
                "md5_digest": "7803d4d074f72b782d54f08e7332384e",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 4913,
                "upload_time": "2014-03-17T20:36:42",
                "url": "https://files.pythonhosted.org/packages/70/30/c9ab49a0add3006b0af4a9ee09e3143ea75496715e4893315d6f0f4ac3c0/killdupes-0.1.6.tar.gz"
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "d689e663ab1444cb8170226bafdba356",
                "sha256": "195ce0818e0154e69a95b802f36c19c21896c546825bd2bb47d4fa03fdcd6a49"
            },
            "downloads": -1,
            "filename": "killdupes-0.1.6-py2.py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d689e663ab1444cb8170226bafdba356",
            "packagetype": "bdist_wheel",
            "python_version": "2.7",
            "requires_python": null,
            "size": 8091,
            "upload_time": "2014-03-17T20:36:45",
            "url": "https://files.pythonhosted.org/packages/50/3b/feb2b7a08063e1ed04251a8184701654b4acdcb2e5128d26433d835111be/killdupes-0.1.6-py2.py3-none-any.whl"
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "7803d4d074f72b782d54f08e7332384e",
                "sha256": "0fb441443fa23cd3087478e4b492afe2fb987a1d437a24613c6f4875a0ad0684"
            },
            "downloads": -1,
            "filename": "killdupes-0.1.6.tar.gz",
            "has_sig": false,
            "md5_digest": "7803d4d074f72b782d54f08e7332384e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 4913,
            "upload_time": "2014-03-17T20:36:42",
            "url": "https://files.pythonhosted.org/packages/70/30/c9ab49a0add3006b0af4a9ee09e3143ea75496715e4893315d6f0f4ac3c0/killdupes-0.1.6.tar.gz"
        }
    ]
}