{ "info": { "author": "Noah Gift", "author_email": "noah.gift@gmail.com", "bugtrack_url": null, "classifiers": [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License" ], "description": "Liten: A deduplication command line tool and library\n===============================================================================\n\n:Author: Noah Gift\n:Version: $Revision: 0.1.5 $\n:Copyright: This document has been placed in the public domain.\n\nSummary\n---------\n\nA deduplication command line tool and library. A relatively efficient\nalgorithm based on searching like sized files, and then performing a full md5\nchecksum, is used to determine duplicate files/file objects. Files can be\ndeleted upon discovery, and pattern matching can be used to limit search\nresults. Finally, configuration file use is supported, and there is a\ndeveloping API that lends itself to customization via an ActionsMixin class.\n\n\n\n.. contents::\n\nExample CLI Usage:\n------------------\n\n\nSize:\n~~~~~~~~~~~~~~~~~~~~~~\n\nSearch by size using --size or -s option::\n\n liten.py -s 1 /mnt/raid is equal to liten.py -s 1MB /mnt/raid\n liten.py -s 1bytes /mnt/raid\n liten.py -s 1KB /mnt/raid\n liten.py -s 1MB /mnt/raid\n liten.py -s 1GB /mnt/raid\n liten.py c:\\in d:\\ is equal to liten.py -s 1MB c:\\in d:\\\n\nReport Location:\n~~~~~~~~~~~~~~~~~~~~~~\n\nGenerate custom report path using -r or --report=/tmp/report.txt::\n\n ./liten.py --report=/tmp/test.txt /Users/ngift/Documents\n\nBy default a report will be created in CWD, called LitenDuplicateReport.csv\n\nConfig File:\n~~~~~~~~~~~~~~~~~~~~~~\n\nYou can use a config file in the following format::\n\n [Options]\n path=/tmp\n size=1MB\n pattern=*.m4v\n delete=True\n\n\nYou can call the config file anything and place it anywhere. \n\nHere is an example usage::\n\n ./liten.py --config=myconfig.ini\n\nVerbosity:\n~~~~~~~~~~~~~~~~~~~~~~\n\nAll stdout can be suppressed by using --quiet or -q.\n\nDelete:\n~~~~~~~~~~~~~~~~~~~~~~\n\nBy using --delete the duplicate files will be automatically deleted. The API\nhas support for an interactive mode and a dry-run mode, they have not been\nimplemented in the CLI as of yet.\n\nExample Library/API Usage:\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n >>> Liten = Liten(spath='testData')\n >>> dupeFileOne = 'testData/testDocOne.txt'\n >>> checksumOne = Liten.createChecksum(dupeFileOne)\n >>> dupeFileTwo = 'testData/testDocTwo.txt'\n >>> checksumTwo = Liten.createChecksum(dupeFileTwo)\n >>> nonDupeFile = 'testData/testDocThree_wrong_match.txt'\n >>> checksumThree = Liten.createChecksum(nonDupeFile)\n >>> checksumOne == checksumTwo\n True\n >>> checksumOne == checksumThree\n False\n\nThere is also the concept of an Action, which can be implemented later, that\nwill allow customizable actions to occur upon an a condition that gets defined\nas you walk down a tree of files.\n\nTests:\n~~~~~~~~~~~~~~~~~~~~~~\n * Run Doctests: ./liten -t or --test\n * Run test_liten.py\n * Run test_create_file.py then delete those test files using liten::\n python2.5 liten.py --delete /tmp\n\nDisplay Options:\n---------------------------\n\nStdout:\n~~~~~~~~~~~~~~~~~~~~~~\nstdout will show you duplicate file paths and sizes such as::\n\n Printing dups over 1 MB using md5 checksum: [SIZE] [ORIG] [DUP]\n 7 MB Orig: /Users/ngift/Downloads/bzr-0-2.17.tar\n Dupe: /Users/ngift/Downloads/bzr-0-4.17.tar\n\nReport:\n~~~~~~~~~~~~~~~~~~~~~~\nA report named LitenDuplicateReport.csv will be created in your current working\ndirectory::\n\n Duplicate Version, Path, Size, ModDate\n Original, /Users/ngift/Downloads/bzr-0-2.17.tar, 7 MB, 07/10/2007 01:43:12 AM\n Duplicate, /Users/ngift/Downloads/bzr-0-3.17.tar, 7 MB, 07/10/2007 01:43:27 AM\n\n\nDebug Mode Environmental Variables:\n------------------------------------------------------------------------\n\n* To enable print statement debugging set LITEN_DEBUG to 1\n* To enable pdb break point debugging set LITEN_DEBUG to 2\n* LITEN_DEBUG_MODE = int(os.environ.get('LITEN_DEBUG', 0))\n* Note: When DEBUG MODE is enabled, a message will appear to standard out\n\nQUESTIONS: noah dot gift at gmail.com\n------------------------------------------------------", "description_content_type": null, "docs_url": null, "download_url": "https://github.com/nogibjj/pyli", "downloads": { "last_day": -1, "last_month": -1, "last_week": -1 }, "home_page": "https://github.com/nogibjj/pyli", "keywords": "", "license": "MIT", "maintainer": "", "maintainer_email": "", "name": "liten", "package_url": "https://pypi.org/project/liten/", "platform": "", "project_url": "https://pypi.org/project/liten/", "project_urls": { "Download": "https://github.com/nogibjj/pyli", "Homepage": "https://github.com/nogibjj/pyli" }, "release_url": "https://pypi.org/project/liten/0.1.6/", "requires_dist": null, "requires_python": "", "summary": "a de-duplication command line tool", "version": "0.1.6" }, "last_serial": 2376721, "releases": { "0.1.4.2": [], "0.1.5.0": [], "0.1.6": [ { "comment_text": "", "digests": { "md5": "8b24947e6a1b611f768c14418554351a", "sha256": "26432a0624d9042916091e250e96e01c996806a1364eb84f9b7fe18c5913a348" }, "downloads": -1, "filename": "liten-0.1.6-py2-none-any.whl", "has_sig": false, "md5_digest": "8b24947e6a1b611f768c14418554351a", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 12495, "upload_time": "2016-10-03T00:30:16", "url": "https://files.pythonhosted.org/packages/e3/06/95ed41fde2b38500e8758d46b1de68ce01a3d6bbc401a178beab1bd96c1b/liten-0.1.6-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9a65b60952d5c51a1fbf64e5a0b26f7c", "sha256": "cfa32ef3faaa9ee50cedb4ababf2d04a697916b8df54d8caa86f1e05ea417377" }, "downloads": -1, "filename": "liten-0.1.6.tar.gz", "has_sig": false, "md5_digest": "9a65b60952d5c51a1fbf64e5a0b26f7c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9544, "upload_time": "2016-10-03T00:30:12", "url": "https://files.pythonhosted.org/packages/f5/37/b609912784be69876beb1f386faf7bd5bcaa3d387f226c69267513dd0b49/liten-0.1.6.tar.gz" } ] }, "urls": [ { "comment_text": "", "digests": { "md5": "8b24947e6a1b611f768c14418554351a", "sha256": "26432a0624d9042916091e250e96e01c996806a1364eb84f9b7fe18c5913a348" }, "downloads": -1, "filename": "liten-0.1.6-py2-none-any.whl", "has_sig": false, "md5_digest": "8b24947e6a1b611f768c14418554351a", "packagetype": "bdist_wheel", "python_version": "2.7", "requires_python": null, "size": 12495, "upload_time": "2016-10-03T00:30:16", "url": "https://files.pythonhosted.org/packages/e3/06/95ed41fde2b38500e8758d46b1de68ce01a3d6bbc401a178beab1bd96c1b/liten-0.1.6-py2-none-any.whl" }, { "comment_text": "", "digests": { "md5": "9a65b60952d5c51a1fbf64e5a0b26f7c", "sha256": "cfa32ef3faaa9ee50cedb4ababf2d04a697916b8df54d8caa86f1e05ea417377" }, "downloads": -1, "filename": "liten-0.1.6.tar.gz", "has_sig": false, "md5_digest": "9a65b60952d5c51a1fbf64e5a0b26f7c", "packagetype": "sdist", "python_version": "source", "requires_python": null, "size": 9544, "upload_time": "2016-10-03T00:30:12", "url": "https://files.pythonhosted.org/packages/f5/37/b609912784be69876beb1f386faf7bd5bcaa3d387f226c69267513dd0b49/liten-0.1.6.tar.gz" } ] }